I will go a step further by stating that metrics, logs and traces are very similar and should be treated as such in a unique platform. Leveraging these 3 sources of data in a micro-services world is more than needed for troubleshooting, documentation and monitoring.
Right now I'm using Prometheus (metrics) + Jaeger (traces) + Fluentd&Clickhouse (logs) + Grafana to render all of that. It's not that easy to correlate data but I'm getting there (with tricky queries in Grafana panels and custom Grafana sources). A PoC about displaying traces in a nice way: https://github.com/alexvaut/OpenTracingDiagram.
Thanks for the links, so from Bruce Schneier, the problem needs to be taken care of by citizens (politic) like it was done for many industries (car, food, pharma...). Except that this is going to be much more complex in the information era where everything is a computer. Hence there is a need to have tech people in the public sector to help decisions to be wisely made. Enforcing the rules is the only way to make the industry to change, in this case, in terms of security and privacy.
However I tend to think I have more power as a consumer than as a citizen. I spent dollars everyday while I vote every 2 years. It seems that since there is no other way, the last resort is to go through the political way. I'm happy we have governments but still, I'm convinced there is a way to convince consumer. Do you ?
It makes me think that there are, at least, 2 ways to move forward:
- What can we do about this situation we are in ? Is it a problem that Technology can solve (I'm thinking about startup in the privacy field) or it's more political and in this case it will take years to fix.
- What can we do about the other fields where we still have some power ? Like Smart Assistant, self driving cars with AI. Someday we are going to wake up again and realize that again someone used one our of weakness and abused it.
It will again return against ourselves by restraining our freedom and/or make us more dumb.
I'm sure history has many examples about that global behavior: "change for the worst, acknowledge it, repeat".
What is the way to avoid taking that direction again ? I'm not sure education is the answer nor politic or technology... I'm out of answers...
To be honest I don't see the majority of developers interested into Category Theory. We have to hand it on a silver plate by solving some problems to show it is useful. It doesn't sound pragmatic enough for people to even start looking at it.
One way I'm thinking is to extract "patterns" automatically from their code. Then, it enables to write code review, give advises, pointers, find pattern duplication, common ground vocabulary... I'm sure structuring automatically the code from CT point of view can be helpful.
Disclaimer:
Few years ago, I felt in love with theory category and more precisely the sketches (from Ehresmann). I linked Machine Learning and Category Theory [1] by automatically mapping data structure and algorithm definitions together (input/output and operations between) in order to be able to run any algorithm on any set of data. Then I introduced an heuristic based on Kolmogorov complexity to find the best model (algorithm output) to summarize the input data. Loved it !
Agreed 100%, I worked with legacy code at different stages: 1 to 2 years: very little of structure but still velocity matters a lot, complexity is not very high; 2 to 5 years: first customers. copy/paste designs and complexity arise; 5 to 10 years: I observed 2 different trends, software where refactoring happened, product is still not in a very good shape but it is still ok and the others, the garbage product with millions of lines: global variables, If/then/else copy/paste design, doesn't scale, broken everytime for no obvious reasons. It's going to take years to stabilise without too much churn. So everytime I'm working on a project, I pledge managers, devs to allocate time to refactor and follow those patterns, implement unit tests etc... everytime at whatever stage, after sometime the team realizes the benefit. So yes, without a doubt, good design patterns are keys to make our dev lifes as easy as possible. And the SOLID ones are definitely my favourites.
What about returning refillable standard cups ? You return them dirty, starbucks/mac do wash them and everytime you want a coffee you get a clean one. It existed in France for glass bottles in the last century, it's still like that in Belgium for beers (and works very well). I don't see any problem to apply it for coffee. You can even think of little stickers of the brand from where you get your coffee, those stickers will be removed in the washing process. So if the big names can agree on few cup design/material and a washing process/logistic, they will have a very durable cup!