I agree, as a long time Business Intelligence developer I‘m still confused and astounded with all the tooling and bits and pieces seemingly necessary to create analytics/dashboards with open source tools.
For years I used a proprietary solution like Qlik Sense for the whole journey from data extraction to a finished dashboard (mostly on-prem). Going from raw data to a finished dashboard is a matter of days (not weeks/month) with one single tool (and maybe some scripts for supporting tasks). There is some „scripting“ involved for loading and transforming data, but if you already understand data models (and maybe have some sql experience) it is very easy. The Dashboard creation itself does not need any coding at all.just drag and drop and some formulas like sum(amount).
But this a standalone tool and it is hard to integrate it into your own piece of software. From my experience, software developers have a much more complicated view on data handling. Often this is just the complexity of their use cases, sometimes it is just a lack of knowledge of data preparation for analytics use cases.
Another part which complicates stuff greatly is the focus on use-cases involving cloud storage and doing all the transformations on distributed systems.
And it is often not clear what amount of data we are talking about and if it is realtime (streaming) data or not. There is a big difference in the possible approaches, if you have 6h hours to prepare data or if it has to be refreshed every second (or when new data arrives etc).
Long story short: Yes it is complicated to grasp. There is also a big difference if you use the data for normal analytics use cases in a company (mostly read only data models) or if you use the data in a (big tech) product.
I would suggest to start simple by looking into a „query engine“ to extract some data from somewhere and then doing some transformations with pandas/polars/cubejs for basic understanding. You will need some schedulers and orchestration on the way forward. But this will be dependent on the real use cases and environment you are in.
The picture you are painting is way too dark. And does not give a realistic picture.
A lot of what you say is true for doctors in their first 5-10 years into their career, when employed in a hospital.
This not true for doctors which reached a certain level like „oberarzt“ and above.
This is especially not true for doctors with their own „office“ (business).
Yeah people may cry, but normally it is very hard to bring a doctor to justice even when there are quite obvious mistakes or misconduct. They are very well protected, suing a doctor not seldom takes 10 years from start to verdict, with a lot of legal costs involved.
And last but not least, it is a very secure profession. You must be really really stupid to end up jobless. So you have 5-10 years with a „ok“ salary compared to the power you invest. And 20-30 Years with a very good to exceptional salary, especially when compared to the broader population.
But in reality there is not an equal distribution between these 3 groups. And there is a high probability that the user base is not as limited as in your pseudo factual simplification. (journalists come to mind for example etc. pp)
Don’t you think it is strange to post this here, when the expected content about „data engineering“ is not (yet) there?
The only content i could find is an introduction to rust which is in my opinion not necessary, you could just reference the existing resources. When I saw that a category like „First data pipeline“ is empty, i closed the page.
Maybe I’m a bit old school, but it seems to me, that you care more about subscribers than delivering useful content.
Idk...throwing around big numbers, implying confidence does not convince me. Also your numbers are mostly not neccessarily related to the amount of workforce needed.
For example my blog/website operates theoretically in all countries on the world ...still it's only me needed. I think you get the point.
I think a measure like active users is more related to the size than number of countries and coins/tokens ...
But it is true I focused a too much on the platform itself etc. They have a lot stuff going on around it for legal, kyc, regulations, Marketing etc ... The operation of the platform itself is probably not the biggest part of their business anymore.
But coinbase for example, does it with less than 5000 ... But probably not so easy to compare
The article saysthat they had 8000 before layoffs started... What are all these people doing at an exchange? Maybe they work with orders on paper in the background? :D
If this platform fails, than it is because of the name.
I think the name "threads" is the worst you can come up with for an app that will be used world wide.
Outside of the U.S. there is a lot which can go wrong in spelling and pronunciation. maybe inside too, idk ;-)
1. threads is very similar to threats (the word threat is much more common)
2. pronunciation of "th" is in a lot of languages a complicated thing, but "thr" is even worse
3. I guess for a lot of people outside the U.S. it is not clear if it is pronounced like in "treat"
4. It is not a very common word my experience
From a company like meta I would expect more thoughtful product naming.
Wikipedia link does not work
I see a lot of claims in your article without any examples etc. to prove these claims.
It's hard for me to distinguish what is factual and what is "only" evangelism ... Additionally it's probably a lack of knowledge on my side, since I only worked with git. So examples would be very helpful to make this topic more accessable.
Is there a open source framework/application in go or rust based on FIX protocol?
In forex quite a lot broker offer FIX apis(because MT4 is using FIX).
The use case would be to have only one application to watch/place orders simultanously on 2-3(...multiple) broker accounts. To spread risk of losing funds in case a broker goes bankrupt (or whatever reason).