Ask HN: What do data consultancies / data teams do?
When I look at some like Mammoth Growth, I see they have lots of big clients. But what do they do? E.g. Dropbox as one of the clients. Doesn't they have engineering team that can build their data needs? Or what is the "data stack"? How does the whole process look like? Im genuinely curious.
2 comments
It’s kind of the same question as to why do companies like meta have their own data engineering and analytics org. Can’t software engineers do that?
In my experience they can but not well. They could acquire the skills but it would take time and more importantly time away from what they are doing. Also, the software engineers working on applications don’t want to create and manage pipelines, manage access controls, data governance, create dashboards etc.
Some observations from first hand experience working as a data engineer at meta and working closely with software engineers - client and s ever side.
- their sql is not great. I was surprised by many engineers not knowing about grouping sets etc.
- very limited understanding of how to build analytical data marts and warehouses
- no experience with concepts of master data management etc. and data governance best practices as a whole
- not good ad visuallybrwpresenting information via charts. So poor dashboarding skills and often have to learn the technology.
- have no experience with etl patterns so end up with pipelines that don’t scale.
- they don’t have any interest in learning data engineering for analytics type of work.
- have no experience with building growth accounting and retention datasets.
So for mature organizations where data is an asset you want to do it well so you either need a specialized team in house or outsource it to a consulting company in your question.
In my experience they can but not well. They could acquire the skills but it would take time and more importantly time away from what they are doing. Also, the software engineers working on applications don’t want to create and manage pipelines, manage access controls, data governance, create dashboards etc.
Some observations from first hand experience working as a data engineer at meta and working closely with software engineers - client and s ever side.
- their sql is not great. I was surprised by many engineers not knowing about grouping sets etc.
- very limited understanding of how to build analytical data marts and warehouses
- no experience with concepts of master data management etc. and data governance best practices as a whole
- not good ad visuallybrwpresenting information via charts. So poor dashboarding skills and often have to learn the technology.
- have no experience with etl patterns so end up with pipelines that don’t scale.
- they don’t have any interest in learning data engineering for analytics type of work.
- have no experience with building growth accounting and retention datasets.
So for mature organizations where data is an asset you want to do it well so you either need a specialized team in house or outsource it to a consulting company in your question.
thanks