Data Engineer Study Guide(docs.google.com)
docs.google.com
Data Engineer Study Guide
https://docs.google.com/spreadsheets/d/1GOO4s1NcxCR8a44F0XnsErz5rYDxNbHAHznu4pJMRkw/edit#gid=0
7 comments
Thanks, I've been looking for something like this! :)
Hope it helps!
Also we are working on a software engineering, data science and hadoop focused one!
Let us know if there are specific topics you would like to see!
Also we are working on a software engineering, data science and hadoop focused one!
Let us know if there are specific topics you would like to see!
It would be useful to see resources about streams (and stream processing) and data lakes, specifically how they compare to data warehouses.
I work as a data scientist but want to learn more about data engineering and software engineering as I prefer those aspects of the work.
Hadoop seems outdated now though as we have moved to the Cloud whereas we used to have Hadoop on-site and it seems most of my friends companies have done the same.
Hadoop seems outdated now though as we have moved to the Cloud whereas we used to have Hadoop on-site and it seems most of my friends companies have done the same.
Awesome! Then we will work on that. Yes, Hadoop is kind of weird. Like most companies use it through the abstraction of hive or presto. So..it's just SQL again.
My two cents is there's still a lot of design patterns and methodologies to pick up on in MapReduce and distributed data architecture.
If we're including movement and integration, ETL/ELT, sharding, partitioning, etc in data engineering...?
If we're including movement and integration, ETL/ELT, sharding, partitioning, etc in data engineering...?
Will definitely submit feedback if I feel like something huge is missing.
Thanks for posting this! Hugely helpful since I've gone from software person who did a lot of data work to full on data engineer.