Ask HN: What are some good projects/resources to learn distributed systems?

54 pointsby gyani958 yıl önce14 comments

I'm a kind of person who likes to hack on projects to learn things. If I am learning a new language, I'll think of a problem that I want to solve. After reading some syntax I'll start hacking on a solution.

I have decided to learn distributed systems.

I think this weekend would be a good start to hack on things. I can continue later in the week I understand I won't pickup distributed systems over a weekend.

What resources do you recommend for one trying to learn distributed systems on their own?

14 comments

jxub8 yıl önce

I'd recommend the book Designing Data Intensive Applications, by Martin Kleppmann and as for online resources, check out http://book.mixu.net/distsys/single-page.html for a throughout overview.

gyani958 yıl önce

Thanks for the suggestion. I've added it to my list. Do you recommend some good first apps for learning distributed systems?

daoran8 yıl önce

Beside Martin Kleppmann's wonderful book, I'd also recommend MIT 6.824 distributed system course, you can find lots of resources on the course website.

cottonseed8 yıl önce

You might find some of these references useful:

Resources for Getting Started with Distributed Systems https://caitiem.com/2017/09/07/getting-started-with-distribu...

Distributed systems theory for the distributed systems engineer http://www.the-paper-trail.org/post/2014-08-09-distributed-s...

Distributed Systems for Fun and Profit http://book.mixu.net/distsys/

Notes on the Theory of Distributed Systems http://cs-www.cs.yale.edu/homes/aspnes/classes/465/notes.pdf

simon_acca8 yıl önce

Aphyr, of Jepsen fame, published an outline for a distributed systems class that makes for a good reference on what is relevant for most modern distributed systems: https://github.com/aphyr/distsys-class

gyani958 yıl önce

Thanks for this! This sounds lovely and is 12-16 hours long. Fits my schedule :D

iampims8 yıl önce

This Github repo is a pretty good start to get familiar with distributed systems:

https://github.com/donnemartin/system-design-primer

gyani958 yıl önce

I've been through this. It's a good starting point and can be fairly useful for interviews. It gives a good high level overview of systems, enough for you to talk about them.

Maybe I can go through this again and dive deeper into topics that interest me.

convolvatron8 yıl önce

do you want to learn how to design distributed systems from first principles or build systems from pre-existing distributed components?

unfortunately these are both called 'distributed systems'

#1 is alot more interesting. #2 is what people are hiring for

gyani958 yıl önce

I'd like to say both! I guess I can get started with the second one and then move onto the first one?

Maybe after playing with a few components I can look into their internals.

dsnuh8 yıl önce

For #2 - Maybe try setting up your own Kubernetes cluster on a cloud service provider, without using shortcuts? Perhaps etcd and the Raft protocol would be interesting and helpful in some cases, but leans more towards #1.

For #1 - CAP Theorem, read through aphyr's blog, google concepts/details/jargon you find in white papers on different of the various distributed consensus protocols, etc.

lambda_func8 yıl önce

Some stuff you can build to learn more about distributed systems (you can start simple, do research, and then iterate to build a more comprehensive solution): build a distributed file system, a distributed lock server, a consensus algorithm (paxos, raft, etc.), two/three phase commit protocol, distributed hash table, etc.

srkiranraj8 yıl önce

MapReduce paper, though published in 2004, it is foundational, relevant and very well written paper.

https://ai.google/research/pubs/pub62

P.S: it has close to 25k citations.

brudgers8 yıl önce

Erlang?