How to progress as a distributed systems engineer?
I wanna know about the career progression and day to day work of a distributed systems engineer.
How do I get into it? Where do I start? What's the day to day work like?
15 comments
IMO if you're working on any non-trivial backend project, then you're already a "distributed systems engineer". Distributed systems problems are everywhere in the software world if you need to manage state and consistency across more than a single machine.
If you're asking how to work on interesting distributed systems, then I'd recommend trying to work at a tech company that is either large in and of itself (i.e. FAANG), or one that makes it's money solving distributed systems problems for others (i.e. Hashicorp, Snowflake, ...).
Either way, you can get a lot of value reading famous distributed systems papers, and building your own software projects that replicate the building blocks of these systems. For example: DNS, Gossip Failure Detector, Consensus (e.g. Raft), SQL Database, etc...
If you're asking how to work on interesting distributed systems, then I'd recommend trying to work at a tech company that is either large in and of itself (i.e. FAANG), or one that makes it's money solving distributed systems problems for others (i.e. Hashicorp, Snowflake, ...).
Either way, you can get a lot of value reading famous distributed systems papers, and building your own software projects that replicate the building blocks of these systems. For example: DNS, Gossip Failure Detector, Consensus (e.g. Raft), SQL Database, etc...
Hi . Thanks for the reply!!
For theoretical foundations, I'd recommend reading about the FLP result (https://dl.acm.org/doi/pdf/10.1145/3149.214121), the reduction of the state machine replication problem to consensus (https://dl.acm.org/doi/pdf/10.1145/78969.78972) and the different consistency criteria.
To climb on the shoulders of giants you can dive into the work of (incomplete list, irrelevant order but these have been eye openers to me) :
- Leslie Lamport
- Maurice Herlihy
- Eli Gafni
- Rachid Guerraoui
- Michel Raynal
- Petr Kuznetsov
- Hagit Attiya
- Sergio Rajsbaum
- Nancy Lynch
To climb on the shoulders of giants you can dive into the work of (incomplete list, irrelevant order but these have been eye openers to me) :
- Leslie Lamport
- Maurice Herlihy
- Eli Gafni
- Rachid Guerraoui
- Michel Raynal
- Petr Kuznetsov
- Hagit Attiya
- Sergio Rajsbaum
- Nancy Lynch
Hi. Thanks for the list and the resources. It's really helpful!!
This seems like a pretty generic question, so I assume you're looking at it from a new grad standpoint. Distributed Systems is more of a technical/resource specialty. You will be optimizing data across multiple systems. You'll want to measure it, improve overall performance/size/cost through algorithm and heuristic optimizations. It can be a terminal career state, meaning that plenty of people have a great career ending with the title: Distributed Systems Engineer. The progression within this career is similar to other Software Engineering roles. You are taking on larger projects, more team coordination, more business critical or technical complex problems.
I see. Thanks for telling me the terminal career state .
I've been working my way through MIT 6.824 Distributed Systems on YouTube (https://youtube.com/playlist?list=PLrw6a1wE39_tb2fErI4-WkMbs...)
It's a great survey of the basic principles and I've already found it useful in some projects I've been working on.
It's a great survey of the basic principles and I've already found it useful in some projects I've been working on.
Before we hire any engineer they must be licensed and bonded.
The iPhone must be an illegal fly-by-night operation then, since the electrical engineers hired to design it have no such requirement.
https://jobs.apple.com/en-us/details/200343080/electrical-en...
https://jobs.apple.com/en-us/details/200343080/electrical-en...
That job is actually for an electronics engineer, as it would be known where I live.
Electrical engineering is more specifically about aspects related to electrical power and often involves lethal voltages and potential to kill both workers and members of the public due to failure of engineering (directly).
Not sure of the licensing arrangements there exactly, but I do know there is statutory issues around following codes/standards as an EE in all developed contries I know of, designing electronics running at 5V, not so much.
It is common for EE to work in other fields such electronics, communications, computers and so on, other way around is not common at all.
Electrical engineering is more specifically about aspects related to electrical power and often involves lethal voltages and potential to kill both workers and members of the public due to failure of engineering (directly).
Not sure of the licensing arrangements there exactly, but I do know there is statutory issues around following codes/standards as an EE in all developed contries I know of, designing electronics running at 5V, not so much.
It is common for EE to work in other fields such electronics, communications, computers and so on, other way around is not common at all.
They're simply not engineers if they're not licensed and bonded. Just "grads."
I'm not in Texas, so an engineering degree is enough to call myself an engineer[1]. I don't need to be licensed and insured to operate a locomotive.
[1] although really, that's just me being credentialist: doing engineering is enough to justify calling oneself an engineer.
[1] although really, that's just me being credentialist: doing engineering is enough to justify calling oneself an engineer.
Coding isn’t engineering. Neither is playing with electronics even if you manage to get a product to market. In some countries you’ll go to jail for impersonating an engineer. Used to be like that in the US but of course, that changed and now quite literally everybody is an engineer if you squint now.
Do you think the field of electrical engineering is mistaken about its own name? As far as I know it is not common practice for people working in this field to be licensed and bonded, yet I have not heard another name for it.
Actually yes, see my post further up.