Software Engineering Insights from 10 Years at Google(addyosmani.com)
addyosmani.com
Software Engineering Insights from 10 Years at Google
https://addyosmani.com/blog/software-eng-10-years/
150 comments
It's somebody's blog post that's passing by and will be gone tomorrow. I don't think we have to worry about it being overly "elevated".
On the other hand we have a bunch of people spending too much time on hacker news, often with all the insight of someone who spent 5 minutes scanning a wikipedia article. I'd worry a lot more about the long term effects of elevating this self congratulatory content, but where's this content cop then?
> It may seem harsh
Honestly it seems like it's trying too hard. Call it feel good vagueries and move on.
On the other hand we have a bunch of people spending too much time on hacker news, often with all the insight of someone who spent 5 minutes scanning a wikipedia article. I'd worry a lot more about the long term effects of elevating this self congratulatory content, but where's this content cop then?
> It may seem harsh
Honestly it seems like it's trying too hard. Call it feel good vagueries and move on.
> somebody's
Addy is pretty well known in the frontend community. He's published a bunch of books, has hosted the Totally tooling tips podcast, gave a bunch of talks on performance, and has appeared in popular frontend publications, such as the Smashing Magazine.
Addy is pretty well known in the frontend community. He's published a bunch of books, has hosted the Totally tooling tips podcast, gave a bunch of talks on performance, and has appeared in popular frontend publications, such as the Smashing Magazine.
[deleted]
Blog is already inaccessible at this moment.
I don't know what this post is about (it is still down). Looks like an issue with GitPage. I had few not-so-important static websites hosted on GitHub Pages but gave up with such similar downtime. It was pretty noticeable even for my lame websites including my own personal website.
Anyways, Addy Osmani did contribute or did talked a lot about Frontend engineering development, especially wrt to Google Chrome. Whenever I think of Lighthouse, or try to debug or get my team to debug with Google Inspector, I remember him. I have read a lot of his articles, public tweets (I think) where I have exclaimed "oh! That's how", "ah! that is cool."
Anyways, Addy Osmani did contribute or did talked a lot about Frontend engineering development, especially wrt to Google Chrome. Whenever I think of Lighthouse, or try to debug or get my team to debug with Google Inspector, I remember him. I have read a lot of his articles, public tweets (I think) where I have exclaimed "oh! That's how", "ah! that is cool."
Looks like he is looking for a job at Amazon. All the right buzzwords
It's not that what he says is bad. It's that it reads like it was censored by Google's PR department, or self-censored to avoid trouble. There's nothing on "what to do when your project is cancelled", or "what to do when what you're doing is becoming incompatible with what someone else on the project is doing", or "when is there too much technical debt to continue". Those are things one might learn in a decade at Google, but can't talk about in public.
I mean, you might learn these, but also you might not. Some people just have generally good experiences (and in fact you'd sort of expect someone who has stayed at the company for a decade to be in that bucket).
Signed, another googler who couldn't answer any of those questions despite half a decade.
Signed, another googler who couldn't answer any of those questions despite half a decade.
I’ve been at google for a decade. Zero of my projects have been cancelled. I’ve never had my work be incompatible with the work of others.
Google publicly cancels a lot of products. It has a pretty bad long term product strategy. But part of that is because it has like 100000 engineers. Most people aren’t experiencing political problems.
Google publicly cancels a lot of products. It has a pretty bad long term product strategy. But part of that is because it has like 100000 engineers. Most people aren’t experiencing political problems.
Then, if Googlers don't have experience in solving the problems that other engineers may face, then what useful insights would even be available there?
Dealing with political problems, technical debt, and changing product requirements are all core parts of navigating the software industry.
Dealing with political problems, technical debt, and changing product requirements are all core parts of navigating the software industry.
Political problems, technical debt, and changing product requirements are not the same thing as what was mentioned above.
On the contrary, I think this post is exactly what it describes - 10 years at Google. Whether you take that as a good or bad thing is up to you and it seems like you'd probably not enjoy working at Google.
To be honest, whenever I see or hear the word "insights" now, I become very skeptical. It stems from when someone goes on stage at a TED talk (or something similar, perhaps even a 'tech talk') and spews a bunch of already well known high level nonsense that is not useful at all in our daily lives and work. Then, when it's Q&A time, a sycophant in the audience will predictably begin a question with: "thanks for your insights".
Software Engineers are not Engineers they are Professional Musicians. They need their Algorithms and Data Structures knowledge, the same way a Jazz musician needs the circle of fifths and a deep understanding of Harmony, to do sophisticated improvisation over a Jazz standard.
If you need to go on tour with your rock band you don't go to a Music conservatory and start interviewing musicians based on their knowledge of scales. You give them a Guitar, or Drums or Bass and you ask them to play something. You will know right away if they are the Guitar player or Drummer you are looking for.
That is why Software Engineering hiring is still broken, and why companies need to keep buying other companies to acquire successful software products. And that is why nobody cracked yet the challenge of successful software development.
Some of the best Software Developers are self-taught, and that include in this category, the ones with a different type of degree, that moved into Software Development.
You don't go to a school and come out an Eric Clapton or a Jimi Hendrix. You either have it or not. It's the same for Software Engineers and Software Development. No amount of schooling will help you if "don't have it" and no lack of schooling will stop you from developing something and share it with the world.
If you need to go on tour with your rock band you don't go to a Music conservatory and start interviewing musicians based on their knowledge of scales. You give them a Guitar, or Drums or Bass and you ask them to play something. You will know right away if they are the Guitar player or Drummer you are looking for.
That is why Software Engineering hiring is still broken, and why companies need to keep buying other companies to acquire successful software products. And that is why nobody cracked yet the challenge of successful software development.
Some of the best Software Developers are self-taught, and that include in this category, the ones with a different type of degree, that moved into Software Development.
You don't go to a school and come out an Eric Clapton or a Jimi Hendrix. You either have it or not. It's the same for Software Engineers and Software Development. No amount of schooling will help you if "don't have it" and no lack of schooling will stop you from developing something and share it with the world.
> You give them a Guitar, or Drums or Bass and you ask them to play something. You will know right away if they are the Guitar player or Drummer you are looking for.
The equivalent of this in software engineering is to give them a coding problem and asking them to write code to solve it and test it, which does happen in interviews.
The theoretical concepts are also often discussed in technical interviews. This is useful as it proves the ability to have a discussion with other engineers without getting bogged down in fundamental misunderstandings. If someone doesn't know what an array or a hash table is, that's going to generate difficulties for collaboration.
I think an orchestra is a better analogy than a small music band, when it comes to big companies. You wouldn't hire a violinist who can't read music or understand all the theory required to interpret written music.
The equivalent of this in software engineering is to give them a coding problem and asking them to write code to solve it and test it, which does happen in interviews.
The theoretical concepts are also often discussed in technical interviews. This is useful as it proves the ability to have a discussion with other engineers without getting bogged down in fundamental misunderstandings. If someone doesn't know what an array or a hash table is, that's going to generate difficulties for collaboration.
I think an orchestra is a better analogy than a small music band, when it comes to big companies. You wouldn't hire a violinist who can't read music or understand all the theory required to interpret written music.
I agree that if dont know what array or a hash table is collaboration and a common language are difficult. But the type of self-taught programmer I am thinking about is the one that probably knows all about hash tables and arrays because they were researching the best solution. The same way Eddie Van Halen knew everything about Guitar Electronics and Tube Amps.
I can't agree with any of these perspectives, or the analogy to being a musician.
Also, I find it to contradicts itself.
Initially, you say: you need the core knowledge of algorithms and data structures (like the circle of fifths and harmony) to do sophisticated improvisation.. so you're advocating for the importance of having a strong technical background. OK, fair.
But I don't get your analogy to a rock band going on tour, and they either have it or they don't? What is the software analogy to going on tour? Is this working in a startup, when you just need results rather than finesse? How does this tie back to your initial point about jazz musicians needing a core base of knowledge?
You also bring up the myth of the self-taught, scrappy musician (or programmer) who, against all odds,'has it' and is able to exceed the abilities of others who are the hardest-working and most studious.
I find this concept pretty toxic. The Eric Claptons and the Jimi Hendrixs of the world are a rarity, a complete exception. Although often cited because they are incredibly visible, compared to the hundreds of thousand if not millions of others who either developed skills and capabilities through education and/or self-taught effort. Often without 'having it'. I think it can be pretty discouraging when everyone holds themselves up to ideal of Claptons and Hendrixs.. rather than considering there might be multiple, varied and nuanced paths to being an excellent engineer, or musician.
Initially, you say: you need the core knowledge of algorithms and data structures (like the circle of fifths and harmony) to do sophisticated improvisation.. so you're advocating for the importance of having a strong technical background. OK, fair.
But I don't get your analogy to a rock band going on tour, and they either have it or they don't? What is the software analogy to going on tour? Is this working in a startup, when you just need results rather than finesse? How does this tie back to your initial point about jazz musicians needing a core base of knowledge?
You also bring up the myth of the self-taught, scrappy musician (or programmer) who, against all odds,'has it' and is able to exceed the abilities of others who are the hardest-working and most studious.
I find this concept pretty toxic. The Eric Claptons and the Jimi Hendrixs of the world are a rarity, a complete exception. Although often cited because they are incredibly visible, compared to the hundreds of thousand if not millions of others who either developed skills and capabilities through education and/or self-taught effort. Often without 'having it'. I think it can be pretty discouraging when everyone holds themselves up to ideal of Claptons and Hendrixs.. rather than considering there might be multiple, varied and nuanced paths to being an excellent engineer, or musician.
I mean that learning everything about Music theory can not guarantee you will ever be able to have the natural skills required to be a professional musician. And you give a guitar to a few adolescents, and in a few weeks some will be at the same stage while others while come back to you playing most songs they hear on the radio.
What I think is toxic is to make many believe they can get a Computer Science education and become a professional Software Developer. The same way finishing a five year Music degree might still mean somebody is terrible at the Piano.
What I think is toxic is to make many believe they can get a Computer Science education and become a professional Software Developer. The same way finishing a five year Music degree might still mean somebody is terrible at the Piano.
Without leet coding, named algorithms (binary search, dynamic programming, and other esoterica) or data structures (only requiring knowledge of arrays, assoc arrays, loops is enough), coding interviews still allow you to demonstrate your thinking ability.
You can construct various simple problems (that require only loops and arrays) that can easily be mapped to real world problems and see how well the programmers perform (problems: that exploit the fact that input is sorted, linear search for min/max, collect data -> loopy state transition -> output state).
A fictional scenario that you can map to a real software problem, there's 7 cities, N people (program input) in city 0. 6 buses, one goes from city 0 to city 1, other from 1 to 2, ... last from 5 to 6.
Bus has a limited capacity (6 additional program inputs). Everyone needs to be at city 6. Time starts at 0. One time tick: all buses go from city X to city (X+1), unload people at city (X+1), and then go back to city X. What's the minimum time it takes for everyone to be at city 6?
There's 7 inputs (N people and bus capacities) and there's multiple programs one can write to solve the problem. There's no math tricks, no data structures.
You can write a program whose time complexity depends on the size of the inputs O(N), or just on the number of inputs O(7), or maybe something worse or in-between.
I have no idea if there is a way for one to teach/learn how to write the simplest program as soon as possible, but some will find it in 1 minute, some in 60 minutes. The amount of time it takes might be irrelevant and the only thing you're looking for is the ability to eventually think simple enough. And the thinking bit is not the only filter. The person might struggle to use the computer (but uses it well enough for these simple problems), might have no idea about the machine that runs the program, etc.
You can construct various simple problems (that require only loops and arrays) that can easily be mapped to real world problems and see how well the programmers perform (problems: that exploit the fact that input is sorted, linear search for min/max, collect data -> loopy state transition -> output state).
A fictional scenario that you can map to a real software problem, there's 7 cities, N people (program input) in city 0. 6 buses, one goes from city 0 to city 1, other from 1 to 2, ... last from 5 to 6.
Bus has a limited capacity (6 additional program inputs). Everyone needs to be at city 6. Time starts at 0. One time tick: all buses go from city X to city (X+1), unload people at city (X+1), and then go back to city X. What's the minimum time it takes for everyone to be at city 6?
There's 7 inputs (N people and bus capacities) and there's multiple programs one can write to solve the problem. There's no math tricks, no data structures.
You can write a program whose time complexity depends on the size of the inputs O(N), or just on the number of inputs O(7), or maybe something worse or in-between.
I have no idea if there is a way for one to teach/learn how to write the simplest program as soon as possible, but some will find it in 1 minute, some in 60 minutes. The amount of time it takes might be irrelevant and the only thing you're looking for is the ability to eventually think simple enough. And the thinking bit is not the only filter. The person might struggle to use the computer (but uses it well enough for these simple problems), might have no idea about the machine that runs the program, etc.
You see but here is the thing...
Let's say I am interviewing, for their corresponding roles, one of the Aerospace Engineers responsible for the design of the latest Airbus, or a specialized heart surgeon with 10 years of experience, or a Civil Engineer that lead 5 year projects building bridges crossed daily by thousands of persons.
What kind of look would they give me....If I start throwing them the type of mental puzzles like that, that you typically find on the back pages of newspapers on dentists waiting rooms:-)
Let's say I am interviewing, for their corresponding roles, one of the Aerospace Engineers responsible for the design of the latest Airbus, or a specialized heart surgeon with 10 years of experience, or a Civil Engineer that lead 5 year projects building bridges crossed daily by thousands of persons.
What kind of look would they give me....If I start throwing them the type of mental puzzles like that, that you typically find on the back pages of newspapers on dentists waiting rooms:-)
But that is not a mental puzzle. It's a straightforward march of thinking.
Programming as a craft, generally, does not have a "great filter" like medicine, civil engineering or aerospace. The longer one survives in these fields, there's a really small chance of surviving out of luck and not out of experience.
If you look at carpentry instead, the mental part of the craft is also one thing you can test. For example, you might provide a limited set of tools (like arrays and loops) and require a solution to a carpentry problem. The mental part is not tested by puzzles (by puzzle I assume there's a set of tricks you need to know, or in the case of programming, a particular algorithm), but by problems. Problems might be more concrete for carpentry.
The problem above with buses can be reframed into whatever the area of programming expertise is for a candidate. There's a simple underlying structure in the problem that can appear in web dev, networking, databases, that maps directly to that bus problem. If you cannot figure out the buses, then how will figuring it out for something else be simpler? The solution is just loops and arrays, there's no fancy algorithms, there's no fancy math, it's pure art of programming.
Like I said, it's only one part that you're testing, which is thinking/problem solving. It can also be affected by personal anxieties or whatever. Failure at a particular problem does not mean much. But I would not say that these kinds of coding tests break hiring.
I mean, I hear people are being asked to pick if they're more hardworking or creative, or to describe where they see themselves in 10 years, or some other similar abstract silly stuff. I'd much rather someone asks me about buses.
Programming as a craft, generally, does not have a "great filter" like medicine, civil engineering or aerospace. The longer one survives in these fields, there's a really small chance of surviving out of luck and not out of experience.
If you look at carpentry instead, the mental part of the craft is also one thing you can test. For example, you might provide a limited set of tools (like arrays and loops) and require a solution to a carpentry problem. The mental part is not tested by puzzles (by puzzle I assume there's a set of tricks you need to know, or in the case of programming, a particular algorithm), but by problems. Problems might be more concrete for carpentry.
The problem above with buses can be reframed into whatever the area of programming expertise is for a candidate. There's a simple underlying structure in the problem that can appear in web dev, networking, databases, that maps directly to that bus problem. If you cannot figure out the buses, then how will figuring it out for something else be simpler? The solution is just loops and arrays, there's no fancy algorithms, there's no fancy math, it's pure art of programming.
Like I said, it's only one part that you're testing, which is thinking/problem solving. It can also be affected by personal anxieties or whatever. Failure at a particular problem does not mean much. But I would not say that these kinds of coding tests break hiring.
I mean, I hear people are being asked to pick if they're more hardworking or creative, or to describe where they see themselves in 10 years, or some other similar abstract silly stuff. I'd much rather someone asks me about buses.
Ok. Let's put your theory to the test by making this more concrete:-)
You are working as an Interviewer for a FAANG and the six candidates today for a SWE position are:
- Linus Torvalds
- John Carmack
- Fabrice Bellard
- Guido van Rossum
- James Gosling
- Donald Knuth
What "puzzle" are you proposing?
You are working as an Interviewer for a FAANG and the six candidates today for a SWE position are:
- Linus Torvalds
- John Carmack
- Fabrice Bellard
- Guido van Rossum
- James Gosling
- Donald Knuth
What "puzzle" are you proposing?
The proof of their experience is publicly visible. That is not the case for most individuals, including me.
FizzBuzz is not a puzzle solvable only with an obscure trick, and easily maps to real world, the buses problem too, and I can list more problems that are solved with 1-10 lines of code but the initial attempt used 20-50 because the structure of the problem was not fully exploited, unnecessary passes over the data, too much intermediate state etc. (all of the listed bloat appears in real world all of the time)
There are no tricks, no maths, no algorithms, no data structures involved. Your ability to pattern match these little bits should grow with your experience.
FizzBuzz is not a puzzle solvable only with an obscure trick, and easily maps to real world, the buses problem too, and I can list more problems that are solved with 1-10 lines of code but the initial attempt used 20-50 because the structure of the problem was not fully exploited, unnecessary passes over the data, too much intermediate state etc. (all of the listed bloat appears in real world all of the time)
There are no tricks, no maths, no algorithms, no data structures involved. Your ability to pattern match these little bits should grow with your experience.
If that is the case, why do experienced Software Engineers, have to spend again, weeks on LeetCode as soon as they decide to get back to the Interview circuit?
And why are so many competitive programmers,(not all) terrible Software Engineers?
https://kislayverma.com/organizations/competitive-programmin...
https://www.freecodecamp.org/news/mythbusting-competitive-pr...
And why are so many competitive programmers,(not all) terrible Software Engineers?
https://kislayverma.com/organizations/competitive-programmin...
https://www.freecodecamp.org/news/mythbusting-competitive-pr...
My first response focused on problems that do not require knowledge of any named algorithms, data structures, math or tricks. I even mentioned I do not care about heaps, linked lists etc. I'm not sure how we're back at LeetCode.
Of course you're going to have to spend weeks on LeetCode if interview questions target knowledge of dynamic programming, heaps etc.
You're preaching to the choir.
My point was that FizzBuzz might test individuals to see if they can code, but there are "harder" problems that happen on a daily basis, and the amount of years you put in will affect your ability to pattern match.
Competitive programming captures a part of the essence of programming, you're writing minimal, fast and efficient programs. You improve your pattern matching to minimize the amount of intermediate state, unnecessary loops etc. After some point, competitive nature pushes you to specialize, and then it probably stops making sense if you want to make programming your craft.
Of course you're going to have to spend weeks on LeetCode if interview questions target knowledge of dynamic programming, heaps etc.
You're preaching to the choir.
My point was that FizzBuzz might test individuals to see if they can code, but there are "harder" problems that happen on a daily basis, and the amount of years you put in will affect your ability to pattern match.
Competitive programming captures a part of the essence of programming, you're writing minimal, fast and efficient programs. You improve your pattern matching to minimize the amount of intermediate state, unnecessary loops etc. After some point, competitive nature pushes you to specialize, and then it probably stops making sense if you want to make programming your craft.
[deleted]
You throw some good arguments and although I am not in agreement with such methodology let me ask would you apply the same process when hiring a senior engineer with a demonstrated experience of N+ years in tech you're looking for?
Isn't that off putting and isn't that ingraining distrust from the very beginning?
Isn't that off putting and isn't that ingraining distrust from the very beginning?
It is just an attempt to shortly evaluate the experience listed on paper. Thinking is only one aspect of programming.
Are open form questions for which the candidate can give an honest answer or that start a discussion better, or do you find them similar to a simple coding problem? What if during discussion I probe for more details about one particular thing?
Just to be clear, I agree that coding questions can be ridiculous. These do appear in some interviews. Questions related to quicksort, radix sort, counting sort, heaps, Dijkstra, 0-1 bfs, dfs, iterative deepening, prime testing or integer factoring, or convoluted problems of counting the many ways of tiling a n x m grid with 2x1 and 1x1 tiles, can all be found in the wild.
My buses problem is completely different and simpler.
Are open form questions for which the candidate can give an honest answer or that start a discussion better, or do you find them similar to a simple coding problem? What if during discussion I probe for more details about one particular thing?
Just to be clear, I agree that coding questions can be ridiculous. These do appear in some interviews. Questions related to quicksort, radix sort, counting sort, heaps, Dijkstra, 0-1 bfs, dfs, iterative deepening, prime testing or integer factoring, or convoluted problems of counting the many ways of tiling a n x m grid with 2x1 and 1x1 tiles, can all be found in the wild.
My buses problem is completely different and simpler.
>>> just vague rewording of colloquial expressions commonly traded around
As I read through the article, I couldn’t figure out if it was me “hating” on the author or if it really read like a 99 cent store hallmark card.
Glad to see I wasn’t the only one who thought the latter.
As I read through the article, I couldn’t figure out if it was me “hating” on the author or if it really read like a 99 cent store hallmark card.
Glad to see I wasn’t the only one who thought the latter.
I could never rationalise technological evangelism outside of a self serving purpose. but maybe that says more about me.
Look, yes this is another post that seems to be written by an idiot but that’s presumably because their simply a bad at this type of communication. Writing for a mass audience is just really difficult to do well and this is just one of the common failure modes for technical people.
Even if you can’t find value in what was written just take it as a lesson in what not to do.
Even if you can’t find value in what was written just take it as a lesson in what not to do.
> because their simply a bad at this
you wrote this. about someone else's writing
you wrote this. about someone else's writing
Recognizing something is difficult doesn’t mean you are good at it.
[deleted]
Is there any evidence that Google is actually better at software engineering than other companies? IMHO, it seems they have enough money to deliberately do things inefficiently which would kill a smaller company, but Google can manage to succeed through brute force.
Honestly I think the better example would be from a tiny company that can't afford to do things wrong.
Honestly I think the better example would be from a tiny company that can't afford to do things wrong.
Maybe 10 years ago when they were pushing the envelope. Nowadays, I just don’t see it to be honest. There are a few brilliant engineers for sure, but the new generation are mostly Leetcoders that brag about TC on another social network. You can see it in their open source code too. Critical bugs not fixed for an entire year. You read the code and it doesn’t stand out as anything out of the ordinary. I get the feeling most are focused on promotion packages to raise their TC.
If you read what the majority of them recommend, its similar to the banker's path in the early 2000s. Graduate from Ivy League, spent 3 years at FANG, then 2 years at Ivy League MBA, then back to FANG or join a startup for a few years so you can get a title bump when you do go back to FANG. The type of people that join Google nowadays aren't the type of people that were early employees. They have a very standard charted course to their careers.
Google has reach maturity stage of the company. They have leeway to screw up like what Microsoft did with Windows and Office in the early 2000s. Will they have the ability to push out something completely new and amazing like Gmail or Google Maps (although Maps was a clone of Mapquest, they executed better), where the end users aren't developers themselves? I highly doubt it with an organization as big. There will be a lot of politics involved to get anything done. They have lost the talent to innovate.
If you read what the majority of them recommend, its similar to the banker's path in the early 2000s. Graduate from Ivy League, spent 3 years at FANG, then 2 years at Ivy League MBA, then back to FANG or join a startup for a few years so you can get a title bump when you do go back to FANG. The type of people that join Google nowadays aren't the type of people that were early employees. They have a very standard charted course to their careers.
Google has reach maturity stage of the company. They have leeway to screw up like what Microsoft did with Windows and Office in the early 2000s. Will they have the ability to push out something completely new and amazing like Gmail or Google Maps (although Maps was a clone of Mapquest, they executed better), where the end users aren't developers themselves? I highly doubt it with an organization as big. There will be a lot of politics involved to get anything done. They have lost the talent to innovate.
I work at Google, and previously at Amazon, I was a bit upset reading your comment, but I have to be honest with myself, you are right. I "admire" more indiehackers and other technical founders than Staff Engineers and the like at Google. On the tech side you are also correct, the innovation at Google is minimal, we all work on somebody else's projects, promotion work and career development. Regarding interviewing, you are also correct, it have become a Leetcode grind fest, that certain demographic nails more often than not, due to their ability to give up life and memorize Leetcode solutions and System Design patterns non stop. I'm jealous of people who find a real life problem and start a company to monetize a solution that actually help people.
What does TC mean in this context ?
Depends what you mean by "better at software engineering".
What Google seems to do really well is building robust systems at a scale that pushes the limits of what humanity is capable of. If you collect anecdotes from engineers who've worked on scaling systems at the small number of companies at this scale, there seems to be general agreement that Google is "the best". My understanding from these anecdotes is that Google core services are "on fire" less than everywhere else and require less maintenance.
But that's a pretty narrow definition of software engineering. AWS, by all accounts, is much better at productizing something useful to the world. I've heard that the reason Microsoft lags Google in stability could plausibly be accounted for by the fact that they care so much more about maintaining backwards compatibility (which Google very clearly does not care about).
All that said, I don't think its "best at software engineering" is a terribly useful category, better to say they are plausibly "best at a very specific and very difficult kind of software engineering".
What Google seems to do really well is building robust systems at a scale that pushes the limits of what humanity is capable of. If you collect anecdotes from engineers who've worked on scaling systems at the small number of companies at this scale, there seems to be general agreement that Google is "the best". My understanding from these anecdotes is that Google core services are "on fire" less than everywhere else and require less maintenance.
But that's a pretty narrow definition of software engineering. AWS, by all accounts, is much better at productizing something useful to the world. I've heard that the reason Microsoft lags Google in stability could plausibly be accounted for by the fact that they care so much more about maintaining backwards compatibility (which Google very clearly does not care about).
All that said, I don't think its "best at software engineering" is a terribly useful category, better to say they are plausibly "best at a very specific and very difficult kind of software engineering".
Google seems to have more than its fair share of good engineers, but the organization itself is not substantially different from other organizations.
Whether core services are "on fire" depends on which team you're on. Some core services are pretty much "on fire" all the time, with high pager loads for the SRE teams supporting them. Some core services are rock solid, and the SRE teams conduct exercises where they turn the service off just to remind people that the uptime SLO is not 100%.
Same goes for code quality. Some teams have code that you'd just love to print out and put the code listings on your desk. Some teams have code that doesn't make sense and nobody understands.
Some teams are highly functional, some teams are exceptionally dysfunctional.
Whether core services are "on fire" depends on which team you're on. Some core services are pretty much "on fire" all the time, with high pager loads for the SRE teams supporting them. Some core services are rock solid, and the SRE teams conduct exercises where they turn the service off just to remind people that the uptime SLO is not 100%.
Same goes for code quality. Some teams have code that you'd just love to print out and put the code listings on your desk. Some teams have code that doesn't make sense and nobody understands.
Some teams are highly functional, some teams are exceptionally dysfunctional.
"What Google seems to do really well is building robust systems at a scale that pushes the limits of what humanity is capable of"
Can you give some examples of things Google did which pushed the limits of what humanity was capable of?
Can you give some examples of things Google did which pushed the limits of what humanity was capable of?
I've never understood all the praise Netflix gets for technical excellence (not that they're bad) for running a video streaming site while Youtube does the same thing about as well with the added difficulty of taking in user submitted video with all the technical and legal troubles that entails. I'm far from one to praise Google, but Youtube is an amazing service.
[deleted]
no. youtube does it way better
Not OP, but doesn't availing any particular tiny slice of all knowledge on the web to you within ~1.5 seconds, by inferring your intent incredibly accurately from a few words, count as "building robust systems ... that push the envelopes of what humanity is capable of"?
Or, how about a video distribution network that serves more than 600,000 hours of video every minute, and manages to do that such that, every time service deteriorates on my 4G connection in a poor country, it's the last website (even considering text only websites) to stop working?
Or, how about a video distribution network that serves more than 600,000 hours of video every minute, and manages to do that such that, every time service deteriorates on my 4G connection in a poor country, it's the last website (even considering text only websites) to stop working?
Yeah, I guess before Google you did have to wait 5 whole seconds to get the same information, because of all the ads AltaVista spewed at you along with their search results. Progress.
As for YouTube, it was bought, not created by Google, wasn't it?
As for YouTube, it was bought, not created by Google, wasn't it?
As to AltaVista, I'm not old enough to have used it. But my guess is that its accuracy and speed was not higher than that of the best competitors to Google right now. And to me, no competitor comes close in matching Google's combination of accuracy and speed across many categories of search (I've tried bing/you.com/yahoo.com/duckduckgo so far).
As for youtube, serving video is not the impressive part. It's the scale at which they do it. And my understanding is that the scale has increased exponentially since the 2006 acquisition.
As for youtube, serving video is not the impressive part. It's the scale at which they do it. And my understanding is that the scale has increased exponentially since the 2006 acquisition.
I'm old enough to remember AltaVista and all of the others. There's a reason I stopped using them almost immediately when I found Google.
a typical search on altavista in its prime often took 30 seconds. Most of the results would be totally irrelevant.
Street view on maps / Google Earth. The end to end execution of capturing the images, making them available, keeping it updated. You can walk down the street in New York City without leaving your home —- it’s something out of sci-fi.
Depends what you mean by "better at software engineering".
I don't mind admitting Google employs people far smarter than I will ever be, but they're creating and maintaining the largest global spying apparatus in history. Maybe we should be learning from people who're actually making the world a better place.
They make you feel like they're smart but everything is relative. Google cloud is constantly the thrid in the market and significantly leg behind AWS in terms of market share while its CEO was replaced every 2 years so there are definitely smarter people (e.g. AWS engineers) out there.
Being third in the market is independent from the quality of services IMO. Whether the services are being offered that a lot of businesses need is a different question. The services Google Cloud does offer tend to be well designed but many of them are simply not what established businesses (and their engineering organizations) are accustomed to. Lower marketshare is not a direct consequence of engineering excellence (or an alleged lack thereof).
Conversely, simply having more market share doesn't mean AWS services are well engineered and the engineers are smarter (they might be, but you cannot make this claim).
Conversely, simply having more market share doesn't mean AWS services are well engineered and the engineers are smarter (they might be, but you cannot make this claim).
I totally agree the quality of a product may or may not be directly proportional to the market share (e.g. Windows). My previous comment has nothing to do with engineering excellence. I never work at Google nor an expert on Google cloud so I don't know whether Cloud Google is well-designed (and I'm not surprised if it's).
There're different ways to measure "smart". My view is that in the cloud computing business there are people smarter than them (at least in terms of market share) so it's perfectly fine that Google Cloud engineers learn from their competitors in terms of engineering, marketing, etc.
There're different ways to measure "smart". My view is that in the cloud computing business there are people smarter than them (at least in terms of market share) so it's perfectly fine that Google Cloud engineers learn from their competitors in terms of engineering, marketing, etc.
They are only smart in the context of what they are optimizing against. It’s like that for all area’s of an expertise or goal/ideal.
Teachers, nurses, and janitors
Trust me, they aren’t THAT smart. People I know who work at Google, let’s just say: they often fit a type.
There's a (semi)famous Googler in one particular area. I had a buddy who worked with him years ago at a different company and took over a project after he left. There was nothing written after nearly a year - the guy either couldn't or wouldn't code anything, and now he's a well known Googler. Obviously there are extremely talented people there, but there are some bozos mixed in too.
He probably couldn’t be bothered by such mundane work.
I know of the famous people that work there, and I don't have anything personal against them. My comment was more directed towards the company. I can't support the way Google and other data mining companies operate. Its such a waste of talent.
Some are. It's now a giant company and recently partnerships with google bought me into contact with very average developers, gluing things together.
[deleted]
No, Googlers in CPython core for example are very unspectacular. Some are extremely power hungry, do nothing but try to rule the project with the boot, using dirty politician's tactics.
I have only worked at a few companies, and I won't comment about the quality of engineering done at Google, but Google's Engineering Systems are the best I have ever used. From Code Search to Critique to Cider (Online IDE) to Blaze (Build) to having the entire code base "mounted" on my dev machine and the integration with golinks, it was an absolute pleasure to work with.
> From Code Search
What could have been... [1] Yes, I'm still bitter about it after all those years.
[1] https://en.wikipedia.org/wiki/Google_Code_Search
What could have been... [1] Yes, I'm still bitter about it after all those years.
[1] https://en.wikipedia.org/wiki/Google_Code_Search
> having the entire code base "mounted" on my dev machine
My impression is that this is a result of a policy that google3 never touches a developer machine, which of course means you can't disconnect and work from a log cabin in the woods. Really sounds like bending yourself around an antipattern to be honest.
My impression is that this is a result of a policy that google3 never touches a developer machine, which of course means you can't disconnect and work from a log cabin in the woods. Really sounds like bending yourself around an antipattern to be honest.
It's not your code, it's Google's and I don't see how this is an anti-pattern. It is actually one of the best thought out solutions to balancing source code supply chain security and productivity. The fact that the industry has moved towards git and allows employees to clone proprietary codebases onto developer machines anywhere in the world is pure insanity.
It’s also Google’s laptop but maybe they should have me remote into it instead of letting me touch it. Who knows, I might steal it…
To Google, a laptop is worthless compared to their intellectual property.
It's more the principle of the thing–if I wanted to steal code from Google their "no source code on laptops" policy wouldn't stop me at all. And given that I don't actually want to steal code the policy sucks, so the real question is what the benefit is here by treating me like a criminal.
It’s not that a google employee would steal the code, it’s that if a laptop is compromised the access to the source code can be immediately revoked. So if someone steals an employee’s laptop and coerces their password from them, the source code can not be accessed for long, and exactly what source code was seen is known.
Very few people are capable of doing engineering from a cabin in the woods even if they can mirror the entire code base on their machine since engineering is a collaborative effort.
I would hope that you are able to work on self-contained projects without having to interact with other people for at least a day or two at a time…
Who is driving out to a cabin in the woods to work for just one day?
Never taken a three day trip but want to get work done on Monday?
Not to a cabin in the woods with no internet.
This is becoming an increasingly remote (hah) scenario where doing development over web-based IDEs and SSH becomes a problem. For the incredibly few people who do this sort of thing regularly, you could get satellite internet reimbursed. There is even a story of a SRE who used to work from the top of mountains on occasion because he hiked so much.
This is becoming an increasingly remote (hah) scenario where doing development over web-based IDEs and SSH becomes a problem. For the incredibly few people who do this sort of thing regularly, you could get satellite internet reimbursed. There is even a story of a SRE who used to work from the top of mountains on occasion because he hiked so much.
Small sample size, but the Google/exGoogle engineers I’ve interviewed are much more capable on average than others (or are at least better at crushing our interview loop). YMMV.
If building, running and evolving very large and complex systems over many years is not a measure of software engineering quality, I don't know how else to compare companies.
Google has not pushed anything relevant to the user space for many, many years. It's not without reason people meme about Google Graveyard. Here in Hackernews, people constantly whine about their ads, their irrelevant google search, and what have you. Their latest "flagship" phone is littered with bugs.
I wouldn't call them "evolving" anything software as of late. Their hard science departments seem good though, but these aren't software engineering
I wouldn't call them "evolving" anything software as of late. Their hard science departments seem good though, but these aren't software engineering
TrueTime and spanner definitely fall in the wow category at least for me
Microsoft one of the worlds top software companies couldn’t keep up with chrome.
There are a lot of projects that look easy at a POC stage and are very different at million of users, millions of tps, millions of TB or full featured
There are a lot of projects that look easy at a POC stage and are very different at million of users, millions of tps, millions of TB or full featured
Chrome was released in 2008. Microsoft is doing fine with Edge, though it's a moot point because Microsoft is even older and more sluggish than Google.
Spanner seems a prototypical Google dev product though; it's only invented to serve Google. I don't think any of their solutions is the de facto choice in any tech stack.
Spanner seems a prototypical Google dev product though; it's only invented to serve Google. I don't think any of their solutions is the de facto choice in any tech stack.
Spanner is definitely the system I would want to use if I had $$$, ran on GCP, and needed to do large-scale distributed database processing.
It was invented to serve Google (really: the people who had outgrown bigtable's limited featureset) but is widely applicable.
It was invented to serve Google (really: the people who had outgrown bigtable's limited featureset) but is widely applicable.
Edge is just reskinned chrome browser. They discontinued their previous browser
> Google has not pushed anything relevant to the user space for many, many years.
80% of software engineering is not about pushing but maintaining. I'm not the one who praises Google (far from that), but using "pushing anything relevant" as a metric is not correct imho.
80% of software engineering is not about pushing but maintaining. I'm not the one who praises Google (far from that), but using "pushing anything relevant" as a metric is not correct imho.
I touched on that. Their search is highly gameable now, they don't care about correctness anymore. Youtube has incessant ads and their recommendation system sucks - I have to turn it off
Is product definition a part of software engineering?
Their flagship product is Google search. But compared to 10-15 years ago, the results are hyper-monetized, full of spam, and frequently just not especially great.
I understand there are many reasons for this (though some of those originate with Google itself), but that's rather the point. The whole reason Google became a thing is because most search engines 20 years ago were full of results that were "hyper-monetized, full of spam, and frequently just not especially great." Then Google came and offered a product that effectively solved this, and quickly became a giant because of it.
Maintaining, let alone evolving, means continuing to work at least as well in the face of changing conditions.
I understand there are many reasons for this (though some of those originate with Google itself), but that's rather the point. The whole reason Google became a thing is because most search engines 20 years ago were full of results that were "hyper-monetized, full of spam, and frequently just not especially great." Then Google came and offered a product that effectively solved this, and quickly became a giant because of it.
Maintaining, let alone evolving, means continuing to work at least as well in the face of changing conditions.
Hey folks. Author of this post here. I sincerely apologize, this was accidentally published live prematurely (my fault). It’s got a number of style and structure issues I’ve been working through locally (you are spot on regarding the heading feedback). I have taken it down and will be publishing it in the future once it is actually complete.
Honestly, I thought that was the joke (see the twitter algorithm[0]). That there actually aren't much insights from Google regular software engineers could relate to. As other comments have said, Google is the company who can afford to fuck up the most.
[0] https://news.ycombinator.com/item?id=31160546
[0] https://news.ycombinator.com/item?id=31160546
Thanks for posting this reply to the article, I thought you had taken it down because half the people commenting seem to have forgotten a real human being exists behind the blog post.
Looking forward to reading it when you publish the revised version, though I'm unsure how I would know. (I guess I should check and see if you have RSS)
I wish the civility rules were enforced here a bit more stringently sometimes.
Looking forward to reading it when you publish the revised version, though I'm unsure how I would know. (I guess I should check and see if you have RSS)
I wish the civility rules were enforced here a bit more stringently sometimes.
Hi Addy,
It is sad to see that more often than not, the top comments on any article are overly negative or even vitriolic and any interesting discussion languishes deeper in the article.
I read your draft, and it is full of excellent advice. Some of it being which I also find very important, some which gives me food for thought and some which I am happy to be reminded of again. I look forward to your finished post.
It is sad to see that more often than not, the top comments on any article are overly negative or even vitriolic and any interesting discussion languishes deeper in the article.
I read your draft, and it is full of excellent advice. Some of it being which I also find very important, some which gives me food for thought and some which I am happy to be reminded of again. I look forward to your finished post.
I look forward to reading the completed post, I think there is a lot of good stuff in the draft.
What I noticed from the draft: There's nothing obvious about Google per se, in terms of the material. It could be company X you've been working at. Also, insights cumulatively cover junior dev, senior dev, and some manager experiences. So, audience for this particular piece was not clear to me.
I am surmising this piece complements your "Efficient Engineering" work-in-progress. I'll be curious to read that too.
What I noticed from the draft: There's nothing obvious about Google per se, in terms of the material. It could be company X you've been working at. Also, insights cumulatively cover junior dev, senior dev, and some manager experiences. So, audience for this particular piece was not clear to me.
I am surmising this piece complements your "Efficient Engineering" work-in-progress. I'll be curious to read that too.
I'm guessing here, but I think the author's intent was to share knowledge that's generally applicable in other software engineering environments, rather than learnings that might be specific to Google (those could be both curiosity-building and/or also frustrating for other people to read, if they don't have access to the same systems).
Similarly, it'd likely be possible to learn many of the same findings at other companies -- Google (and the author) have been fortunate to encounter lots of tricky scaling and human issues in technology, and to evaluate potential strategies to deal with those, and it's good to see that knowledge shared.
Similarly, it'd likely be possible to learn many of the same findings at other companies -- Google (and the author) have been fortunate to encounter lots of tricky scaling and human issues in technology, and to evaluate potential strategies to deal with those, and it's good to see that knowledge shared.
Hopefully it doesn't get cancelled a week before it's finally ready.
I look forward to your post (perhaps after another 10 years) about testing before release and your use of CD.
(Opinions are my own)
My favorites from the list, that I did not see other companies (specifically startups) do at all:
> A single large release may be divided into a series of lower-risk well-understood rollouts.
> Design documentation should not be an afterthought but an integral part of the software engineering process.
> Coordinate reviews for the design doc and compare the design as it evolves with the original doc to verify that all the relevant constraints are being addressed.
It might suck to spend 4hr writing a doc and 1hr*(N engineers) reviewing it but getting everyone aligned, and writing down why you did something and what was considered but ignored, is one of the super powers people have. At larger companies this process, when done informally, can be done in under a day and save months of headaches down the line. Especially so when the reviewer leaves comments like "can we break X up into milestone A, B, and C instead of rolling our a single piece".
My favorites from the list, that I did not see other companies (specifically startups) do at all:
> A single large release may be divided into a series of lower-risk well-understood rollouts.
> Design documentation should not be an afterthought but an integral part of the software engineering process.
> Coordinate reviews for the design doc and compare the design as it evolves with the original doc to verify that all the relevant constraints are being addressed.
It might suck to spend 4hr writing a doc and 1hr*(N engineers) reviewing it but getting everyone aligned, and writing down why you did something and what was considered but ignored, is one of the super powers people have. At larger companies this process, when done informally, can be done in under a day and save months of headaches down the line. Especially so when the reviewer leaves comments like "can we break X up into milestone A, B, and C instead of rolling our a single piece".
What's critical is that this remains an informal process.
The moment this process becomes codified into a formal doc that requires approvals and reviews across teams, is the moment this ends up becoming a far larger part of the work than it should. Making it a required aspect of the job dramatically slows down the work and makes the doc act as a gatekeeper to technical work.
The moment this process becomes codified into a formal doc that requires approvals and reviews across teams, is the moment this ends up becoming a far larger part of the work than it should. Making it a required aspect of the job dramatically slows down the work and makes the doc act as a gatekeeper to technical work.
Unfortunately, once you have 100+ people across multiple teams and disciplines working on the same product release, formal documentation and all the ceremony that goes with it is no longer optional. Things have to be written down to keep that many people all on the same page and focused on the same goals.
[deleted]
> It might suck to spend 4hr writing a doc
Maybe our releases are too big but our technical specs take much longer to write and even longer to coordinate stakeholders when conflicts do come up.
Can people really crank these docs out in 4 hours?
Maybe our releases are too big but our technical specs take much longer to write and even longer to coordinate stakeholders when conflicts do come up.
Can people really crank these docs out in 4 hours?
Depends on the size and level of detail. I'm currently on page 25 and week three of a doc. I've also drafted a quick one or two pager with summary, goals, non-goals, and a plan with links to code + libraries + docs.
Also, grass is green, sky is blue.
For those that are getting a 404 not found, here's the link from archive:
https://web.archive.org/web/20220519020040/https://addyosman...
I sincerely didn't expect that from all the negative comments. At some point HN/reddit started confusing healthy skepticism and alternative views with being an... downright negative and rude.
> Start with the User Experience and work backwards to the technology you need.
> The best software is built by engineers who have empathy for their users.
I’m not sure where all the criticism here comes from but those alone are some of the most valuable insights any developer can obtain.
The fact that Google, in my opinion, is as far from those insights as a company can possibly come is in my eyes unrelated to the lessons that the author has learned working for Google. I think there is way too much anger in this thread, and I’m surprised to see it here on HN. I’ve personally moved all my services away from Google, including the paid ones like gsuite, but this article doesn’t seem to warrant any of the harsh criticism here based on its content.
> The best software is built by engineers who have empathy for their users.
I’m not sure where all the criticism here comes from but those alone are some of the most valuable insights any developer can obtain.
The fact that Google, in my opinion, is as far from those insights as a company can possibly come is in my eyes unrelated to the lessons that the author has learned working for Google. I think there is way too much anger in this thread, and I’m surprised to see it here on HN. I’ve personally moved all my services away from Google, including the paid ones like gsuite, but this article doesn’t seem to warrant any of the harsh criticism here based on its content.
In fairness, writing is a specific skill. Most people aren't very good at it. It's one thing being able to draft an email or a blog post. Even if it has the quality of being clear and sounding empathetic, that doesn't magically turn it into good writing.
This post is the perfect example. It lacks an original premise, a unique sense of language, or an understanding of the relationship between incident and narrative. Most of its core ideas are hokum: 'I think it was the immortal Steve Jobs who said "don't worry about getting wet, just dance in the rain"'.
But Google isn't a writing company. You could work there for a hundred years and never gain the ability to write anything worth reading. There's no evidence that the experience produces unique literary insight. Instead, it seems to create exactly this kind of sludgy, imprecise advice. All of which is unintentionally revealing.
This post is the perfect example. It lacks an original premise, a unique sense of language, or an understanding of the relationship between incident and narrative. Most of its core ideas are hokum: 'I think it was the immortal Steve Jobs who said "don't worry about getting wet, just dance in the rain"'.
But Google isn't a writing company. You could work there for a hundred years and never gain the ability to write anything worth reading. There's no evidence that the experience produces unique literary insight. Instead, it seems to create exactly this kind of sludgy, imprecise advice. All of which is unintentionally revealing.
Sorry about that. The post will be live again (in complete form) in the next week or two. This was an accidental early deploy :/
Oh sorry to repost this!
Since the link 404's now I guess the best Insight is to keep posts like Google products, briefly and terminate them unexpectedly :P
Wow. This is such BS.
This whole company and it's bests products were build by engineers who couldn't follow this rules, because they simple didn't had decades of required experience.
Instead of following dozens of weird rules, they just built what felt right.
That's it.
The story is only applicable to soulless coders in a corp team. Any potential for innovation is dead now.
Well, for a draft, you just got one hell of a community proof read.
Good luck with the final article!
Good luck with the final article!
[deleted]
> Start with the User Experience and work backwards to the technology you need.
I have a counter example. Founder of a very successful startup (early 2000's) once told me that they first built the technology (adding tracing to Java apps at run-time) just because it was interesting, without any clue if it can be useful to someone. Later they showed it to everyone and they got their first customer who agreed to try it (it required modification to JRE).
I have a counter example. Founder of a very successful startup (early 2000's) once told me that they first built the technology (adding tracing to Java apps at run-time) just because it was interesting, without any clue if it can be useful to someone. Later they showed it to everyone and they got their first customer who agreed to try it (it required modification to JRE).
That sounds more like success in spite of a lack of market interest, not because of it. The amount of successful products that started on the inverse likely will massively outweigh these anecdotal experiences.
Yup these are all very good, couldn’t find one I disagreed with. I loved the statement on nuance around idioms like YAGNI, which can easily be abused.
i wish what you would write about is why MDL became MDC became Lit / MWC, and while it's still a complete mess.
the last time Material worked flawlessly with Bazel outside Google was arguably MDL, if you could even count that as working.
i just don't understand why Google isn't investing in OSS the way it used to. why it cannot seem to do the thing that should be easiest: to make its own software work with itself. it seemingly cannot do this even in high-value categories it cares deeply about, like Cloud.
i loved MDL. thank you for writing it. but i wish you would write something that cuts to why it is 2022 and Google just announced Flutter integration with Firebase like we all don't know that absolutely should have been a basic staple feature from the get go.
the last time Material worked flawlessly with Bazel outside Google was arguably MDL, if you could even count that as working.
i just don't understand why Google isn't investing in OSS the way it used to. why it cannot seem to do the thing that should be easiest: to make its own software work with itself. it seemingly cannot do this even in high-value categories it cares deeply about, like Cloud.
i loved MDL. thank you for writing it. but i wish you would write something that cuts to why it is 2022 and Google just announced Flutter integration with Firebase like we all don't know that absolutely should have been a basic staple feature from the get go.
pardon me, and *why it's still a complete mess. because it undeniably is
I saw this guy speak once, he was insufferable, smug, self-promoting and actually pretty boring.
These are some really uncharitable takes. A lot of us have found value in the author’s writing and code contributions over the years, many of which he made on his own time. This type of post is subjective by nature, give the guy a break.
Currently getting 404 File Not Found. Guess there are no insights to be found.
This draft post was accidentally made public. Please move along
[deleted]
a long time ago addy made some demo page showing the use of webcam and displaying it on the screen and capturing a screenshot of the webcam view. It helped me out more than once
I’m sorry but I’m having a hard time following this page. There are at least two sizes of section headers, and I can’t tell if one is a subsection of the other or they communicate some kind of relative importance or what.
The large pull quotes (which I’ll be honest I’m never a fan of) just interrupt me trying to figure out the layout.
So I’m not sure how things are organized.
I’m on my phone now, maybe it’s better on desktop. Reader mode didn’t really change anything so unfortunately that didn’t help.
The article sounds interesting.
The large pull quotes (which I’ll be honest I’m never a fan of) just interrupt me trying to figure out the layout.
So I’m not sure how things are organized.
I’m on my phone now, maybe it’s better on desktop. Reader mode didn’t really change anything so unfortunately that didn’t help.
The article sounds interesting.
It's just as bad on desktop. I found it hugely helpful to tweak <h2> with margin-left: -4rem, and <h3> with margin-left: -2rem (using the Developer Tools that the author works/worked on?). Worth the effort, great advice here.
404 File not found
Figures.
Figures.
dead link?
It may seem harsh, but the industry should stop elevating this kind of pablum.