Architecting large software projects [video](youtube.com)
youtube.com
Architecting large software projects [video]
https://www.youtube.com/watch?v=sSpULGNHyoI
65 comments
He basically just described the FCIS[0] architecture—the same one Gary Bernhart laid out thirteen years ago. We love reinventing the wheel. Chaplicki did it with ELM, Abramov with Redux, day8 did it with re-frame, and the beat goes on.
I’m still amazed it isn’t obvious: every piece of software should be a black box with a pin-hole for input and an even tinier pin-hole for output. The best code I’ve ever touched worked exactly like that and maintaining it was a pleasure, everything else was garbage. I push this rule in every project I touch.
[0] https://www.destroyallsoftware.com/screencasts/catalog/funct...
I’m still amazed it isn’t obvious: every piece of software should be a black box with a pin-hole for input and an even tinier pin-hole for output. The best code I’ve ever touched worked exactly like that and maintaining it was a pleasure, everything else was garbage. I push this rule in every project I touch.
[0] https://www.destroyallsoftware.com/screencasts/catalog/funct...
> Chaplicki did it with ELM, Abramov with Redux, day8 did it with re-frame, and the beat goes on.
None of these projects reinvented the concept of FCIS.
These are just projects that had functional elements, or were outright functional languages. This is like saying jsx reinvented markup languages.
None of these projects reinvented the concept of FCIS.
These are just projects that had functional elements, or were outright functional languages. This is like saying jsx reinvented markup languages.
I think the actual point was,
> We love reinventing the wheel.
> We love reinventing the wheel.
Could you point to the place in the video that you felt resembled this principle most? I don’t really see the connection, but am open to it.
"The Unaccountability Machine" does a great job summarizing the thought of Stafford Beer, who explained basically exactly what you said as the only viable strategy for designing and managing complex systems.
I think the mistake people make when trying to teach this stuff is in generalizing too much.
His input layer is good because it helped him emulate local multiplayer with keyboard and mouse. It solved _his problem_.
The graphics layer is good because it is so much easier to work with than OpenGL _for him_.
The library wrappers are good for him because they solve _his desire_ to run portably while maintaining the smallest possible interface.
This stuff matters to Eskil because he’s: - just one person - has deep expertise in what he’s wrapping (win32, OpenGL) - wants to make an impressive program solo
I think his expertise, historical perspective, and culture make it feel as if this is the only way to do this very hard task, so he wants to share it. It helps him believe his way is right that many great, and reliable projects are done in C89.
I think the truth at this point is that folks still using old versions of C have, on average, more experience than everyone else. It’s not just the language that’s making them strong, either. It’s having evolved with the platforms.
Now the only question that leaves is whether it makes a huge difference to really stick with one language over the decades. I know we’ve all heard both sides of that quandary.
His input layer is good because it helped him emulate local multiplayer with keyboard and mouse. It solved _his problem_.
The graphics layer is good because it is so much easier to work with than OpenGL _for him_.
The library wrappers are good for him because they solve _his desire_ to run portably while maintaining the smallest possible interface.
This stuff matters to Eskil because he’s: - just one person - has deep expertise in what he’s wrapping (win32, OpenGL) - wants to make an impressive program solo
I think his expertise, historical perspective, and culture make it feel as if this is the only way to do this very hard task, so he wants to share it. It helps him believe his way is right that many great, and reliable projects are done in C89.
I think the truth at this point is that folks still using old versions of C have, on average, more experience than everyone else. It’s not just the language that’s making them strong, either. It’s having evolved with the platforms.
Now the only question that leaves is whether it makes a huge difference to really stick with one language over the decades. I know we’ve all heard both sides of that quandary.
> ...reliable projects are done in C89.
Only if static and dynamic analysers are part of the development cycle.
Only if static and dynamic analysers are part of the development cycle.
This type of software architecture is good when you're building stable, well known systems. Notice that none of his examples were new or novel functionality.
Most people don't work in that world.
Most people don't work in that world.
The majority of people work in that world. Even the majority of startups, where you’d expect to find the novel pursuits, are building glorified spreadsheets.
Startups are glorified spreadsheets in the same sense that you and I are glorified collections of atoms. Technically correct (spreadsheets are turing complete after all) but totally useless as a model of how they work.
I’m just making the point that most software dev work is not novel.
You’re either making a productivity app where CRUD and UX are pretty well known patterns.
Or a scalable web system - also very well tried territory.
Or analytics and data processing - again well trod.
If you’re not a good pattern matcher you might think every UI framework, or your next API abstraction, is some next general theory of relativity.
But otherwise the major novelty in most software project pursuits is going to be the context, people, and industry your building it into not the tech
You’re either making a productivity app where CRUD and UX are pretty well known patterns.
Or a scalable web system - also very well tried territory.
Or analytics and data processing - again well trod.
If you’re not a good pattern matcher you might think every UI framework, or your next API abstraction, is some next general theory of relativity.
But otherwise the major novelty in most software project pursuits is going to be the context, people, and industry your building it into not the tech
Right, fair enough, I agree with that.
Thanks, I'm 13m in and I haven't found anything I would consider valuable or insightful.
I haven't watched many of these videos but the two that stand out for me as being genuinely helpful is Jonathan Blow's talk about software quality [0-4] and Daniel Sockwell's "Imagining the Ideal Language for Writing Free Software" [5].
From Blow's talk, my takeaways are:
* Large organizations don't necessarily know how to make software and small teams can make quality, large scale projects
* Abstraction is its own form of optimization and should be used sparingly. Stripping away a level of abstraction is sometimes desirable.
* Design should be data centric. That is, focus on how to pass data between different domains of concern. Don't be shy about exposing data and don't hide data. This differs slightly from "API-centric" approaches as the contract is in the data, not in the function call interface.
* Keep inheritance to a maximum of 2-3 levels deep, with a preference of 1 or none. Not all language features should be used. Use language features sparingly, and this is one of them.
* Duplicate code (e.g. "cut-and-paste") for clarity. Dispensing with code-reuse isn't a sin and is sometimes desirable. DRY is fine but sometimes having a common function increases cognitive load to understand what code is doing and often portions of code that look very similar are different in substantial but subtle ways that make de-duplication overly confusing to handle all cases. When reviewing a function, having all code available locally, without having to page out to different areas of the code base to see what a common function is doing, often helps with readability and isn't as detrimental to reliability as people make it out to be.
From Sockwell's talk:
It's a weird premise (how can software be ideal for writing FOSS?) but the point is that many software architectural concerns stem from what economic environment they're being developed in. FOSS tends to be developed by a single or small team. The focus is on "master-craft" tools that allow a single contributor to be more effective at the cost of a high learning curve. Since corporate environments tend to have high turn over rate with software developers, the focus is on stable "commodity" tools and methodologies that can better take advantage of the fungible nature of their software developer labor.
The talk itself isn't focused on large software projects architecture directly, but, from my perspective, it gives some very actionable items for tackling larger projects on an individual or small teams basis:
* Focus on expressibility. That is, choose languages that allow mastery over being "safe/boring/simple". Don't worry about making code readable to the lowest common denominator, make it readable for someone who's at your skill level but is trying to find out what the code does (aka "future you"). Allow "footguns" if you trust you can avoid them. Don't worry about language popularity. Enjoy using a "dialects" if it helps empower you.
* Focus on composability. Composability is a broad term but focusing on data-centric design is one way to make things composable as the design can be made around the data being transported.
For me, the major insight is to understand the context of what software is being developed. A corporate environment will have different concerns, for both the employer and employee, than a small project with one or a few developers.
For example, a corporate environment might favor a language that encourages a type of code structure, testing suite, code review process and idiomatic way of solving problems. This makes labor more fungible but means that increased productivity of any individual developer will be met with resistance if it deviates from corporate convention. It also means employees will be motivated to recommend projects using technology that's perceived to be corporate friendly to make sure they're more marketable.
Small projects (FOSS, say) might focus on niche languages, dialects, idioms or other non-standard conventions that incorporate well with the skills of the primary developer. This means the individual developer is more productive but makes it harder for others to contribute.
[0] https://www.youtube.com/watch?v=oJ4GcZs7y6g
[1] https://www.youtube.com/watch?v=FvBySbGueww
[2] https://www.youtube.com/watch?v=URSOH4F3vbU
[3] https://www.youtube.com/watch?v=MlIZ2Q3Rvc8
[4] https://www.youtube.com/watch?v=xK6YZ3NFjuE
[5] https://ftp.belnet.be/mirror/FOSDEM/video/2021/D.perl/progra...
I haven't watched many of these videos but the two that stand out for me as being genuinely helpful is Jonathan Blow's talk about software quality [0-4] and Daniel Sockwell's "Imagining the Ideal Language for Writing Free Software" [5].
From Blow's talk, my takeaways are:
* Large organizations don't necessarily know how to make software and small teams can make quality, large scale projects
* Abstraction is its own form of optimization and should be used sparingly. Stripping away a level of abstraction is sometimes desirable.
* Design should be data centric. That is, focus on how to pass data between different domains of concern. Don't be shy about exposing data and don't hide data. This differs slightly from "API-centric" approaches as the contract is in the data, not in the function call interface.
* Keep inheritance to a maximum of 2-3 levels deep, with a preference of 1 or none. Not all language features should be used. Use language features sparingly, and this is one of them.
* Duplicate code (e.g. "cut-and-paste") for clarity. Dispensing with code-reuse isn't a sin and is sometimes desirable. DRY is fine but sometimes having a common function increases cognitive load to understand what code is doing and often portions of code that look very similar are different in substantial but subtle ways that make de-duplication overly confusing to handle all cases. When reviewing a function, having all code available locally, without having to page out to different areas of the code base to see what a common function is doing, often helps with readability and isn't as detrimental to reliability as people make it out to be.
From Sockwell's talk:
It's a weird premise (how can software be ideal for writing FOSS?) but the point is that many software architectural concerns stem from what economic environment they're being developed in. FOSS tends to be developed by a single or small team. The focus is on "master-craft" tools that allow a single contributor to be more effective at the cost of a high learning curve. Since corporate environments tend to have high turn over rate with software developers, the focus is on stable "commodity" tools and methodologies that can better take advantage of the fungible nature of their software developer labor.
The talk itself isn't focused on large software projects architecture directly, but, from my perspective, it gives some very actionable items for tackling larger projects on an individual or small teams basis:
* Focus on expressibility. That is, choose languages that allow mastery over being "safe/boring/simple". Don't worry about making code readable to the lowest common denominator, make it readable for someone who's at your skill level but is trying to find out what the code does (aka "future you"). Allow "footguns" if you trust you can avoid them. Don't worry about language popularity. Enjoy using a "dialects" if it helps empower you.
* Focus on composability. Composability is a broad term but focusing on data-centric design is one way to make things composable as the design can be made around the data being transported.
For me, the major insight is to understand the context of what software is being developed. A corporate environment will have different concerns, for both the employer and employee, than a small project with one or a few developers.
For example, a corporate environment might favor a language that encourages a type of code structure, testing suite, code review process and idiomatic way of solving problems. This makes labor more fungible but means that increased productivity of any individual developer will be met with resistance if it deviates from corporate convention. It also means employees will be motivated to recommend projects using technology that's perceived to be corporate friendly to make sure they're more marketable.
Small projects (FOSS, say) might focus on niche languages, dialects, idioms or other non-standard conventions that incorporate well with the skills of the primary developer. This means the individual developer is more productive but makes it harder for others to contribute.
[0] https://www.youtube.com/watch?v=oJ4GcZs7y6g
[1] https://www.youtube.com/watch?v=FvBySbGueww
[2] https://www.youtube.com/watch?v=URSOH4F3vbU
[3] https://www.youtube.com/watch?v=MlIZ2Q3Rvc8
[4] https://www.youtube.com/watch?v=xK6YZ3NFjuE
[5] https://ftp.belnet.be/mirror/FOSDEM/video/2021/D.perl/progra...
Oh wow thanks for writing out that summary of Sockwell's talk. I had a sort of similar line of thought a few years ago which I haven't followed since, but this just brought me back. https://josh8.com/blog/personal_computing.html
I was musing about the point on which technologies one ought to use for writing personal (or FOSS) software vs. corporate software like in the talk, but also whether the ethos we bring to the table between the two should differ -- i.e. 'software as a soap bubble' rather than only allowing yourself to write scalable and maintainable programs that are generalized for re-use. It's as if a whole class of programs which would help us Personally Compute never come into existence because of this mindset. I think the AI vibe coding thingy majingies are not too bad of an antidote to this actually
> Everything is exposed as an API with little no insight to inner workings
This is how I build large scale biz software (30+ years of experience). And I have never seen a case where it wasn't a good idea.
For example, the largest software system in the world (the internet) operate this way.
However I am always open to learn something new?
This is how I build large scale biz software (30+ years of experience). And I have never seen a case where it wasn't a good idea.
For example, the largest software system in the world (the internet) operate this way.
However I am always open to learn something new?
I had the same reaction.
I’m eager to find someone who has managed to abstract expertise.
I’m 25 years into repeatedly building large software systems. I was thinking, look here maybe someone’s managed to codify what I do.
Nope.
To your point, everything is contextual. If you try to apply a suite of rules the systems (which extend far outside of the bounds of a software subsystem deployment) will mock you by burying your efforts leaving no intentional outcome.
Everything is systems thinking and every seat & context is different. You have to navigate mostly with realtime navigations based on long- and hard- earned muscle memory.
I’m eager to find someone who has managed to abstract expertise.
I’m 25 years into repeatedly building large software systems. I was thinking, look here maybe someone’s managed to codify what I do.
Nope.
To your point, everything is contextual. If you try to apply a suite of rules the systems (which extend far outside of the bounds of a software subsystem deployment) will mock you by burying your efforts leaving no intentional outcome.
Everything is systems thinking and every seat & context is different. You have to navigate mostly with realtime navigations based on long- and hard- earned muscle memory.
> Don't ever implement good-enough-for-now APIs
Agree in theory, in practice this is impossible. Even if you're an absolute domain expert in whatever you're doing, software and requirements will evolve and you will end up needing to implement something for which your current API is not suitable. Just ask S3:ListObjectsV2 or golangs' `encoding/json/v2` etc.
I push back hard on this one because a lot of developers will try to be "clever" and basically turn their api into
It's better to accept that your API will change at some point and have a versioning and deprecation strategy in place. With well-architected software this usually isn't hard.
Agree in theory, in practice this is impossible. Even if you're an absolute domain expert in whatever you're doing, software and requirements will evolve and you will end up needing to implement something for which your current API is not suitable. Just ask S3:ListObjectsV2 or golangs' `encoding/json/v2` etc.
I push back hard on this one because a lot of developers will try to be "clever" and basically turn their api into
def my_api(parameters: dict[str, str]) -> dict[str, str]:
It's better to accept that your API will change at some point and have a versioning and deprecation strategy in place. With well-architected software this usually isn't hard.
Yes, and also don't try to anticipate everything by implementing features that won't be used soon, ie "just in case". If the soon turns to never, any unused feature is basically dead code and represents future costs and constraints to maintain or remove it.
The most sustainable approach is designing APIs with explicit extension points and versioning mechanisms from day one, rather than trying to predict all future requirements or resorting to overly generic interfaces.
That’s not how I read “Don't ever implement good-enough-for-now APIs”.
Requirements _may_ change, but it’s much harder to have consumers move to a new API. Once it’s published, it’s a contract you don’t want to break.
That doesn’t mean you need to design an extremely flexible and future-proof API to the point it stops making sense — it’s a false dichotomy.
What you can do is take your time designing an API that still makes sense as long as possible, until your understanding of the domain has changed so much that it’s a different shape altogether.
Throwing your hands up and saying “it’s impossible” is easy, the art is in figuring out how much has to change until you have to say that.
Design is a game of push-pull between context and form, and you can add tolerances both ways.
Requirements _may_ change, but it’s much harder to have consumers move to a new API. Once it’s published, it’s a contract you don’t want to break.
That doesn’t mean you need to design an extremely flexible and future-proof API to the point it stops making sense — it’s a false dichotomy.
What you can do is take your time designing an API that still makes sense as long as possible, until your understanding of the domain has changed so much that it’s a different shape altogether.
Throwing your hands up and saying “it’s impossible” is easy, the art is in figuring out how much has to change until you have to say that.
Design is a game of push-pull between context and form, and you can add tolerances both ways.
Another way to put it: Software isn't just stale building blocks, it's an ever evolving ecosystem with varying lifetimes. Developers should embrace the different periods during the lifetime of software, and model with them, not around them.
Yeah, it’s too hard of a rule. In reality interfaces have points of resistance; where they aren’t really helping you do what you’re wanting, and they have fitting-points where they do what you need done.
I’d argue it’s your job to strike some kind of balance there. If you know you’re working with something stable, why settle for good-enough? Well, because the task of assessing what is stable requires years of mistakes, and good-enough varies drastically between languages. I think I see a point here for using primitive C; there’s hardly a type-system to waste time with, but you can cut yourself with macros. This is why I use Odin.
I’d argue it’s your job to strike some kind of balance there. If you know you’re working with something stable, why settle for good-enough? Well, because the task of assessing what is stable requires years of mistakes, and good-enough varies drastically between languages. I think I see a point here for using primitive C; there’s hardly a type-system to waste time with, but you can cut yourself with macros. This is why I use Odin.
I would add a caveat...
Don't ever implement good-enough-for-now APIs without a plan to come back and fix it and a deadline to fix it
Most of the time "good-enough-for-now" really is just "good-enough-forever".
Don't ever implement good-enough-for-now APIs without a plan to come back and fix it and a deadline to fix it
Most of the time "good-enough-for-now" really is just "good-enough-forever".
I’ve never once seen “come back and fix with deadline” work. It’s really only the “with deadline” that does not work. Good teams will constantly be judging if now is the right time to fix the thing, and that fluid process works pretty well. Setting deadlines for tech debt just leads to the people setting the deadlines getting upset and quitting when they are inevitably missed. Save the urgency for when it’s warranted, like discovering a time bomb in the code, a scaling limit that is about to be breached— people will take you more seriously, you will be happier, they will be happier, hooray.
By all means, shoot for absolute perfection on your own projects. I work very, VERY differently on my own code than I do on work code, and I get different types of satisfaction out of both (though of course massively prefer working solo).
By all means, shoot for absolute perfection on your own projects. I work very, VERY differently on my own code than I do on work code, and I get different types of satisfaction out of both (though of course massively prefer working solo).
I’ve implemented APIs with routes like `/temp-get-order` in an attempt to force a return and even then we never returned to it. Like you’re saying it was eventually deemed insufficient for reasons outside of it being imperfect and that’s when we finally replaced it. Maybe places with more resources or discipline can manage otherwise but I’ve never worked somewhere that can.
That only works with product teams, in consulting it is either done alongside the ticket or it is gone forever, there isn't even any guarantee who will stay around after the delivery cycle is done.
And the new team might not even be allowed budget to fix those issues, and better not spend hours that do not fit tickets on the Excel budget plan.
And the new team might not even be allowed budget to fix those issues, and better not spend hours that do not fit tickets on the Excel budget plan.
Started watching, but "C89 is the safe option for long lived software" kind of turned me off. There are plenty of safe long lived stable languages out there where you dont have to manually manipulate memory. At the very least, this guy should be advocating for Java.
But even that's too far really. Like it or not, "shiny fad" languages like Python & Javascript have been around forever, support for them isn't going away, this should be a non-concern at the architectural level. (Bigger language concerns: is it performant enough for my expected use? Does it have strong types to help with correctness? Can I hire programmers who know it? etc)
But even that's too far really. Like it or not, "shiny fad" languages like Python & Javascript have been around forever, support for them isn't going away, this should be a non-concern at the architectural level. (Bigger language concerns: is it performant enough for my expected use? Does it have strong types to help with correctness? Can I hire programmers who know it? etc)
It really depends. I was at talk of architect of car company when he was talking about need to develop and support a car software for 20-30 years - few years before release, 10-20 years of production, and the critical fixes after end of support. And it includes not only soft itself, but all compilers, etc.
> languages like Python & Javascript have been around forever, support for them isn't going away
??? Python 2 went out of support five years ago.
??? Python 2 went out of support five years ago.
I mean C89 has no support, it's not getting an update or a bugfix, the standard is what it is. So if vendor support is your overriding concern, you should be constantly updating your software to LTS versions.
I meant support in terms of there's an active community of people using the language and building things with it. It's not going to die as a language like Algol 68 or Pascal.
I meant support in terms of there's an active community of people using the language and building things with it. It's not going to die as a language like Algol 68 or Pascal.
C89 still has an active community of people using the language and building things with it.
In addition to this, its existence and validity is still relied on by basically every other language via standard library, transient dependency, or calling convention. Heck, practically every C++ project probably depends on it.
The Linux Kernel, ffmpeg, SQLite, curl and many others chose C89 and often consider using C99, but most do not. Each of those projects also write at-length via newsletter or blog as to why they’re still not quite ready to update (and sometimes why they are.)
In addition to this, its existence and validity is still relied on by basically every other language via standard library, transient dependency, or calling convention. Heck, practically every C++ project probably depends on it.
The Linux Kernel, ffmpeg, SQLite, curl and many others chose C89 and often consider using C99, but most do not. Each of those projects also write at-length via newsletter or blog as to why they’re still not quite ready to update (and sometimes why they are.)
The Linux kernel and ffmpeg require C11.
Yes, “chose” in past-tense (and that whole clause about the tension displayed in public communications of these projects contemplating the change.
Both projects made this change in the last couple years. ffmpeg still requires c99 compatibility for their headers, and may indefinitely.
Both projects made this change in the last couple years. ffmpeg still requires c99 compatibility for their headers, and may indefinitely.
When Linux was created, there was not much choice. In reality, most active projects moved on and only projects with relatively extreme portability requirements stick with C89, mostly because MSVC does not support anything later properly.
I think that’s fair, but I also believe that the same things that make C89 meet “extreme portability requirements” are why Eskil still uses it.
Should everyone? No, probably not.
Should everyone? No, probably not.
Yes, but also because he made it work well for him. He also understands that most of the innovations in programming languages is actually not nearly as useful as people think and often does more harm than good. Rejecting newer C standards is a bit too extreme in my opinion, as C is very slowly evolving already.
Is there an active community of people using Python 2 and building things with it? Meanwhile, there are plenty of actively maintained compilers for C89.
I have two different compilers that implement C89 on my computer right now and I know of at least one other. How much support do you require before you consider something supported?
I've seen relational database schemas degenerating into untyped free-for-all key-values because they weren't expressive enough. I've seen massive code duplication (and massive amounts of bugs) because no one invested into a quality core set of libraries to address basic use-cases of a given platform. I've seen systems crumbling under tech debt because of chronic prioritization of features over maintenance.
I've worked on large software projects. The only ones I've met that weren't dreadful to work on had engineering and management working together with a shared, coherent vision.
I've worked on large software projects. The only ones I've met that weren't dreadful to work on had engineering and management working together with a shared, coherent vision.
I think the "how easy will this be to replace" question should be a guiding factor in terms of some architectural decisions. Having modular code, and abstractions can add a lot of value, but if you do it in a way that locks you down and makes everything it touches a multiple in terms of complexity, is it really worth it.
One thing I often do, in terms of modularizing, especially for re-share or external use is sit down and think about the ergonomics of what I'm creating and how I would like to be able to use/consume said module... often writing documentation before writing the code.
One thing I often do, in terms of modularizing, especially for re-share or external use is sit down and think about the ergonomics of what I'm creating and how I would like to be able to use/consume said module... often writing documentation before writing the code.
Ages ago, before I had any real professional experience, I was blown away by Steenberg’s demos and enamored by his tech talks. The demos still impress me and have aged well, but today I’m glad I didn’t fall into the trap of taking his development advice as scripture.
His name has been popping up a lot more recently. I would be worried about the impact he might be having on young programmers, but honestly I doubt his worst advice would survive in industry anyway. I didn’t watch this particular video but his recent talk at Better Software Conference was illuminating, in a bad way.
His name has been popping up a lot more recently. I would be worried about the impact he might be having on young programmers, but honestly I doubt his worst advice would survive in industry anyway. I didn’t watch this particular video but his recent talk at Better Software Conference was illuminating, in a bad way.
I am assuming you're talking about this one: https://www.youtube.com/watch?v=EGLoKbBn-VI
The title is the only good thing about this talk, his whole ideology about building a WRAPPER around everything and maintaining hundreds of these wrappers is going to make you NEVER finish a single project. Also it just feels like this entire talk is an advert for C89, I do enjoy playing with the language sometimes especially when I want to scratch my software architecture itch (I use it kind of like pseudocode to develop a data model for my application), but it's simply not a silver bullet.
I've been watching his screencasts on and off for a couple of years and am always slightly aghast at how he can be really spot on about some very specific things and be wildly off the mark about most of the things regarding software design/architecture.
Anyway, people should finish projects, that's the most important part, my 50 weekend projects and 12 registered domains have been screaming this at me for years.
The title is the only good thing about this talk, his whole ideology about building a WRAPPER around everything and maintaining hundreds of these wrappers is going to make you NEVER finish a single project. Also it just feels like this entire talk is an advert for C89, I do enjoy playing with the language sometimes especially when I want to scratch my software architecture itch (I use it kind of like pseudocode to develop a data model for my application), but it's simply not a silver bullet.
I've been watching his screencasts on and off for a couple of years and am always slightly aghast at how he can be really spot on about some very specific things and be wildly off the mark about most of the things regarding software design/architecture.
Anyway, people should finish projects, that's the most important part, my 50 weekend projects and 12 registered domains have been screaming this at me for years.
The "one module should be written by only one person" dogma is kind of interesting.
But I got to the "my wrapper around SDL supports multiple simultaneous mouse inputs, even though no operating system does" and noped out. YAGNI, even in a large project
But I got to the "my wrapper around SDL supports multiple simultaneous mouse inputs, even though no operating system does" and noped out. YAGNI, even in a large project
He’s sitting at a system that thousands of people built together simultaneously. We have gripes with our OSes but they’re all capable of nearly perfect uptime (depending on the hardware and workload.) So I am not convinced individuals need to own modules. I think it’s good for things to work that way, but not necessary.
I didn’t find much fault at all with what he’s saying about SDL. It’s just an example of the “layered design” he’s advocating for. You may have drawn your conclusion a little early; he immediately follows up with his API for graphics, which is actually a very practical example. He’s really just saying people should consider writing their own APIs even if it’s implemented with a library, because you can select exactly how complex the API needs to be for your app’s requirements. This makes the task of replacing a dependency much simpler.
He’s actually trying to prevent relying on things you don’t need. YAGNI.
I didn’t find much fault at all with what he’s saying about SDL. It’s just an example of the “layered design” he’s advocating for. You may have drawn your conclusion a little early; he immediately follows up with his API for graphics, which is actually a very practical example. He’s really just saying people should consider writing their own APIs even if it’s implemented with a library, because you can select exactly how complex the API needs to be for your app’s requirements. This makes the task of replacing a dependency much simpler.
He’s actually trying to prevent relying on things you don’t need. YAGNI.
> YAGNI
“You Aren’t Gonna Need It”
“You Aren’t Gonna Need It”
I feel like a lot of his takes -- like c89 being the best -- may be true in the context of the kind of complex desktop applications he seems to develop, but not universally applicable.
Still, he gives a lot of good advice and fundamental engineering rules.
Still, he gives a lot of good advice and fundamental engineering rules.
I'm about 4 years into my career (mostly self-taught and learned-on-the-job), and I actually found Eskil's talk inspiring. The part that hit me hardest (in the good sense) was "don’t implement good-enough APIs."
As someone still figuring out my footing, I often feel pretty torn: should I train myself to always push for the cleanest (or near-perfect) design I can manage (as books like "The Pragmatic Programmer" and talks like this suggest), or is that just setting myself up for frustration in industry where pragmatism and deadlines usually win?
What worries me is exactly what some people here are saying – that "temporary" APIs usually last forever. I’ve already seen that on my team. So part of me wants to internalize Eskil's idealism early, even if it means clashing with more pragmatic coworkers.
For those of you with decades of experience: if you were mentoring someone at my stage, would you tell them to lean toward ideals (aiming high, then compromising when needed), or to embrace trade-offs and "good enough" thinking from the start?
As someone still figuring out my footing, I often feel pretty torn: should I train myself to always push for the cleanest (or near-perfect) design I can manage (as books like "The Pragmatic Programmer" and talks like this suggest), or is that just setting myself up for frustration in industry where pragmatism and deadlines usually win?
What worries me is exactly what some people here are saying – that "temporary" APIs usually last forever. I’ve already seen that on my team. So part of me wants to internalize Eskil's idealism early, even if it means clashing with more pragmatic coworkers.
For those of you with decades of experience: if you were mentoring someone at my stage, would you tell them to lean toward ideals (aiming high, then compromising when needed), or to embrace trade-offs and "good enough" thinking from the start?
6:55> "C98"
7:05> "It's faster to write five lines of code today, than to write one line today, and have to edit it in the future"
This is my experience writing C as well, which is why I now avoid it.
There are plenty of times where writing something in C seems like it would be simpler and more reliable than in other languages. But I found I couldn't edit it. Too many of my assumptions get baked into the code and I can't see them later.
7:05> "It's faster to write five lines of code today, than to write one line today, and have to edit it in the future"
This is my experience writing C as well, which is why I now avoid it.
There are plenty of times where writing something in C seems like it would be simpler and more reliable than in other languages. But I found I couldn't edit it. Too many of my assumptions get baked into the code and I can't see them later.
IMHO there are a lot of problems with the advice from Eskil.
His ideas about abstraction of the platform made a lot of sense in the past decades, but IMHO are not that applicable nowadays: It is well understood to separate domain logic from UI and storage, and a desktop application has a very different UI from a mobile app or even a web app.
My next critic point is, that Eskil assumes all domain knowledge is there from the beginning and will not change over time (his idea of having perfect APIs from the beginning).
No mention about error handling, reliability, response times, atomic operations etc.
To make it very clear: I believe Eskil gives his advice in good faith and his advice served him well in his own projects. In my opinion most of his advice is just not good advice in general.
(Besides having modules as black box building blocks if possible).
His ideas about abstraction of the platform made a lot of sense in the past decades, but IMHO are not that applicable nowadays: It is well understood to separate domain logic from UI and storage, and a desktop application has a very different UI from a mobile app or even a web app.
My next critic point is, that Eskil assumes all domain knowledge is there from the beginning and will not change over time (his idea of having perfect APIs from the beginning).
No mention about error handling, reliability, response times, atomic operations etc.
To make it very clear: I believe Eskil gives his advice in good faith and his advice served him well in his own projects. In my opinion most of his advice is just not good advice in general.
(Besides having modules as black box building blocks if possible).
Yeah, I think the first mistake is thinking 'large software project' is specific enough to have any good advice at all.
Large software projects could include AAA console videogame, or banking mobile app, or embedded vehicle entertainment system, or airport baggage handling system, or hundreds of other totally different domains.
The only good advice I can think of is given a large software project try to hire someone (or a whole team) that have built something pretty similar before.
Large software projects could include AAA console videogame, or banking mobile app, or embedded vehicle entertainment system, or airport baggage handling system, or hundreds of other totally different domains.
The only good advice I can think of is given a large software project try to hire someone (or a whole team) that have built something pretty similar before.
So I watched about 15 minutes. I'm not already aware of this person, and their website isn't too illuminating, and there is an hour left. So I have to confess I closed the tab.
His ideas feel, frankly, like catnip for developers. Each dev writes there own modules, so you get to do whatever you want without having to deal with other people's coding styles or philosophies (or better ideas), and no meetings, and you also spend the "right" amount of time building everything.
But it doesn't sound like it would work on "LARGE" (his capitals) software projects, at least none that I've worked on. In those, creating modules small enough to be built once by one dev would just push all discussion (and dreaded meetings) into the API layer, with double the meetings because you should get it right first time, actually double them again because the strategy if someone leaves and their modules aren't right is to just reimplement them so you have to be even more sure they are thoughtfully designed.
It actually sounds like a pitch for agentic coding: if you define your APIs small enough and correctly enough, you don't need to worry about implementation quality, so just let the api do it. I think it's a good way of thinking about it, and thinking about all the ways in which you might not be comfortable with that.
Anyway, what do I know? Maybe he's a super expert on this stuff. I would gentle suggest, if he redid the talk, to open with a) what he means by LARGE software projects, and b) an example where this strategy he is about to explain has been implemented.
His ideas feel, frankly, like catnip for developers. Each dev writes there own modules, so you get to do whatever you want without having to deal with other people's coding styles or philosophies (or better ideas), and no meetings, and you also spend the "right" amount of time building everything.
But it doesn't sound like it would work on "LARGE" (his capitals) software projects, at least none that I've worked on. In those, creating modules small enough to be built once by one dev would just push all discussion (and dreaded meetings) into the API layer, with double the meetings because you should get it right first time, actually double them again because the strategy if someone leaves and their modules aren't right is to just reimplement them so you have to be even more sure they are thoughtfully designed.
It actually sounds like a pitch for agentic coding: if you define your APIs small enough and correctly enough, you don't need to worry about implementation quality, so just let the api do it. I think it's a good way of thinking about it, and thinking about all the ways in which you might not be comfortable with that.
Anyway, what do I know? Maybe he's a super expert on this stuff. I would gentle suggest, if he redid the talk, to open with a) what he means by LARGE software projects, and b) an example where this strategy he is about to explain has been implemented.
Aside - Why do we need the word "architecting" anyway? Why not just use designing?
Software architecture is about the higher-level, big-picture stuff: https://stackoverflow.com/a/704909/623763
Architecting at least somewhat harkens to engineering; where there are costs, limits, tolerances, and to some degree aesthetics.
I'm pretty sure every engineering field calls it designing. Perhaps software devs feel a need to inflate what they actually do.
I think the analogy is to construction, not engineering.
In construction, architects and designers do different jobs. The software equivalents of those roles map reasonably well to the construction equivalents.
In construction, architects and designers do different jobs. The software equivalents of those roles map reasonably well to the construction equivalents.
In my (software) experience, the terms are basically interchangeable. Some people will violently defend “architect right, design wrong” and others the opposite, So uh, pretty hard for me, a normal person, to care much about which word is right for the “you sit down and think before you build” part of software engineering.
In my country, we have both the academic title "engineer", and "engineer architect". People view this as "proper engineer" and "not so proper engineer".
Anyway, "you sit down and think before you build" is indeed what you want and the word for that is "strategy".
Anyway, "you sit down and think before you build" is indeed what you want and the word for that is "strategy".
> the “you sit down and think before you build” part of software engineering.
Well, Agile, Scrum, and backlogs took care of that. /s
Well, Agile, Scrum, and backlogs took care of that. /s
Design and architecture are 2 different things.
Design is about appearance.
Architecture is about structure.
Software design [0] isn’t about appearance.
[0] https://en.wikipedia.org/wiki/Software_design
[0] https://en.wikipedia.org/wiki/Software_design
Obligatory mention of A Philosophy of Software Design by John Ousterhout as the arguably the most important book every developer should read to understand proper modularization and complexity management.
1. Everything is exposed as an API with little no insight to inner workings (black box)
2. Everything should be broken down into modules at a level where one person works on that module
3. He shows a video of his video editor, saying it supports multiple inputs (like two keyboards or two mice), then says no platform supports it, but if they ever do, it will work
4. Don't implement 'good enough apis'
I hope that anybody who has ever worked on software understands that there are virtues to doing exactly the opposite of what is described in each of these points. Even if you can make an argument for any of these, you would have to qualify them with so many exceptions that you would effectively negate the entire argument.
I spent a lot of evenings early in my career watching similar videos, hoping to find some magic bullet in how people better than me do what I do. People make a living doing this on the conference circuit. Maybe it is a fools errand to try to distill something as complex and situationally dependent as software into a video, but I'm having a hard time finding any major insights in all of the videos I watched.