Architecture Playbook(nocomplexity.com)
nocomplexity.com
Architecture Playbook
https://nocomplexity.com/documents/arplaybook/index.html
47 comments
because of my own work history and experience, when I see this I hear "build a Rails app". but I am curious, what do other people that agree with this feel like is their go-to for something like that?
There are several stacks like spring-boot (no Hibernate), flask-sqlalchemy, elixir-phoenix that fit the description the parent has given. All these are pragmatic choices.
My work experience is mostly python, so I hear "build a Django app"[1].
However Python and Ruby will hit scaling issues due to their performance limitations. I hear a lot of positivity on HN about what a single instance of C# monolith can handle, but I'm not familiar with the ecosystem.
[1] But don't use common Django patterns because they suck. Use it because of the completeness of the tooling and libraries.
However Python and Ruby will hit scaling issues due to their performance limitations. I hear a lot of positivity on HN about what a single instance of C# monolith can handle, but I'm not familiar with the ecosystem.
[1] But don't use common Django patterns because they suck. Use it because of the completeness of the tooling and libraries.
> However Python and Ruby will hit scaling issues due to their performance limitations.
This is entirely determined by the problem domain it's being used for. The vast majority of web applications could use Django just fine without ever hitting a performance bottleneck outside the odd viral hug of death. You should prove you will have performance problems before you try to design solutions for it.
This is entirely determined by the problem domain it's being used for. The vast majority of web applications could use Django just fine without ever hitting a performance bottleneck outside the odd viral hug of death. You should prove you will have performance problems before you try to design solutions for it.
True, I just noted that because if you're starting from true zero with no obvious preference for a particular language/framework, you have the opportunity to pick something that can reach a higher top end of performance before it requires fiddling.
> I hear a lot of positivity on HN about what a single instance of C# monolith can handle, but I'm not familiar with the ecosystem.
I work a lot in the C# and .NET space, but also write quite a lot of Python. The performance differences come down to a few key things that are baked in:
* The .NET runtime (ignore the web framework for now) is highly optimized compared to the Python or Ruby interpreters. In practice, you can even save VM costs given how inefficient the interpreters can be. StackOverflow's original dev, Jeff Atwood, has written extensively about this over the years (https://www.google.com/search?q=c%23+asp.net+performance+sit....) For many web developers, you could consider runtime performance of the platform a solved problem. Obviously, there are situations where you still need to measure and optimize. If you have to optimize, some of the tools seem ugly, but work out well in practice (e.g. WinDbg and other low-level debugging tools which have existed for 15+ years.)
* The web framework options look like libraries compared to frameworks like Django or Rails. ASP.NET started with some sucky abstractions (I spent years elegantly working around these with custom HttpModules and HttpHandlers), but around 2007 they introduced ASP.NET MVC which was a big improvement. It is closer to Flask than Django, but there is some structure by default.
* Many C# developers are building on SQL Server. It has warts, but like the .NET runtime, it has some fairly serious optimization behind it - and, you have to buy into any bespoke SQL extensions as part of T-SQL. The tools available were always more advanced than those for open source database. In my experience, better .NET devs more frequently have intermediate and advanced SQL/RDBMS skills. As a result, they'll use ORMs, but catch things that I've seen missed in other frameworks. Dapper (https://github.com/StackExchange/Dapper) and other low-level tools are available for even better data access performance.
* Generally, the tooling is well optimized as long as you buy into Microsoft's workflows. With ASP.NET MVC and later, things improved a lot because the original tooling was built for the ASP.NET Web Forms abstraction.
All of the items above come together for great performance, but you have to be prepared to buy into the platform. This is becoming less of a challenge, but many people left .NET from 2006 to recently because of some enforced choices with the experience. I see a really good future with .NET Core because they've recognized and are fixing the earlier issues.
I work a lot in the C# and .NET space, but also write quite a lot of Python. The performance differences come down to a few key things that are baked in:
* The .NET runtime (ignore the web framework for now) is highly optimized compared to the Python or Ruby interpreters. In practice, you can even save VM costs given how inefficient the interpreters can be. StackOverflow's original dev, Jeff Atwood, has written extensively about this over the years (https://www.google.com/search?q=c%23+asp.net+performance+sit....) For many web developers, you could consider runtime performance of the platform a solved problem. Obviously, there are situations where you still need to measure and optimize. If you have to optimize, some of the tools seem ugly, but work out well in practice (e.g. WinDbg and other low-level debugging tools which have existed for 15+ years.)
* The web framework options look like libraries compared to frameworks like Django or Rails. ASP.NET started with some sucky abstractions (I spent years elegantly working around these with custom HttpModules and HttpHandlers), but around 2007 they introduced ASP.NET MVC which was a big improvement. It is closer to Flask than Django, but there is some structure by default.
* Many C# developers are building on SQL Server. It has warts, but like the .NET runtime, it has some fairly serious optimization behind it - and, you have to buy into any bespoke SQL extensions as part of T-SQL. The tools available were always more advanced than those for open source database. In my experience, better .NET devs more frequently have intermediate and advanced SQL/RDBMS skills. As a result, they'll use ORMs, but catch things that I've seen missed in other frameworks. Dapper (https://github.com/StackExchange/Dapper) and other low-level tools are available for even better data access performance.
* Generally, the tooling is well optimized as long as you buy into Microsoft's workflows. With ASP.NET MVC and later, things improved a lot because the original tooling was built for the ASP.NET Web Forms abstraction.
All of the items above come together for great performance, but you have to be prepared to buy into the platform. This is becoming less of a challenge, but many people left .NET from 2006 to recently because of some enforced choices with the experience. I see a really good future with .NET Core because they've recognized and are fixing the earlier issues.
Any stack really if you have a well-gelled team that knows how to communicate and collaborate well together. If you don't have that as your starting point, things will be hard.
Rails is great for certain types of web applications because of the fast feedback loop.
With that said, there are many other good stacks out there today that allow you to go fast. I think Rails is really a victim of its own success. What DHH got right was to focus on developer UX. Now all the other major stacks have understood that there's a new normal in terms of minimum acceptable developer UX quality level and minimum feedback cycle speed. Spring Boot (possibly with Kotlin), .Net Core (possibly with F#), SwiftUI, Jetpack Compose, React Native, React, etc. etc. Everyone is building to the "sexy" dev experience that Rails introduced. jQuery and Prototype were a big deal when they first came out as well.
You also need a good CI/CD pipeline and modern DevOps practices so you can decouple deployment from release.
But like I said before, team communication above all else.
Rails is great for certain types of web applications because of the fast feedback loop.
With that said, there are many other good stacks out there today that allow you to go fast. I think Rails is really a victim of its own success. What DHH got right was to focus on developer UX. Now all the other major stacks have understood that there's a new normal in terms of minimum acceptable developer UX quality level and minimum feedback cycle speed. Spring Boot (possibly with Kotlin), .Net Core (possibly with F#), SwiftUI, Jetpack Compose, React Native, React, etc. etc. Everyone is building to the "sexy" dev experience that Rails introduced. jQuery and Prototype were a big deal when they first came out as well.
You also need a good CI/CD pipeline and modern DevOps practices so you can decouple deployment from release.
But like I said before, team communication above all else.
This reads like GPT-2 generated business platitudes.
"Data is reasonably structured to allow automated processing.". "Data is made available as quickly as necessary to preserve the value of the data."
"Data is reasonably structured to allow automated processing.". "Data is made available as quickly as necessary to preserve the value of the data."
From skimming the document I don't get what "Architecture" even means.
What do business processes have to do with architecture? I always assumed the term describes a category of software engineering practices.
In all of the software things I've participated in, the main driver for any tool, application, library etc. was always data and data flow. Data comes from some place and then gets processed and displayed or consumed. And from these requirements a higher level view, an architecture so to speak, emerges.
As someone who works in small teams, I have never even heard of most of the tools mentioned in this document. The business focused language is very repelling as well. I'm not sure whether I'm too simple minded and incompetent or just live in a completely different world.
What do business processes have to do with architecture? I always assumed the term describes a category of software engineering practices.
In all of the software things I've participated in, the main driver for any tool, application, library etc. was always data and data flow. Data comes from some place and then gets processed and displayed or consumed. And from these requirements a higher level view, an architecture so to speak, emerges.
As someone who works in small teams, I have never even heard of most of the tools mentioned in this document. The business focused language is very repelling as well. I'm not sure whether I'm too simple minded and incompetent or just live in a completely different world.
Business processes have everything to do with architecture. You've said it yourself:
"Data comes from some place and then gets processed and displayed or consumed."
If you don't factor in who that data comes from and why, and who's consuming it and why, then you don't have all the information you need to be a good architect.
"Data comes from some place and then gets processed and displayed or consumed."
If you don't factor in who that data comes from and why, and who's consuming it and why, then you don't have all the information you need to be a good architect.
With business processes I rather meant things related to strategic decision making and so on and not domain modelling and I/O processes. Those are obviously related to engineering.
But the advice under the business category is rather generic and strategic, what to buy, what to focus on etc. I don't see how any of that is related to domain modelling, I/O etc.
Also I'm not an architect, I'm a programmer. I literally don't understand what an architect is as stated above, especially not when looking at this document.
But the advice under the business category is rather generic and strategic, what to buy, what to focus on etc. I don't see how any of that is related to domain modelling, I/O etc.
Also I'm not an architect, I'm a programmer. I literally don't understand what an architect is as stated above, especially not when looking at this document.
> Start simple
Proceeds to write thousand words.
Jokes aside:
https://martinfowler.com/articles/designDead.html#TheValueOf...
Proceeds to write thousand words.
Jokes aside:
https://martinfowler.com/articles/designDead.html#TheValueOf...
If the extra details simplify the contents, go for it. Why should simplicity be conflated with conciseness?
"I would have written a shorter letter, but I did not have the time." -- Blaise Pascal
This sort of writing is great, it gives you a docs-in-a-box that's so important for selling to and pitching giant companies.
> business IT architecture
> Reuse before Buy, Buy before Build
Are there still giant companies anymore that don't author things for their own needs?
Anyone who can afford to write and adopt a literal playbook, written this long, digested at such a slow speed - anyone who has a team of people who disseminate software playbooks will have full time development staff.
Anyway, if you're a software developer, read some new fiction first before reading this book.
> business IT architecture
> Reuse before Buy, Buy before Build
Are there still giant companies anymore that don't author things for their own needs?
Anyone who can afford to write and adopt a literal playbook, written this long, digested at such a slow speed - anyone who has a team of people who disseminate software playbooks will have full time development staff.
Anyway, if you're a software developer, read some new fiction first before reading this book.
The pdf version is 67 pages, hardly something only a giant company could afford to read, and the rest of this comment falls apart once you notice this detail.
> Creating a MVP should take not more than 1 month.
Depending on your domain, 1 month is not enough to even understand the problem.
> Reuse before Buy, Buy before Build
The accidental complexity to integrate w/ a 3rd party solution can sometimes surpass that of building your own solution.
I call BS on dogmatic rules like that. Good architecture is analysing the requirements at hand and explicitly articulating and weighting the trade-offs.
Depending on your domain, 1 month is not enough to even understand the problem.
> Reuse before Buy, Buy before Build
The accidental complexity to integrate w/ a 3rd party solution can sometimes surpass that of building your own solution.
I call BS on dogmatic rules like that. Good architecture is analysing the requirements at hand and explicitly articulating and weighting the trade-offs.
> The accidental complexity to integrate w/ a 3rd party solution can sometimes surpass that of building your own solution.
This is one of the most common and damaging mistakes I see. Here are some other hidden costs that are often invisible to people:
1. The cost of understanding the 3rd party solution, which increases with the generality of the tool. That is, you are using a tool which solves A, B, C, and D to solve A. Using a well-documented tool helps here, but doesn't eliminate the problem.
2. The cost of upgrades, especially when forced by a security issue.
3. Framework vs library costs. It usually much less costly to import pure functions (think: an assertion library) versus importing a framework which forces a bespoke way of doing things (think: rspec).
This is one of the most common and damaging mistakes I see. Here are some other hidden costs that are often invisible to people:
1. The cost of understanding the 3rd party solution, which increases with the generality of the tool. That is, you are using a tool which solves A, B, C, and D to solve A. Using a well-documented tool helps here, but doesn't eliminate the problem.
2. The cost of upgrades, especially when forced by a security issue.
3. Framework vs library costs. It usually much less costly to import pure functions (think: an assertion library) versus importing a framework which forces a bespoke way of doing things (think: rspec).
I can't agree with you and the GP enough. 3rd party software is hell to integrate!
We can at least hope that in a science fiction future there will be a standardized container abstraction for delivering working software, with well-defined conventions regarding an admin interface for setup, an operator interface to investigate issues, and a user interface, which we make as reliable, comprehensive, trustworthy, and buttery smooth as we can make it.
We can at least hope that in a science fiction future there will be a standardized container abstraction for delivering working software, with well-defined conventions regarding an admin interface for setup, an operator interface to investigate issues, and a user interface, which we make as reliable, comprehensive, trustworthy, and buttery smooth as we can make it.
I agree that integrating a 3rd party solution has multiple hidden cost.
> This is one of the most common and damaging mistakes I see
Interestingly, the most common mistake I see is people neglecting the cost of developing a solution themselves:
1. opportunity cost
2.long term maintenance
3. documentation & training new people
...
> This is one of the most common and damaging mistakes I see
Interestingly, the most common mistake I see is people neglecting the cost of developing a solution themselves:
1. opportunity cost
2.long term maintenance
3. documentation & training new people
...
Agreed rolling your own has hidden costs too and is often not the right choice.
Ime, though, this lesson has almost achieved meme status. So while both types of mistakes are made, the "hidden costs of 3rd party tools" is often not a consideration at all. The node "left pad string" debacle is the archetypal example here.
Ime, though, this lesson has almost achieved meme status. So while both types of mistakes are made, the "hidden costs of 3rd party tools" is often not a consideration at all. The node "left pad string" debacle is the archetypal example here.
> I call BS on dogmatic rules like that. Good architecture is analysing the requirements at hand and explicitly articulating and weighting the trade-offs.
I particularly like your point above. I apply that same perspective to several other domains. For example, purchasing a new phone, choosing a new tool for work or personal use, purchasing a car.
It's easy to make decisions based on personal preference or, like you mentioned, dogmatic rules and call them "the right way of doing things".
I particularly like your point above. I apply that same perspective to several other domains. For example, purchasing a new phone, choosing a new tool for work or personal use, purchasing a car.
It's easy to make decisions based on personal preference or, like you mentioned, dogmatic rules and call them "the right way of doing things".
I agree, but it also fairly easily to be dogmatic overanalyzing things instead of executing.
Especially early stage, you have limited knowledge what the requirements of a product that will stick might be.
I think considering cost of future change and try to keep it low as a general rule is more important, than trying to make all the right choices from the get go for something you don't yet understand. You might have a good understanding of the problem, but that doesn't mean your solution will work and you have to adapt.
Especially early stage, you have limited knowledge what the requirements of a product that will stick might be.
I think considering cost of future change and try to keep it low as a general rule is more important, than trying to make all the right choices from the get go for something you don't yet understand. You might have a good understanding of the problem, but that doesn't mean your solution will work and you have to adapt.
> I call BS on dogmatic rules like that. Good architecture is analysing the requirements at hand and explicitly articulating and weighting the trade-offs.
This is quite the straw-man argument. For example, the section [1] whose title you quoted explicitly calls out the need to perform an evaluation first:
> When functionality is required, existing IT assets in the organization must be evaluated and used first, unless they do not exist and/or are a significant mismatch to the required functionality.
The "book" is clearly a draft-quality rough collection of notes, many of which are not fleshed out extensively, but it's quite unfair to consider it as a set of "dogmatic rules".
[1] https://nocomplexity.com/documents/arplaybook/businessarchit...
This is quite the straw-man argument. For example, the section [1] whose title you quoted explicitly calls out the need to perform an evaluation first:
> When functionality is required, existing IT assets in the organization must be evaluated and used first, unless they do not exist and/or are a significant mismatch to the required functionality.
The "book" is clearly a draft-quality rough collection of notes, many of which are not fleshed out extensively, but it's quite unfair to consider it as a set of "dogmatic rules".
[1] https://nocomplexity.com/documents/arplaybook/businessarchit...
> When functionality is required, existing IT assets in the organization must be evaluated and used first (...)
Why must be evaluated and used first? Will I even have time to evaluate? What am I optimising for?
Rules like "do this, do that", departed from the actual problems and the constraints, are by definition ignoring 100% of the data - that's why I call it dogmatic.
I would appreciate more if it at least supported these claims w/ examples or links to studies, but it doesn't.
Why must be evaluated and used first? Will I even have time to evaluate? What am I optimising for?
Rules like "do this, do that", departed from the actual problems and the constraints, are by definition ignoring 100% of the data - that's why I call it dogmatic.
I would appreciate more if it at least supported these claims w/ examples or links to studies, but it doesn't.
I guess I agree the word "must" is maybe too far, but I'd hardly go so far as to say it satisfies the requirements for "dogmatic BS".
> Depending on your domain, 1 month is not enough to even understand the problem.
I ran across this too in brief skimming (There's a lot there) but I assumed it was talking about building the MVP after doing the learning needed (in as much as you ever fully understand most domains) and designing of the MVP system.
I ran across this too in brief skimming (There's a lot there) but I assumed it was talking about building the MVP after doing the learning needed (in as much as you ever fully understand most domains) and designing of the MVP system.
If you follow Lean (where "Minimum Viable Product" originates from), there's no hard split between design/implementation phases, since the whole concept of MVP is avoiding waste in building something nobody wants.
The keyword is "avoiding waste", not "build something in 1 month". You can still waste 1mo work pretending to be fast and shipping crap, that's not the point.
The time a Minimum Viable Product will take is still completely determined by the requirements (definition of "viable", budget, experience, regulations, etc).
The keyword is "avoiding waste", not "build something in 1 month". You can still waste 1mo work pretending to be fast and shipping crap, that's not the point.
The time a Minimum Viable Product will take is still completely determined by the requirements (definition of "viable", budget, experience, regulations, etc).
From one of the greatest articles I've ever read ( https://iism.org/article/why-are-ceos-failing-software-engin... )
> When discovering new value, drive to value; when scaling discovered value, drive to date. Meaning: don’t gate value discovery by setting arbitrary release dates.
In effect, a release date is nothing more than saying to the team, “we are going to give up on discovery of the actual needed value by <insert arbitrary date here>.” Of course, that’s nonsense.
Also, dates don’t increase the rate of learning from failure; they in fact have the opposite effect, the team will be much less likely to discover the value needed as they will actually increasingly fear failure as the date approaches, and they will therefore slow down the rate at which they are breaking things, which effectively puts an end to new learning – no more bugs found, no more value found.
If the customers need hasn’t been met yet, it’s just a huge waste and confusing to all involved to release something that doesn’t actually meet the need. It’s even worse to release something by an arbitrary date that has the appearance of meeting the need but is riddled with business risking- or life risking- levels of chaos.
> When discovering new value, drive to value; when scaling discovered value, drive to date. Meaning: don’t gate value discovery by setting arbitrary release dates.
In effect, a release date is nothing more than saying to the team, “we are going to give up on discovery of the actual needed value by <insert arbitrary date here>.” Of course, that’s nonsense.
Also, dates don’t increase the rate of learning from failure; they in fact have the opposite effect, the team will be much less likely to discover the value needed as they will actually increasingly fear failure as the date approaches, and they will therefore slow down the rate at which they are breaking things, which effectively puts an end to new learning – no more bugs found, no more value found.
If the customers need hasn’t been met yet, it’s just a huge waste and confusing to all involved to release something that doesn’t actually meet the need. It’s even worse to release something by an arbitrary date that has the appearance of meeting the need but is riddled with business risking- or life risking- levels of chaos.
The build vs buy question should definitely never go one way or another dogmatically, it's context-specific. Buy for parity, build for competitive advantage. Only within the context of the business can you decide.
Also, sometimes that third party solution will go out of business if its proprietary or it may simply stop getting code patches/updates if its an open source one.
In the former case, contracts will help to some extent to mitigate the danger but you will still be stuck with using a legacy technology.
I would say one of the best ways to understand a domain is to rapidly prototype and test in it. One month to me would be the absolute longest.
I don't see MVP as necessarily what you launch with, but what you need to test with to prove market viability and reduce risk when spending more money/time.
I don't see MVP as necessarily what you launch with, but what you need to test with to prove market viability and reduce risk when spending more money/time.
MVP is a term that is tossed around a lot but few people actually use as the author intended.
If you seek the literature where the term "MVP" comes from, you'll see 1) MVP is not the same as prototype (you can validate an idea on the market without even a prototype), and 2) there's no definition of "too long", it's situational.
> As I talked about in a previous interview, IMVU’s original MVP took us six months to bring to market. That was a pretty big improvement over a previous company, where we spent almost five years before launching. Yet in another situation we spent two weeks building a particular feature that absolutely nobody wanted. In retrospect, two weeks was way too long. We could have found out that nobody wanted the product a lot sooner. At a minimum, a simple AdWords smoke test would have revealed how utterly bad the concept was. [1]
[1] https://leanstartup.co/what-is-an-mvp/
If you seek the literature where the term "MVP" comes from, you'll see 1) MVP is not the same as prototype (you can validate an idea on the market without even a prototype), and 2) there's no definition of "too long", it's situational.
> As I talked about in a previous interview, IMVU’s original MVP took us six months to bring to market. That was a pretty big improvement over a previous company, where we spent almost five years before launching. Yet in another situation we spent two weeks building a particular feature that absolutely nobody wanted. In retrospect, two weeks was way too long. We could have found out that nobody wanted the product a lot sooner. At a minimum, a simple AdWords smoke test would have revealed how utterly bad the concept was. [1]
[1] https://leanstartup.co/what-is-an-mvp/
I took the playbook with a grain of salt. It can be used as guidelines rather than a roadmap.
I'm curious why this is at the top of HN? My initial thoughts:
1. Title's use of "architecture" is overly vague, but they do define it somewhat in the intro. It still doesn't seem to overlap with the main bulk of the HN community.
2. Authors aren't listed up front which is frustrating for anyone that has spent time with enterprise work.
3. It's part of a site called "nocomplexity.com". This is frustrating for anyone who's been around the industry for some time because there is a lot of material (good and bad) on these issues. "Complex architecture" and "complicated architecture" often get used interchangeably when the terms mean different things. Many domains are necessarily more complex than others, but you can make a complicated solution to a less complex domain. It's even noted in the Zen of Python.
1. Title's use of "architecture" is overly vague, but they do define it somewhat in the intro. It still doesn't seem to overlap with the main bulk of the HN community.
2. Authors aren't listed up front which is frustrating for anyone that has spent time with enterprise work.
3. It's part of a site called "nocomplexity.com". This is frustrating for anyone who's been around the industry for some time because there is a lot of material (good and bad) on these issues. "Complex architecture" and "complicated architecture" often get used interchangeably when the terms mean different things. Many domains are necessarily more complex than others, but you can make a complicated solution to a less complex domain. It's even noted in the Zen of Python.
Anyway to get the entire thing in pdf instead of just the current page/section?
Github repository says you can download it on ePub or PDF formats, but I also haven't found how.
The "IT Responsibility" and "Ease of Use" statements are incredibly pithy rewordings of "don't make me think, do what I mean".
Whoever wrote this thinks that infrastructure systems come about by businessperson a one way royal decree, not actual engineering processes that are interactive.
Whoever wrote this thinks that infrastructure systems come about by businessperson a one way royal decree, not actual engineering processes that are interactive.
To download ePub or PDF, open the menu on this page and the link is in the bottom:
https://architectureplaybook.readthedocs.io/en/latest/
I appreciate the comments guys so I don't dive deep into this lol
Discussed recently here: https://news.ycombinator.com/item?id=24298235
I think it's a good list of things to take in consideration.
I wouldn't take it as a dogma but it's definitely a good guide to use as starting point.
As an aside: anyone know what this documentation was made with? this looks like it would be great for dev docs.
Looks like JupyterBooks[0]: https://jupyterbook.org/intro.html
[0]: https://github.com/nocomplexity/ArchitecturePlaybook#help
[0]: https://github.com/nocomplexity/ArchitecturePlaybook#help
> nocomplexity.com
nope
nope
2. Use a RDBMS
3. Use a CDN
4. Let your telemetry guide you on how to scale, cache, or split your workloads
5. Don't hire people who want to boil the ocean
Keep things simple, fast, and efficient.