Don't microservice, do module(yekta.dev)
yekta.dev
Don't microservice, do module
https://yekta.dev/posts/dont-microservice-do-module/
162 comments
As I’ve grown older I have developed a heuristic, which is that people who talk/write like this, i.e. being completely convinced that one way of doing things is always wrong and one way of doing things is always right, and that people doing things the other way are just ignorant/stupid/wrong, usually have no idea what they are talking about.
The best advice I got early in my career on this topic was: “Nobody serious is going to take you seriously if you give one option as the ‘way it has to be.’ Everything has trade-offs, you will be much more effective as a communicator if you list a few options, along with their trade-offs (and be honest about these). You can even have your preferred / recommended option but it also must come with trade-offs.”
One of the problems I often see is that when trying to explain one of the possible options, people don't understand it. When that happens, you try to explain it to them harder. Your explanation is then mistaken for pushing for that option. It's important to carefully navigate around this perception. People should understand that first you just want to be on the same page what your options are, and then discuss the best one.
My approach to that has always been to make sure I'm calling my assumptions with a possible option, usually that's either the use cases we care about or the scaling/infrastructure concerns.
I learned pretty early on that its really easy for that context to be lost and that miscommunication go missed. Its so much easier to catch miscommunication early when a few extra seconds or minutes are spent calling out the context and assumptions that make one option best.
I learned pretty early on that its really easy for that context to be lost and that miscommunication go missed. Its so much easier to catch miscommunication early when a few extra seconds or minutes are spent calling out the context and assumptions that make one option best.
This is good sometimes, other times it risks losing the audience. Extra info works better when they're ready (i.e. asked for it). This is why it's such a difficult balancing act. You don't want to preemptively answer questions that people didn't realize they should ask yet.
For sure, I was mainly thinking about later stage discussions where decisions like tech stack, architecture, or infrastructure should really be made.
Early on its often all about use cases and users you're trying to reach. If microservices versus monoliths becomes much of a debate that early engineering is probably getting too far ahead of the product IMO.
Early on its often all about use cases and users you're trying to reach. If microservices versus monoliths becomes much of a debate that early engineering is probably getting too far ahead of the product IMO.
I usually have my assumptions written, so I can flip to them quickly if there is any confusion
Very true. It can be hard to properly articulate a position without feeling like you’re being too aggressive, I struggle with this a lot.
Well, you mileage can vary. I see trade-off almost everywhere but many managers are not exited to hear about trade-offs and want to get a perfect solution when you don’t need to sacrifice anything. Though it is not as bad as a manager who already made a decision ignoring trade-offs of different options.
Unfortunately there are a lot of non-serious people in the industry.
I do this instinctively but I feel like I'm coming across as confused and unsure.
That's how I give advice and it can be a huge struggle when working with people out of their element, or in a job they frankly didn't earn.
It's really the only way a productive design conversation can go, but those who can't contribute from their side of the fence will get frustrated. Saying things like "you just need to tell me what to do".
On the flip side, those who think they know better will have pretty much already made up their mind. Which makes things even worse if they don't know better, but for the competent ones this is the easiest and most effective setup of them all.
It's really the only way a productive design conversation can go, but those who can't contribute from their side of the fence will get frustrated. Saying things like "you just need to tell me what to do".
On the flip side, those who think they know better will have pretty much already made up their mind. Which makes things even worse if they don't know better, but for the competent ones this is the easiest and most effective setup of them all.
I agree with you. And at the same time, I often feel like it is more difficult being heard when being nuanced. It seems like what gets discussed most are strong opinions.
Yep. It’s easier for a simple message to be carried by the wind.
But the process of growing up is one of increasing capacity for discernment. Ie, you learn more subtlety discerning when thing A or B is a better idea in any given moment. Will a hard or soft approach work better? Use my old tools or learn this new framework? Make a long term or short term decision here?
It’s hard to communicate because this kind of learning takes a lifetime to accumulate.
But the process of growing up is one of increasing capacity for discernment. Ie, you learn more subtlety discerning when thing A or B is a better idea in any given moment. Will a hard or soft approach work better? Use my old tools or learn this new framework? Make a long term or short term decision here?
It’s hard to communicate because this kind of learning takes a lifetime to accumulate.
The hardest part is when you're being nuanced and people misconstrue it as you being indecisive xD
could you give an example? my approach is usually something like "I've come up with three options here, I think the first two are equally good, I'm mentioning the third for completeness, but I don't think we should do it because...."
Time is limited, why waste time talking about a third when you've already decided against it?
Who’s to say he’s necessarily right? The third approach (or pieces from it) could actually be the right one, even if he doesn’t know it.
You have a point, but that's also where you come off as indecisive. Since the question was explicitly about that that, presenting 3 options, one of which you have reasons against, when we're all busy and meeting time is constrained, is, in the abstract, a waste of everybody's time. If later on, someone comes up with objections; options A won't work because problem X, option B has issues Y and Z, then sure, bring up option C, which addresses X and Y but has other issues, for further debate, but unless that happens, that's time wasted. imo.
This does hinge on you knowing what you're talking about, and rejecting option C for unbiased reasonable logical reasons you're sure about.
This does hinge on you knowing what you're talking about, and rejecting option C for unbiased reasonable logical reasons you're sure about.
Maybe we need to work on Sound bites
- The only thing every project has in common is that they are all different projects.
- Success in one project doesn't guarantee success in another.
- The only thing every project has in common is that they are all different projects.
- Success in one project doesn't guarantee success in another.
My approach is to have strong opinions (weakly held), and ask if people have objections to the tradeoffs. That tends to keep the focus on specific reasons to choose a given path rather than you and someone else just having different preferences. Doesn't always work, but it's a lot easier than fighting over whether option A or option B is just universally better.
I've definitely seen that happen, and in my opinion its just a sign of bad culture or bad leadership. That's not to say its toxic or widespread, maybe its just a poorly run meeting, but nuance should be a focus of any important discussion rather than the voice that goes ignored.
Depends on your audience. Maybe you need to dumb it down for some people sometimes. That's life. I just try to stay in situations where the audience appreciates nuance if I can
Yeah, so it presents a real conundrum. If you read the article, then he still presents arguments in favor of microservices.
> I often feel like it is more difficult being heard when being nuanced. It seems like what gets discussed most are strong opinions.
I really resent this phenomenon. It traps us in poor local maxima because our systems optimize for engagement over actual development of complex, nuanced opinions. It feels like the dopamine-addled end up indirectly pulling the levers on how we talk even if they're less interested in the actual craft.
> I often feel like it is more difficult being heard when being nuanced. It seems like what gets discussed most are strong opinions.
I really resent this phenomenon. It traps us in poor local maxima because our systems optimize for engagement over actual development of complex, nuanced opinions. It feels like the dopamine-addled end up indirectly pulling the levers on how we talk even if they're less interested in the actual craft.
The author did provide an exception:
> Unless your specific use case demands the unique advantages of microservices, it is wiser to stick with a well-structured monolith.
The author also has a section in the article called “When should you consider using microservices?”.
> Unless your specific use case demands the unique advantages of microservices, it is wiser to stick with a well-structured monolith.
The author also has a section in the article called “When should you consider using microservices?”.
I feel the same way about people who are "Everything needs to be containerized in K8s". Sure, it's useful in some cases, but it isn't some universal panacea.
Understand the problem space, understand the solution space, and try to find a fit, don't just grab your favorite hammer and start whacking away.
Understand the problem space, understand the solution space, and try to find a fit, don't just grab your favorite hammer and start whacking away.
The k8s evangelists are operating under the assumption that if your product or service is successful, you're probably going to want to scale it, and if you believe in that vision, then you're probably going to want to think about k8s early in your project, not when you've grown it to millions of users worldwide.
But I get what you mean.
But I get what you mean.
That assumption that you will want to scale does not always hold;
The assumption that you want to scale into millions of users do not always hold;
The assumption that you need kubernetes to scale into millions of users (or that you will even want it) does not always hold;
The assumption that solving that problem from the start leads to a better outcome than waiting that the problem appears has "winning the lottery" odds of not being the opposite of the truth.
The assumption that you want to scale into millions of users do not always hold;
The assumption that you need kubernetes to scale into millions of users (or that you will even want it) does not always hold;
The assumption that solving that problem from the start leads to a better outcome than waiting that the problem appears has "winning the lottery" odds of not being the opposite of the truth.
The amount of assumptions involved in all these really suggests it's mostly rationalization doesn't it?
It's absolutely perfectly fine to scale to millions of users with yesterday's proven simple tech using the benefits of time passed. You got everything that was hard before in managed services, why start complicating the simplest part (stateless app servers) now that we've got everything served on a platter?
I reckon 99% of the apps HN talks about that needs to "scale" can do it with VMs behind a load balancer, a cache, and a relational database. All of these are now available as managed services. Not to mention that you now got tailor made languages for it like Go that compiles to a simple single binary for your VMs to just kick off. Use the progress instead of trying to be clever. I mean, if actually producing a product is the goal, rather than a vehicle to unnecessarily squeeze in the latest tech into. There's no need to make a big up-front investment in building a platform on top of k8s etc. Spend the time on the product instead. And if you turn out to be in the 1% that's a good problem to have, and no, it won't be over night, so you're fine.
It's absolutely perfectly fine to scale to millions of users with yesterday's proven simple tech using the benefits of time passed. You got everything that was hard before in managed services, why start complicating the simplest part (stateless app servers) now that we've got everything served on a platter?
I reckon 99% of the apps HN talks about that needs to "scale" can do it with VMs behind a load balancer, a cache, and a relational database. All of these are now available as managed services. Not to mention that you now got tailor made languages for it like Go that compiles to a simple single binary for your VMs to just kick off. Use the progress instead of trying to be clever. I mean, if actually producing a product is the goal, rather than a vehicle to unnecessarily squeeze in the latest tech into. There's no need to make a big up-front investment in building a platform on top of k8s etc. Spend the time on the product instead. And if you turn out to be in the 1% that's a good problem to have, and no, it won't be over night, so you're fine.
I wouldn't say it's rationalization.
Surviving the initial scaling is an horrible experience, full of technical gotchas. People that have seen that will naturally want to avoid it. The problem is that it's an irrational desire, because what they do to avoid it destroys most of their chances to even get there.
It looks like a manifestation of the second system syndrome.
Surviving the initial scaling is an horrible experience, full of technical gotchas. People that have seen that will naturally want to avoid it. The problem is that it's an irrational desire, because what they do to avoid it destroys most of their chances to even get there.
It looks like a manifestation of the second system syndrome.
Could be indeed. I've also found it interesting how rarely I hear about actual proper load testing. Like, it doesn't need to be a horrible experience, and it can be done in a controlled setting before even launching if one wants to. But this doesn't really seem to be that common, just by judging how rarely it's brought up in the pop-software dev sphere.
I know you are advocating that position. But, I have seen lots of failed projects because of that thinking. They failed to get to customer #1 because they were building something to handle customer #1,000,000.
Unless you are building something that can kill people if it goes wrong just get it out the door and be as disciplined as you can along the way. If it is successful then you have the money to split it up into a SoA when you need it.
Unless you are building something that can kill people if it goes wrong just get it out the door and be as disciplined as you can along the way. If it is successful then you have the money to split it up into a SoA when you need it.
There are some none obvious ways to kill people with bad API design even if you don’t work in a medical industry and don’t deal with heavy machinery. Corners being cut here and there, context gets lost, then it gets reused in a (non)obviously inappropriate and suboptimal way and somebody indeed dies.
Please take effort and do quality stuff as if your life depends on it.
Please take effort and do quality stuff as if your life depends on it.
There's a balance. Scope creep and solving engineering problems you don't have are obviously really bad, but too many people see the bell curve meme for tech stacks and lean into stupid. When building something, you want to take a step back and consider some of the most likely places that thing will evolve to, then architect the software to make those transitions/evolutions easy. That way even if you build the wrong thing, you don't have to start from scratch, instead you've created a ratchet of progress that makes each pivot faster and easier.
Oh yeah I agree.
That is why I put in their “be as disciplined as you can be along the way”. If it never ships what’s the point; but if you ship something that has to be rewritten right away it’s almost as bad.
That is why I put in their “be as disciplined as you can be along the way”. If it never ships what’s the point; but if you ship something that has to be rewritten right away it’s almost as bad.
> Understand the problem space, understand the solution space, and try to find a fit, don't just grab your favorite hammer and start whacking away.
People who understand the problem space will eventually notice that kubernetes isn't a hammer but whole toolbox. So you can reach for it when you need a hammer, a different hammer, or a screwdriver, or when you're not sure what you need, or when you're 90% sure what you need but also willing to admit you might be wrong.
Not a universal panacea, but if we're going to get into this "right tool for the job" discussion, it's a complete misunderstanding to characterize it as a single tool.
People who understand the problem space will eventually notice that kubernetes isn't a hammer but whole toolbox. So you can reach for it when you need a hammer, a different hammer, or a screwdriver, or when you're not sure what you need, or when you're 90% sure what you need but also willing to admit you might be wrong.
Not a universal panacea, but if we're going to get into this "right tool for the job" discussion, it's a complete misunderstanding to characterize it as a single tool.
It's not about everything fits, it's just much easier for orgs to have a consistent environment for developers to use. Google does this to great success, getting out of the box logging, metrics, a universal way to define a system, access secrets. A whole class of decisions stop being made all the time. Anyone in the org can look in a repo and understand what and where everything is deployed. It's simple stuff like this that make it compelling.
I.e. every architectural question in IT is answered by "It depends".
PS: And every single problem is a "problem in communication".
PS: And every single problem is a "problem in communication".
While true, but that’s totally devoid of information. People still need to make decisions. Even people who lack the wisdom to make them well.
That's really where "it depends" is important though. First the decisions need to be what matters and what to prioritize. With that context its much easier to decide on the best technical implementation for your needs.
The problem is conversations often end up being "we're starting this shiny new project, what tech stack should we use?"
What is really needed is a shortlist of priorities, from scaling concerns to types of users and how frequently content/data may change. Without that its just a grab bag of tools people are familiar with and the latest hype trend.
The problem is conversations often end up being "we're starting this shiny new project, what tech stack should we use?"
What is really needed is a shortlist of priorities, from scaling concerns to types of users and how frequently content/data may change. Without that its just a grab bag of tools people are familiar with and the latest hype trend.
Same. Basically an inverse relationship between certainty and knowledge. Sort of like those stories told by entrepreneurs about why their startups were successful.
Yep, this post is awful in many aspects, throwing rants here and there w/o ever making a point.
I have no idea how this got so many upvotes!!.
It got upvoted because microservices are currently sliding into the trough of disillusionment. It's trendy right now to hate on them, and the author is fully on the mindless hate bandwagon right along with a lot of HN readers.
In a few years we'll hopefully be out onto the slope of enlightenment, with microservices applied where they're useful and not applied where they're not. If we don't get there, then we'll just run the whole hype cycle over again with yet another rebrand of the same concept.
In a few years we'll hopefully be out onto the slope of enlightenment, with microservices applied where they're useful and not applied where they're not. If we don't get there, then we'll just run the whole hype cycle over again with yet another rebrand of the same concept.
I dunno man, I worked at FB well before the micro services hype and saw a bunch of problems with them, particularly in debugging.
And in general, putting a network boundary between function calls is gonna add a whole bunch of complexity.
That being said, splitting services so that teams could deploy independently definitely also had a lot of benefits at FB, but I could never understand why so many much smaller companies took the micro services approach.
And in general, putting a network boundary between function calls is gonna add a whole bunch of complexity.
That being said, splitting services so that teams could deploy independently definitely also had a lot of benefits at FB, but I could never understand why so many much smaller companies took the micro services approach.
I'm not into the microservices hype either, I'm just opposed to the reactionary claims in places like TFA that you should basically never split out code into a new service. Both extremes are wrong.
My opinion is that the default should be to keep things in one service and only split them out if there's a very good technical or organizational case to be made.
My opinion is that the default should be to keep things in one service and only split them out if there's a very good technical or organizational case to be made.
I've worked on quite a few defining products with a friend, business partner, and all-round designer-engineer genius. About a decade ago, we decided to answer, especially in the early stages, that anything that needed our opinion was - "It depends."
There isn't one silver bullet for most problems or situations; it all depends on several factors.
There isn't one silver bullet for most problems or situations; it all depends on several factors.
My only real question when I have interviewed anyone for a senior level role or similar is to find out whether they'll start with "it depends" or similar.
I'll lay out what sounds like a pretty clear algorithm or infrastructure scenario but skip some key assumptions that are needed. What I want to see is follow-up questions related to users, scaling, data retention, etc. Its a big red flag for me if a candidate jumps right into a solution.
I'll lay out what sounds like a pretty clear algorithm or infrastructure scenario but skip some key assumptions that are needed. What I want to see is follow-up questions related to users, scaling, data retention, etc. Its a big red flag for me if a candidate jumps right into a solution.
Your comment reads like you did not read beyond the title. In the article they provide analysis and contexts in which microservices are usable. Article does not fit in your description of "ignorant" or "completely convinced of one way".
> Let’s do a 1:1 comparison of microservices and modules. Spoiler alert: the argument favors modules, as there is little to be said in support of microservices when pitted against modules!
This is the introduction to their argument and it definitely accurately captures the tone of the rest of the article. As for ignorance, you get frequent sections like this:
> With a monolithic architecture, your system is either UP or DOWN, with no in-between. With microservices, you need to monitor every service. All the services need to be UP, and they need to be able to communicate with each other, all for you to be able to say the system is UP. If even one out of your 888 services is down, the system can no longer be called UP!
The author manages to take one of the most compelling uses of microservices and turn it into a negative thing. Somehow in their mind switching from a model where you're either UP or DOWN to a model where you can be partially UP is worse, which suggests to me that they don't have very much experience operating either type of architecture. Managing an incident during a partial outage is infinitely less stressful than managing a complete outage. Whether you can officially label the entire system as UP is immaterial in a real ops context if the impact of the isolated service that is DOWN is minimal.
This is the introduction to their argument and it definitely accurately captures the tone of the rest of the article. As for ignorance, you get frequent sections like this:
> With a monolithic architecture, your system is either UP or DOWN, with no in-between. With microservices, you need to monitor every service. All the services need to be UP, and they need to be able to communicate with each other, all for you to be able to say the system is UP. If even one out of your 888 services is down, the system can no longer be called UP!
The author manages to take one of the most compelling uses of microservices and turn it into a negative thing. Somehow in their mind switching from a model where you're either UP or DOWN to a model where you can be partially UP is worse, which suggests to me that they don't have very much experience operating either type of architecture. Managing an incident during a partial outage is infinitely less stressful than managing a complete outage. Whether you can officially label the entire system as UP is immaterial in a real ops context if the impact of the isolated service that is DOWN is minimal.
In smaller/earlier projects, there may not be such a thing as partially up or partially down. Every module/service can be on the critical path. And if that's the case, it's easier to not be dealing with a distributed system (or to have a minimally distributed one).
I mean, yeah, don't use microservices if you're in a small or early project. That's a totally valid argument that I 100% agree with. Microservices solve a specific category of problems that you don't have until you reach a certain scale (either organizationally or operationally).
But again, this is their thesis statement:
> Let’s do a 1:1 comparison of microservices and modules. Spoiler alert: the argument favors modules, as there is little to be said in support of microservices when pitted against modules!
Their essay leaves very little room for legitimate technical reasons for a service to be split out from the monolith. Even their section titled "When Should You Consider Using Microservices?" basically boils down to "if you already have microservices, if you absolutely must use a different language, or if you're an irresponsible idiot".
But again, this is their thesis statement:
> Let’s do a 1:1 comparison of microservices and modules. Spoiler alert: the argument favors modules, as there is little to be said in support of microservices when pitted against modules!
Their essay leaves very little room for legitimate technical reasons for a service to be split out from the monolith. Even their section titled "When Should You Consider Using Microservices?" basically boils down to "if you already have microservices, if you absolutely must use a different language, or if you're an irresponsible idiot".
Oh yeah I'm not gonna defend the article. I was only really responding to this line.
> Somehow in their mind switching from a model where you're either UP or DOWN to a model where you can be partially UP is worse, which suggests to me that they don't have very much experience operating either type of architecture.
> Somehow in their mind switching from a model where you're either UP or DOWN to a model where you can be partially UP is worse, which suggests to me that they don't have very much experience operating either type of architecture.
Ah, got it. Yeah, I can imagine someone coming to their conclusion if the only system they've ever worked with doesn't have any major components that aren't on the critical path.
The only right answer when asked "should we use (X)" is "it depends."
I really dislike using react and can easily say off-hand that people just shouldn't use it, but that's because I also dislike working on exactly the types of projects that react is a good fit for. There's a time and a place for everything, if not then why would anyone have bothered to build and maintain the thing?
I really dislike using react and can easily say off-hand that people just shouldn't use it, but that's because I also dislike working on exactly the types of projects that react is a good fit for. There's a time and a place for everything, if not then why would anyone have bothered to build and maintain the thing?
I can agree halfway, thing is many times "it depends" is such a non answer. like it always stops with "it depends", not "in this case you do x, in this case you prioritize other things and you do Y"
To quote the movie Four Rooms: "The less a man makes declarative statements, the less apt he is to look foolish in retrospect."
> being completely convinced that one way of doing things is always wrong and one way of doing things is always right, and that people doing things the other way are just ignorant/stupid/wrong
> people who talk/write like this usually have no idea what they are talking about
According to your heuristic, you have no idea what you're talking about?
In all seriousness though, your comment comes across as unnecessarily personal. I disagree with the OP's conclusion, but he raises some interesting arguments on an interesting topic. I came to the comments to see a technical discussion around the pros/cons of microservices. It's off-putting to see that the top comment is an ad-hominem attack
> people who talk/write like this usually have no idea what they are talking about
According to your heuristic, you have no idea what you're talking about?
In all seriousness though, your comment comes across as unnecessarily personal. I disagree with the OP's conclusion, but he raises some interesting arguments on an interesting topic. I came to the comments to see a technical discussion around the pros/cons of microservices. It's off-putting to see that the top comment is an ad-hominem attack
I think you might have misread OP. The first quote you extracted isn't their description of the author, it's their description of what the author is conveying in TFA, and that's honestly a pretty accurate way to capture the piece. TFA's section giving the reasons why you might consider a microservice architecture basically boils down to "if you already have microservices, if you absolutely must use a different language, or if you're an irresponsible idiot". That's not the attitude of someone who is open to the idea that someone reasonable might make a different choice than they would.
What OP actually has to say about the author themselves is that people who write like that "usually have no idea what they are talking about", which is also absolutely true in this case. There are many tells throughout the essay that give away that the author has very little experience operating any type of system, microservices or otherwise.
What OP actually has to say about the author themselves is that people who write like that "usually have no idea what they are talking about", which is also absolutely true in this case. There are many tells throughout the essay that give away that the author has very little experience operating any type of system, microservices or otherwise.
Yup. Anybody that doesn't understand that most problems are complicated doesn't understand most problems.
Yep, I used to be that guy as well. It is easier to see things as black and white because then you don’t need to dive deep and it is easier to sell to management. The really strong engineers/architects I have known can explain the trade offs of the various approaches.
For small teams or early stage development I love a well architected modular monolith. Grow past a couple of teams and maybe add some dedicated ops people and service oriented architectures start making more sense. Get real big and have the capacity to handle the monitoring and orchestration challenges and microservices enable that.
It’s all trade offs and recognizing that allows you to evolve with your operational and organizational needs.
For small teams or early stage development I love a well architected modular monolith. Grow past a couple of teams and maybe add some dedicated ops people and service oriented architectures start making more sense. Get real big and have the capacity to handle the monitoring and orchestration challenges and microservices enable that.
It’s all trade offs and recognizing that allows you to evolve with your operational and organizational needs.
I much prefer "service oriented architecture" terminology to "microservice" because services need not be the same size. Have a daemon listening to a queue to send emails? Sure, microservice! Have 10 REST endpoints that are backed by the same interconnected data? Sure you could have 10 lambdas or something but.. why?
Yeah I try to use “service oriented architecture” in discussions because it implies as small as necessary instead of as small as possible. I find it reframes the discussion a bit and encourages a better set of initial domain boundaries.
While I work on things that are large and need to scale well I don’t work at the hyper scaler level which is where I think the “as small as possible” type services are more commonly needed. But, that is just speculation on my part from reading lessons learned whitepapers from those companies
While I work on things that are large and need to scale well I don’t work at the hyper scaler level which is where I think the “as small as possible” type services are more commonly needed. But, that is just speculation on my part from reading lessons learned whitepapers from those companies
[deleted]
Only a sith deals in absolutes
You are absolutely spot on.
Yeah, I am also suspicious about people with radical opinions. Truth is USUALLY somewhere not at the end of scale.
BUT, have you noticed yours is also a radical opinion?
The reality is that sometimes, a radical opinion is actually a correct one.
Which is to say heuristics are useful but they still do not replace critical thinking.
The problem with Microservices is actually that IMO most developers simply have no time, willingness, experience or mental capacity to think critically about all that stuff. People frequently need to make decisions efficiently. The theory of efficient decisionmaking I have is that frequently enough it is more important to make a decision than to make a perfect decision.
And it kind of makes sense because I also suspect majority of the population (and that includes developers) are simply unable to think critically or retrospect about their own performance.
And what you do when you have little experience in the field and can't yet think critically about things? You use training wheels.
In case of IT (and business in general), a powerful training wheel is imitation.
So what happens is that somebody at some company will publish a paper about how they solved a problem and suddenly a bunch of people will try to jump on that bandwagon ("you can't go wrong if they succeeded with it") but without the hassle of having to actually think through it, understand what were the circumstances at that other company, how those circumstances are different from own use case, etc. They will imitate what others have done before but without realising a lot of important things about the problem. Which is how we got to this whole microservices mess.
Anyway, I am personally rolling back microservices implementations pretty much every project I join. People do not realise how much time they spend solving problems that are simply due to the fact they have partitioned large application into hundreds of small services which need to each be maintained separately. We have dedicated teams to do stuff that is simply unnecessary (they usually call them "devops", but really they are just ops because more frequently than not they are not actively developing the functionality). All the performance problems, all that inefficiency usually vanishes when you roll all that functionality into a single application and just scale that one application instance over multiple servers.
My current team is even more funny. The microservices were originally meant to allow teams to work independently, but at my current team they work hard to bind all of the development process into a single stream of work. So there is some 80 people in 10 different teams all working on same set of environments, applications, with the same monthly release process, coordinating their work everywhere. But there is about 1 service maintained for each developer which means people spend half the time dealing with complex configuration. And the other half of the time figuring out how to improve performance of an application which copies all of its data from service to service.
BUT, have you noticed yours is also a radical opinion?
The reality is that sometimes, a radical opinion is actually a correct one.
Which is to say heuristics are useful but they still do not replace critical thinking.
The problem with Microservices is actually that IMO most developers simply have no time, willingness, experience or mental capacity to think critically about all that stuff. People frequently need to make decisions efficiently. The theory of efficient decisionmaking I have is that frequently enough it is more important to make a decision than to make a perfect decision.
And it kind of makes sense because I also suspect majority of the population (and that includes developers) are simply unable to think critically or retrospect about their own performance.
And what you do when you have little experience in the field and can't yet think critically about things? You use training wheels.
In case of IT (and business in general), a powerful training wheel is imitation.
So what happens is that somebody at some company will publish a paper about how they solved a problem and suddenly a bunch of people will try to jump on that bandwagon ("you can't go wrong if they succeeded with it") but without the hassle of having to actually think through it, understand what were the circumstances at that other company, how those circumstances are different from own use case, etc. They will imitate what others have done before but without realising a lot of important things about the problem. Which is how we got to this whole microservices mess.
Anyway, I am personally rolling back microservices implementations pretty much every project I join. People do not realise how much time they spend solving problems that are simply due to the fact they have partitioned large application into hundreds of small services which need to each be maintained separately. We have dedicated teams to do stuff that is simply unnecessary (they usually call them "devops", but really they are just ops because more frequently than not they are not actively developing the functionality). All the performance problems, all that inefficiency usually vanishes when you roll all that functionality into a single application and just scale that one application instance over multiple servers.
My current team is even more funny. The microservices were originally meant to allow teams to work independently, but at my current team they work hard to bind all of the development process into a single stream of work. So there is some 80 people in 10 different teams all working on same set of environments, applications, with the same monthly release process, coordinating their work everywhere. But there is about 1 service maintained for each developer which means people spend half the time dealing with complex configuration. And the other half of the time figuring out how to improve performance of an application which copies all of its data from service to service.
There’s a Persian proverb, which is a shorter form of a poem by Saib Tabrizi, which says "don't microservice". We don’t live forever, do we? Laying the first brick correctly is the first step to avoid building a crooked wall.
While I hate the cargo culting of microservices, there is one use case they handle very well - reusable services with obnoxious dependencies that don't really need to be integrated in with the rest of your codebase. I would much rather take a docker image of a whisper or text-to-speech service and add it to my compose file and call it via HTTP than try to wrestle a CUDA build and get whisper/TTS bindings working for every language where I might need them.
A similar case for me has been processing large files. It's really nice to be able to scale up the memory of the file processing server when there are large files being processed, they scale it back down for normal usage
Microservices can also be a great fit for certain team/org structures. Its not ideal to pick the tech based on the team structure, but that's life.
I've also found it micro services helpful for heavy data processing projects. A micro service architecture driven by events as data is processed and moved through different steps can be a surprisingly simple setup to reason about and debug.
I've also found it micro services helpful for heavy data processing projects. A micro service architecture driven by events as data is processed and moved through different steps can be a surprisingly simple setup to reason about and debug.
Microservices are not solving a technical problem. This is related to another issue: many technical people think tech is the hard part. Technology is the easy part. If you think the tech is hard, you have no idea how hard The Rest is. I know it sucks, but the sooner you learn this the better.
So true. Communication is insanely difficult. Hiring competent, motivated engineers is insanely difficult. Working around short-term thinking executives is insanely difficult. Iterating until you stumble on a success product is insanely difficult.
How often does anyone talk about these problems in meetings? It's wild, I feel like all I ever do is argue over mostly irrelevant technical minutia.
How often does anyone talk about these problems in meetings? It's wild, I feel like all I ever do is argue over mostly irrelevant technical minutia.
welcome to bike shedding
https://en.wikipedia.org/wiki/Law_of_triviality
https://en.wikipedia.org/wiki/Law_of_triviality
Exactly! It's this huge elephant in the room and all we want to talk about is our editors and how Java is faster than Go or whatever.
I'm sitting in this meetings thinking "Guys, using XML or JSON truly, truly does not matter here. You have already wasted months on this shit and the value meter is still at $0. I'm telling you we can make this work with either Scala or 6502 assembly or anything in between. What matters is that we need a coherent vision, formulate actionable goals and work on improving our communication and keeping it there."
Maybe I need to go into management? (/s ?)
I'm sitting in this meetings thinking "Guys, using XML or JSON truly, truly does not matter here. You have already wasted months on this shit and the value meter is still at $0. I'm telling you we can make this work with either Scala or 6502 assembly or anything in between. What matters is that we need a coherent vision, formulate actionable goals and work on improving our communication and keeping it there."
Maybe I need to go into management? (/s ?)
It's almost as if you are describing the meetings in my company (and my previous)
I think focusing on technical trivialities is a symptom of what type of experience the developers have and how the current workplace is organized.
In traditional companies, the developers are at the tail of a process that they have limited participation in. Most of the product decisions has been done by others and the developers often don't have enough contact with the business side to make any real impact.
You end up with an isolated group that can only influence technical desicions, so that's what they will focus on.
Choosing tech X vs tech Y is actually only 1 choice in a long chain of product, design and business choices that has led to the Jira ticket. From their point of view, it seems like the most important desicion because it is taken out of context.
This is where experience is actually a good thing, because it increases the chance that you have been exposed to "the other side". They will also know that these types of decisions are not likely to be the reason why they fail.
By participating from the beginning you also tend to be more motivated by outcome. Technical discussions that don't have a real impact become less interesting.
I think focusing on technical trivialities is a symptom of what type of experience the developers have and how the current workplace is organized.
In traditional companies, the developers are at the tail of a process that they have limited participation in. Most of the product decisions has been done by others and the developers often don't have enough contact with the business side to make any real impact.
You end up with an isolated group that can only influence technical desicions, so that's what they will focus on.
Choosing tech X vs tech Y is actually only 1 choice in a long chain of product, design and business choices that has led to the Jira ticket. From their point of view, it seems like the most important desicion because it is taken out of context.
This is where experience is actually a good thing, because it increases the chance that you have been exposed to "the other side". They will also know that these types of decisions are not likely to be the reason why they fail.
By participating from the beginning you also tend to be more motivated by outcome. Technical discussions that don't have a real impact become less interesting.
This resonates with me, although I would go a step further to say the makeup/experience of the developers is a product of the competency of leadership. Most leaders wind up in their positions out of pure luck and chicanery. I can't tell you how many lay offs I've been through where the best people are let go and the worst are kept on and unintentionally sabotage the entire engineering team.
focusing on technical trivialities is a symptom of what type of experience the developers have and how the current workplace is organized
It comes from development echo chambers (including this one, sorry HN), where people tend to discuss The Right Way even if they see it not working at their own job, cause right ways feel good. New/young developers absorb it and bring it to their workplace. I did that too. It took decades to beat this shit out of myself. Now when I hear someone talking about true ways of a qualified professional developer I just remember that the business will probably pivot a couple of times before they deliver an mvp that is neither v or p.
It comes from development echo chambers (including this one, sorry HN), where people tend to discuss The Right Way even if they see it not working at their own job, cause right ways feel good. New/young developers absorb it and bring it to their workplace. I did that too. It took decades to beat this shit out of myself. Now when I hear someone talking about true ways of a qualified professional developer I just remember that the business will probably pivot a couple of times before they deliver an mvp that is neither v or p.
I for one would love to have a boss that cares about those things.
Trying to solve a communication problem with more code and infra is often a mistake. This is something many engineers don't realize they're essentially doing. Some communication problems can be somewhat alleviated by processes and technology enforcing those processes, but you must always consider the cost, and weigh it against alternatives that don't require engineering overhead. In almost every case like this, increasing the amount of code and infrastructure is the wrong option.
Yes, but Conway is not wrong
Yes, but his law is descriptive, not prescriptive. Just because it tends to happen doesn't mean it should happen, or happens for good reasons.
But it means it happens. So you can either weaponize it or get steam rolled by it.
Not wrong, but also not a Law.
It depends on what kind of tech we are talking about. Making a truly good and flexible tech product is truly hard. There will always be someone arguing YAGNI at inappropriate moments, influencing design negatively in ways, that will usually never be overthrown again, because of the additional mindset of not wanting rewrites and mindset of not wanting to change a (halfassed) working system.
I always thought microservices did solve a technical problem, but one that only appears when you're the size of Twitter or similar. I would agree with you for the huge majority of companies that don't build their own datacenters.
It's an organizational pattern. I've seen it being used at very small companies to good effect. It matched their (dysfunctional) culture: islands, little communication, etc. Working on a monolith together is a nightmare if you hate each other.
> If you think the tech is hard, you have no idea how hard The Rest is.
If people/orgs really believed this, they'd keep headcount as minimal as possible.
If people/orgs really believed this, they'd keep headcount as minimal as possible.
Don't they? The only experience I've had where this isn't the case is startup mania, where gasoline is poured over open flame.
It really depends on where you work I suspect. Some places incentivize having lots of direct reports and hiring can outpace the ability to dole out work. Currently some of those places seem to be seeing how few developers they actually need to keep the lights on. In both cases the decision isn't made from a technical POV but more just following the current fashions.
Many vogue tech decisions are over-engineered band-aids for organizational dysfunction.
The author seems to be assuming that microservices are only used to break down a single application into different services, in which case their arguments are valid. If you instead consider that a single service can in fact be used by multiple applications, most of their arguments are plain wrong.
For example: >Why would you ever want to allocate more resources to one particular part? It’s not like the other parts will eat up the extra resources. If your system needs more RAM, it needs more RAM. Why would you care about which part needs more RAM?
For example: >Why would you ever want to allocate more resources to one particular part? It’s not like the other parts will eat up the extra resources. If your system needs more RAM, it needs more RAM. Why would you care about which part needs more RAM?
There's a benefit of microservices that I don't hear talked about. At my workplace, there's absolutely no respect for api boundaries. Devs will monkey patch and reach into other module's internals without a second thought. I can't imagine the monstrosity that would exist if the network boundary wasn't there to stop them ...
Microservices (like a lot of things) might just be there to solve human developer problems as much as anything else.
There are times where I'm all "No the authentication service is doing great, leave it alone!!!".
There are times where I'm all "No the authentication service is doing great, leave it alone!!!".
>Devs will monkey patch and reach into other module's internals without a second thought. I can't imagine the monstrosity that would exist if the network boundary wasn't there to stop them ...
We have a tool in our modular monolith, which is like a linter which checks architectural violations like that. If someone tries to bypass the API layer of a module and peek directly into the internals, it refuses to build (and CI/CD refuses to deploy/release). No need to replace a basic linter with a network boundary...
We have a tool in our modular monolith, which is like a linter which checks architectural violations like that. If someone tries to bypass the API layer of a module and peek directly into the internals, it refuses to build (and CI/CD refuses to deploy/release). No need to replace a basic linter with a network boundary...
We've encountered the exact same problem in the past, and open sourced our solution to enforce those boundaries! https://github.com/gauge-sh/tach Alongside CODEOWNERs, you can effectively constrain teams to focus on where they need to be.
We can enforce module boundaries in code with tooling.
An example for Python: https://github.com/gauge-sh/tach
An example for Python: https://github.com/gauge-sh/tach
You don't need network level separation for that, UNIX sockets and pipes are just as effective at enforcing those boundaries.
What's stopping them from violating network boundaries?
Typically teams won't have access to the code that's being deployed on the other side of the network boundary. This physical separation is overkill though; something like CODEOWNERs and https://github.com/gauge-sh/tach can do this without incurring remotely the same overhead.
the network
That's just another way of saying that microservices don't solve a technical problem, they solve an organizational problem.
yes - and writing code also doesn't solve a technical problem, a computer runs fine with 0s and 1s, no need for code, in fact code produces a performance overhead - but it solves a human abilities problem, namely that 0s and 1s aren't particularly expressive for us. Almost everything we do solves for communication, some things on an individual scale, like programming languages, and other things on an organisational scale like micro-services (I'm also not blindly advocating for micro-services, but I do get why if done somewhat competently it's easier to organise a team of 1000 developers into 50 teams of 20 each responsible for one service rather than 1000 developers responsible for everything)
My intuition is that at a certain scale, microservices are preferable to a monolith. Two principled reasons:
- Autonomy over deployment rollbacks. If your team owns a microservice, and you noticed that you deployed buggy code, you can easily and autonomously do a rollback. Your rollback doesn't impact other teams. It isn't visible company-wide. This allows teams to take more risks with fewer downsides and more psychological safety
- Isolation of secrets, api-keys etc. If one module has access to an api-key, then it's effectively made visible to every single module in the monolith. Other teams can decide to bypass your API entirely and read directly from your database, making it impossible for you to abstract away your implementation details
Problems solved by microservices that could in theory be solved in a monolith:
- Build/test times. A big monolith will have a huge build time and a large number of CI/CD tests. In theory you could parallelize your build/test-suite to make it infinitely fast, but I'm not sure how practical this is
- IDE slowness. A monolith will make your average IDE run much more slowly. In theory also solvable, but would require major IDE changes and investments
- Code ownership by specific teams. In theory you can use things like regex-CODEOWNERs to enforce code-ownership of individual modules, but it's easier with microservices
- Migrating to better languages/frameworks. If your original monolith was written in perl or on a bad framework, it's very hard to move away from it. With microservices, each microservice can be independently and incrementally migrated, allowing the company's tech stack to evolve gracefully.
Any others I missed?
- Autonomy over deployment rollbacks. If your team owns a microservice, and you noticed that you deployed buggy code, you can easily and autonomously do a rollback. Your rollback doesn't impact other teams. It isn't visible company-wide. This allows teams to take more risks with fewer downsides and more psychological safety
- Isolation of secrets, api-keys etc. If one module has access to an api-key, then it's effectively made visible to every single module in the monolith. Other teams can decide to bypass your API entirely and read directly from your database, making it impossible for you to abstract away your implementation details
Problems solved by microservices that could in theory be solved in a monolith:
- Build/test times. A big monolith will have a huge build time and a large number of CI/CD tests. In theory you could parallelize your build/test-suite to make it infinitely fast, but I'm not sure how practical this is
- IDE slowness. A monolith will make your average IDE run much more slowly. In theory also solvable, but would require major IDE changes and investments
- Code ownership by specific teams. In theory you can use things like regex-CODEOWNERs to enforce code-ownership of individual modules, but it's easier with microservices
- Migrating to better languages/frameworks. If your original monolith was written in perl or on a bad framework, it's very hard to move away from it. With microservices, each microservice can be independently and incrementally migrated, allowing the company's tech stack to evolve gracefully.
Any others I missed?
One that I don't think a monolith can solve is partial availability.
A monolith is either available or it isn't, which is fine as long as you can guarantee that nothing ever takes down the monolith, but we all know that that's not something you can guarantee.
If you split out services that live outside the critical path, then when those services go down you can still engineer your system to keep going and gracefully degrade the functionality provided by those currently-down services.
This makes incident response much less stressful because there's less urgency, which in turn leads to fewer mistakes during the response and faster recovery times.
(Yes, there are some pieces of functionality that will always be in the critical path, like authorization, but many things don't have to be.)
A monolith is either available or it isn't, which is fine as long as you can guarantee that nothing ever takes down the monolith, but we all know that that's not something you can guarantee.
If you split out services that live outside the critical path, then when those services go down you can still engineer your system to keep going and gracefully degrade the functionality provided by those currently-down services.
This makes incident response much less stressful because there's less urgency, which in turn leads to fewer mistakes during the response and faster recovery times.
(Yes, there are some pieces of functionality that will always be in the critical path, like authorization, but many things don't have to be.)
This is why I like having things split up. Even with zero service redundancy, it massively improves the reliability of the system overall. Often users don't even notice that a less important service had to restart because of an issue. If that service was just another component of a monolith, then it failing would guarantee that the whole monolith fails.
In my experience you’ll be fine with monolith instances and failover. Probably until you’re Google scale
Monolith plus failover gets you better availability, not partial availability. Code errors or network problems or many other types of errors can still take down both the main instance and failover unless you're perfect at preventing them, and when that happens you have zero availability until you fix it.
Not really. Even with a monolith, you get partial availability. Most often as a default.
Imagine a web server handling requests. It usually has a way to recover from a crashed request handler. On top of that, you have multiple web servers behind a load balancer. Some code paths may be failing, some may be available.
Then, you have some background workers, maybe ingesting data from third party services. They're completely different processes from your web servers, even though they still use the monolithic code base. If they're down, the web app can still work with stale data.
The partial availability can be compromised in microservices as well as in a monolith (e.g. auth as you mentioned).
Imagine a web server handling requests. It usually has a way to recover from a crashed request handler. On top of that, you have multiple web servers behind a load balancer. Some code paths may be failing, some may be available.
Then, you have some background workers, maybe ingesting data from third party services. They're completely different processes from your web servers, even though they still use the monolithic code base. If they're down, the web app can still work with stale data.
The partial availability can be compromised in microservices as well as in a monolith (e.g. auth as you mentioned).
> Imagine a web server handling requests. It usually has a way to recover from a crashed request handler. On top of that, you have multiple web servers behind a load balancer. Some code paths may be failing, some may be available.
This is a form of partial availability but not what I'm talking about. I'm talking about a scenario where a DoS attack or an initialization error or an external service timing out causes the entire service (all instances) to fail to respond to any requests. Past a certain scale of app these things do happen periodically, and if you've split out non-critical code into separate services and handled for graceful degradation that code can no longer bring down the critical paths.
> They're completely different processes from your web servers, even though they still use the monolithic code base.
I... wouldn't call these part of a monolith, though they're not exactly microservices either. They're separately deployed units of code that happen to exist in a monorepo. This gives them similar resilience characteristics to microservices, so they're not a counterexample, they're more of a subcategory of what I'm talking about.
> The partial availability can be compromised in microservices as well as in a monolith (e.g. auth as you mentioned).
This is the "but sometimes" fallacy. If solution B allows for graceful degradation 80% more often than solution A, it's still better at graceful degradation even if it doesn't reach 100% success.
This is a form of partial availability but not what I'm talking about. I'm talking about a scenario where a DoS attack or an initialization error or an external service timing out causes the entire service (all instances) to fail to respond to any requests. Past a certain scale of app these things do happen periodically, and if you've split out non-critical code into separate services and handled for graceful degradation that code can no longer bring down the critical paths.
> They're completely different processes from your web servers, even though they still use the monolithic code base.
I... wouldn't call these part of a monolith, though they're not exactly microservices either. They're separately deployed units of code that happen to exist in a monorepo. This gives them similar resilience characteristics to microservices, so they're not a counterexample, they're more of a subcategory of what I'm talking about.
> The partial availability can be compromised in microservices as well as in a monolith (e.g. auth as you mentioned).
This is the "but sometimes" fallacy. If solution B allows for graceful degradation 80% more often than solution A, it's still better at graceful degradation even if it doesn't reach 100% success.
Google famously still primarily maintains a monorepo, but they absolutely don't deploy a monolith. They published a paper last year that describes an approach that is quite close to what they have internally - https://dl.acm.org/doi/pdf/10.1145/3593856.3595909
Effectively, by setting strong boundaries between modules, they can develop on their application as if it was a monolith, but at runtime, separate those modules physically deploy them as micro-services. This solves a lot of the versioning issues (since the app is rolled out atomically) and correctness issues (easier to reason through a monorepo).
Effectively, by setting strong boundaries between modules, they can develop on their application as if it was a monolith, but at runtime, separate those modules physically deploy them as micro-services. This solves a lot of the versioning issues (since the app is rolled out atomically) and correctness issues (easier to reason through a monorepo).
Modulith is the underrated happy middle ground imo.
There certainly are many cases where microservices are used where they are not actually appropriate, but this entire post just betrays a lack of experience working in a large monolithic codebase with many engineers.
Separation is a critical part, and that could be modules that form a separate microservice, or split off into it's own "macroservice", the mechanics are the same.
I've seen many cases where there is a service that does like 3 things. 2 of them are low importance and one of them super critical. A low importance one will have a bug, crash, introduce latency regression, etc and take down the critical one.
Time to separate the critical one into it's own deployment to isolate and improve reliability.
The other is on trade-offs, super important that every decision makes has them, just respect that and realize the answer is often "it depends".
I've seen many cases where there is a service that does like 3 things. 2 of them are low importance and one of them super critical. A low importance one will have a bug, crash, introduce latency regression, etc and take down the critical one.
Time to separate the critical one into it's own deployment to isolate and improve reliability.
The other is on trade-offs, super important that every decision makes has them, just respect that and realize the answer is often "it depends".
I wish we had more articles not about what good practices are, or how things should be designed in theory, but rather about how to enforce those practices in a large organization, when you are not necessary a guy who makes decisions.
Unfortunately, the "wisdom of the tribe" cannot be written down for the laymen.
Most juniors can't look at a design-pattern, and somehow immediately understand why it is good or inappropriate.
Almost all recent grads are similar to the following fellow:
https://en.wikipedia.org/wiki/Eddie_the_Eagle
Most juniors can't look at a design-pattern, and somehow immediately understand why it is good or inappropriate.
Almost all recent grads are similar to the following fellow:
https://en.wikipedia.org/wiki/Eddie_the_Eagle
> when you are not necessary a guy who makes decisions
I think a lot of it comes down to: if the guys who do make decisions make them badly, stuff is going to suck. You need the right people at the top.
I think a lot of it comes down to: if the guys who do make decisions make them badly, stuff is going to suck. You need the right people at the top.
Probably the author never worked on large projects that need to scale. Almost no one of his objections does make sense. Imagine Facebook, Uber, AirbNB, Google as monoliths.
Tell this to Instagram or WhatsApp before acquisition. ;-) They have scaled well enough, IMHO.
On the other hand, I agree that for the larger teams, microservices make sense. It's much easier when you can roll back a failed deployment and block new releases just for your service, while the team tries to fix it. Also, monitoring and resource accounting is easier. All at the cost of some inefficiencies compared to a monolith.
On the other hand, I agree that for the larger teams, microservices make sense. It's much easier when you can roll back a failed deployment and block new releases just for your service, while the team tries to fix it. Also, monitoring and resource accounting is easier. All at the cost of some inefficiencies compared to a monolith.
His LinkedIn is on the article. He has 3 years of professional/non academic experience so you're highly likely to be correct.
OTOH I only know micro services from 5-20 dev teams where they made no sense whatsoever.
Yeah but all those companies invest an absurd amount into tolling to make this practical, and they AFAIK don't do insane things like having separate databases for individual services.
> Microservices are the Wrong Answer
The author never states the question. This makes it impossible to evaluate the answer, which presumably is stated in the title. The article would be much stronger with a clear statement of the perceived problem that microservices solve and why modules are a superior solution.
The author never states the question. This makes it impossible to evaluate the answer, which presumably is stated in the title. The article would be much stronger with a clear statement of the perceived problem that microservices solve and why modules are a superior solution.
I’ve yet to see an article which explores the pros and cons of different strategies and demonstrates each with examples rather than picks one and insists it’s better for arbitrary reasons.
I’ve used microservices, monoliths, serverless, and other similar permutations based around different core architectures (event buses, peer to peer, client-server). Some aspects of each were awesome. Some weren’t. Knowing why and how/when to leverage each is far more interesting than demoting or excluding a solution because you like another one more.
There are some suspicious ideas in here. “If your system needs more RAM, it needs more RAM. Why would you care about which part needs more RAM?” I suppose. But what if I only spin up a service once per day, and it needs far more ram than the rest of my system? I should use those resources for the 23 hours of the day that I don’t need them and pay for them nonetheless? It I don’t own my hardware, that’s an insane waste of money. Sometimes ephemeral services are a huge cost saver even for smaller teams… And it isn’t strictly due to poor communication. It’s a legitimate challenge presented by resource and finance constraints.
“Microservices introduce a lot of overhead in terms of communication latency” - sometimes this is perfectly acceptable. Say you have jobs you need done, and sometimes it goes from ten at a time to perhaps 100. A monolith would get bogged down severely. The client’s expectation of having the job done is likely not instantaneous if it’s something that needs to be sent to a queue. In this case, ephemeral job runners make sense. They can take on however many jobs they can at once, and if they’re maxed out, spawn another worker. This does not work well with monoliths in my experience. The communication overhead doesn’t matter here, as it could be worse if the monolith is bogged down while working anyway.
There’s more I don’t agree with, but I don’t claim to be a seer or anything. Maybe I’m just old and incapable of learning (though I do like the module approach as well).
I’ve used microservices, monoliths, serverless, and other similar permutations based around different core architectures (event buses, peer to peer, client-server). Some aspects of each were awesome. Some weren’t. Knowing why and how/when to leverage each is far more interesting than demoting or excluding a solution because you like another one more.
There are some suspicious ideas in here. “If your system needs more RAM, it needs more RAM. Why would you care about which part needs more RAM?” I suppose. But what if I only spin up a service once per day, and it needs far more ram than the rest of my system? I should use those resources for the 23 hours of the day that I don’t need them and pay for them nonetheless? It I don’t own my hardware, that’s an insane waste of money. Sometimes ephemeral services are a huge cost saver even for smaller teams… And it isn’t strictly due to poor communication. It’s a legitimate challenge presented by resource and finance constraints.
“Microservices introduce a lot of overhead in terms of communication latency” - sometimes this is perfectly acceptable. Say you have jobs you need done, and sometimes it goes from ten at a time to perhaps 100. A monolith would get bogged down severely. The client’s expectation of having the job done is likely not instantaneous if it’s something that needs to be sent to a queue. In this case, ephemeral job runners make sense. They can take on however many jobs they can at once, and if they’re maxed out, spawn another worker. This does not work well with monoliths in my experience. The communication overhead doesn’t matter here, as it could be worse if the monolith is bogged down while working anyway.
There’s more I don’t agree with, but I don’t claim to be a seer or anything. Maybe I’m just old and incapable of learning (though I do like the module approach as well).
Honestly I never really understood the hatred that people on HN have for microservices. Everyone acts like "OMG THEY'RE SO COMPLEX!!!!", but a) that's not unique to microservices, b) monoliths aren't somehow immune from this complexity, and c) most big systems end up getting pretty complicated regardless of the model.
Yes, Kubernetes is a pain in the ass, and libraries getting out of sync can be real problem, and distributed systems are inherently difficult, but I think that the benefits of something like event-sourcing with lots of individual, self-contained, "I don't care where this data came from" services outweigh a lot of the complaints. It's not like most of us are reinventing Paxos every day; we're using well-tested off the shelf components like Kafka and Consul and RabbitMQ.
Also, I don't really understand the author's point about individual scaling not being a good thing. It can be very hard to measure things like memory usages for individual components of a huge service; the smaller and more atomic the service is, the easier it is to find out how many resources it actually needs. With a big ol' monolith, it seems like people just go off intuition (AKA self-righteous guessing).
I dunno, even for home servers, I still basically do the microservice model. Maybe I'm weird.
Yes, Kubernetes is a pain in the ass, and libraries getting out of sync can be real problem, and distributed systems are inherently difficult, but I think that the benefits of something like event-sourcing with lots of individual, self-contained, "I don't care where this data came from" services outweigh a lot of the complaints. It's not like most of us are reinventing Paxos every day; we're using well-tested off the shelf components like Kafka and Consul and RabbitMQ.
Also, I don't really understand the author's point about individual scaling not being a good thing. It can be very hard to measure things like memory usages for individual components of a huge service; the smaller and more atomic the service is, the easier it is to find out how many resources it actually needs. With a big ol' monolith, it seems like people just go off intuition (AKA self-righteous guessing).
I dunno, even for home servers, I still basically do the microservice model. Maybe I'm weird.
> I never really understood the hatred that people on HN have for microservices.
If you're an early stage company and just trying to ship something with a small team, the overhead of microservices (testing, deployment, monitoring, platform infra such as k8s, etc) can be pretty obnoxious (like an order of magnitude more time than the work to actually deliver business value). And microservices give you enough rope for one overly clever person to hang the whole team, so there are definitely some traumatized developers out there.
If you are doing something simple enough, throw it in a single docker container and spend your time focused on building something useful.
If you're an early stage company and just trying to ship something with a small team, the overhead of microservices (testing, deployment, monitoring, platform infra such as k8s, etc) can be pretty obnoxious (like an order of magnitude more time than the work to actually deliver business value). And microservices give you enough rope for one overly clever person to hang the whole team, so there are definitely some traumatized developers out there.
If you are doing something simple enough, throw it in a single docker container and spend your time focused on building something useful.
This is all true, but I think tends to cover up root-causes.
Over the last 5-10 years, tons of people got jobs on the strength of "k8s experience" which was more like "I was on the software-side of an application development that was deployed with kubernetes once" and then got hired based on that as if they had architecture and infrastructure experience/expertise, when they actually don't have either one.
So yeah, when you need 6-8 months of on-the-job training for your "experts" to get some expertise, that will set you back 6-8 months when it comes to productionizing your product (assuming the product itself is already finished!). It's not the sheer amount of work that's involved with microservices vs monoliths. It's the skill set, which is large enough that you can't get it overnight. Even setting aside the wider ecosystem and just counting the number core concepts at play, it's easier to pick up a new programming language than to learn kubernetes architecture/administration/operations.
Startups (or a company of any size that simply doesn't have the ability to evaluate candidates) will be better off if they won't saddle themselves with full-time people of questionable qualifications, working instead with some limited-engagement technical consultants and contractors in a tactical way. Contractors like this will have actually solved the problems involved multiple times and will probably have bootstrap-a-whole-org testing/deployment/monitoring code in their back pocket already.
To me it seems like all this is the predictable downside of the whole "full stack engineer" thing. A server-side language + html/css/javascript can probably fit into the average human brain, but even there it seems misguided. Once "full stack" is further twisted to mean SWE+Infrastructure+Platform+Cloud+Architecture and then you throw in a little "No-Ops" and "Low Code" optimism, then it's getting unrealistic, but naive management will imagine anyone can do anything and it's all very easy. It is easy, but not anyone can do it. Expertise is required, especially during design/bootstrap phases. As a company you don't have to buy that expertise forever with a FTE, but you also can't just assume you'll multipurpose some existing FTE for this stuff if they have less than adequate experience.
Over the last 5-10 years, tons of people got jobs on the strength of "k8s experience" which was more like "I was on the software-side of an application development that was deployed with kubernetes once" and then got hired based on that as if they had architecture and infrastructure experience/expertise, when they actually don't have either one.
So yeah, when you need 6-8 months of on-the-job training for your "experts" to get some expertise, that will set you back 6-8 months when it comes to productionizing your product (assuming the product itself is already finished!). It's not the sheer amount of work that's involved with microservices vs monoliths. It's the skill set, which is large enough that you can't get it overnight. Even setting aside the wider ecosystem and just counting the number core concepts at play, it's easier to pick up a new programming language than to learn kubernetes architecture/administration/operations.
Startups (or a company of any size that simply doesn't have the ability to evaluate candidates) will be better off if they won't saddle themselves with full-time people of questionable qualifications, working instead with some limited-engagement technical consultants and contractors in a tactical way. Contractors like this will have actually solved the problems involved multiple times and will probably have bootstrap-a-whole-org testing/deployment/monitoring code in their back pocket already.
To me it seems like all this is the predictable downside of the whole "full stack engineer" thing. A server-side language + html/css/javascript can probably fit into the average human brain, but even there it seems misguided. Once "full stack" is further twisted to mean SWE+Infrastructure+Platform+Cloud+Architecture and then you throw in a little "No-Ops" and "Low Code" optimism, then it's getting unrealistic, but naive management will imagine anyone can do anything and it's all very easy. It is easy, but not anyone can do it. Expertise is required, especially during design/bootstrap phases. As a company you don't have to buy that expertise forever with a FTE, but you also can't just assume you'll multipurpose some existing FTE for this stuff if they have less than adequate experience.
So do you believe that microservices are no harder than monoliths for small-scale projects, it's just a matter of expertise?
Pretty much, yeah.
Look at it this way. If microservice madness needs N monitors, well, monolith still needs at least 1 right? With the relevant expertise, making N monitors isn’t harder than making 1. Programmers shouldn’t get scared about generalizing existing instructions and adding a loop, and infracode is just another kind of code or you’re doing it wrong.
Rejecting microservices because of this kind of “complexity” is usually just FUD that is screening someone’s preference for not working, learning, or hiring. Which desires are ok I guess, but it’s annoying if it’s presented as a discussion around architecture, best practices, etc.
Look at it this way. If microservice madness needs N monitors, well, monolith still needs at least 1 right? With the relevant expertise, making N monitors isn’t harder than making 1. Programmers shouldn’t get scared about generalizing existing instructions and adding a loop, and infracode is just another kind of code or you’re doing it wrong.
Rejecting microservices because of this kind of “complexity” is usually just FUD that is screening someone’s preference for not working, learning, or hiring. Which desires are ok I guess, but it’s annoying if it’s presented as a discussion around architecture, best practices, etc.
While I don't agree with quite some assumptions and comparisons the author makes I tend to agree that microservices are often not the right answer for your problem.
They can be great when:
* You need just one function that is not directly related to your core application logic and might be needed by other service. Great usecase for a microservice or lambda
* You need to separate statefull and stateless components
* Have async workflows
* Scaling to infinity and beyond
But most don't application problems don't have these requirements for it's core logic. Microservices have a huge cost. Code and dependency dublication, complex deployments, latency, harder debugging and tracing and the cognitive load is much higher.
A lot of read blogs and new form netflix and google and want to do the same. The management of my current project is asking for microservices because it is the new hot sh*t, so teams are doing it just like SAFe and AI - even when it does not help to solve the problem.
They can be great when:
* You need just one function that is not directly related to your core application logic and might be needed by other service. Great usecase for a microservice or lambda
* You need to separate statefull and stateless components
* Have async workflows
* Scaling to infinity and beyond
But most don't application problems don't have these requirements for it's core logic. Microservices have a huge cost. Code and dependency dublication, complex deployments, latency, harder debugging and tracing and the cognitive load is much higher.
A lot of read blogs and new form netflix and google and want to do the same. The management of my current project is asking for microservices because it is the new hot sh*t, so teams are doing it just like SAFe and AI - even when it does not help to solve the problem.
Microservices are just Enterprise Java Beans reinvented, but without the efficiency.
At least with EJBs we had the option of bypassing the network layer and having tightly coupled services communicate directly in-process.
Someone looked at that and thought: “Too efficient!” and made the network step mandatory.
This is how we ended up with web apps like Jira that take a solid minute to open an empty form.
At least with EJBs we had the option of bypassing the network layer and having tightly coupled services communicate directly in-process.
Someone looked at that and thought: “Too efficient!” and made the network step mandatory.
This is how we ended up with web apps like Jira that take a solid minute to open an empty form.
> Why would you ever want to allocate more resources to one particular part? It’s not like the other parts will eat up the extra resources. If your system needs more RAM, it needs more RAM. Why would you care about which part needs more RAM?
Looks like resource isolation and avoiding a non performant piece of code to bring your whole system down is overrated.
Just scale vertically! Get a bigger instance!
Looks like resource isolation and avoiding a non performant piece of code to bring your whole system down is overrated.
Just scale vertically! Get a bigger instance!
> Just scale vertically! Get a bigger instance!
That works great until it suddenly and unexpectedly stops working.
That works great until it suddenly and unexpectedly stops working.
We use a microservice architecture at the company and like every second incident is a cascade failure which is actually harder to monitor and manage... And having microservices doesn't help.
Things like, someone forgot to batch the events when posting to the event bus so a large tenant generating many events suddenly overflowed RabbitMQ which ran out of memory and started blocking producers who are holding DB connections and so we're out of DB connections now and the whole thing goes down. The only difference is that when it's going down it's harder to understand who's calling who in all this mess (compared to a single beefy server).
Things like, someone forgot to batch the events when posting to the event bus so a large tenant generating many events suddenly overflowed RabbitMQ which ran out of memory and started blocking producers who are holding DB connections and so we're out of DB connections now and the whole thing goes down. The only difference is that when it's going down it's harder to understand who's calling who in all this mess (compared to a single beefy server).
Behold: the distributed monolith.
The main problem with adopting microservices is the hype and many times the person who defends it doesn't have enough data! "We'll rewrite this into technology X, because this doesn't scale", but sometimes anyone measured it, tested or doesn't have a good knowledge about. Another common pattern is if a module is buggy, you start to listen people screaming that this need to be a microservices. My basic thought is to write modules and when non core features evolve in such a way that the team needs to be splitted, demanding more allocated people, this features needed to move somewhere and at that point the projects will benefit!
The fault tolerance aspect is underplayed here. Different modules might have different reliability and uptime requirements. I don’t want an unreliable module without SLAs running on the same instance as something on the critical path for users.
Probably the most formative experience in my early career was seeing a massive (attempted) rewrite from a monolith to microservices. Multi-unicorn company spent a year and $Ms in developer hours just to realize that the network separation did more harm than good. There are use cases for network separation (fault tolerance, independent scaling, polyglot teams etc.) but for essentially every startup I've been at, the better bet is to enforce logical boundaries in the codebase itself.
Tbh, the best way to do this is likely the hybrid model adopted by, e.g., gitlab rails and Grafana Loki etc.. air usually let's the operator select if they want to run as a monolith, service with multiple components where some are instanced, and full micro service mode.
For small deployments you can just use the monolith and be done with a single "service" while you still have the ability to scale out for large deployments. This gets even better when they allow you to do an in between where one component does A and B while only C is handled by micro services.
For small deployments you can just use the monolith and be done with a single "service" while you still have the ability to scale out for large deployments. This gets even better when they allow you to do an in between where one component does A and B while only C is handled by micro services.
Having worked with several systems using both modules and micro services I would say that using modules is the way to go from a debugging and understandability perspective, so modules are a more productive method.
However micro services can still be valid but to me they are actually “developer UI” and should only be used when services have to be shared between developers who don’t know each other. For example a public API will almost always be an API with a micro service behind it
However micro services can still be valid but to me they are actually “developer UI” and should only be used when services have to be shared between developers who don’t know each other. For example a public API will almost always be an API with a micro service behind it
Microservice design is a solution to scaling up people in a large organization.
Regardless what HN people believe, some people like & proficient in a few languages that may not match the existing codebase. Eg. 50% of the team is Go enthusiasts in a Java/Python shop.
Microservices solve that. If your company is tiny, you don’t win much using microservices.
Regardless what HN people believe, some people like & proficient in a few languages that may not match the existing codebase. Eg. 50% of the team is Go enthusiasts in a Java/Python shop.
Microservices solve that. If your company is tiny, you don’t win much using microservices.
It's also an excellent way to create large organizations due to the complexity chain reaction it seems to kick off.
So I can just rename my "microservice" to "module" right? That is all the author seems to have done in my eyes, but then also thrown away the bits that make it scale - for no less complexity.
First of, I am going to disagree with the microservices.io definition the author references, because it is short-sighted and ends up contradicting itself when talking about assemblage. Chris Richardsons book is definitely worth a read, but we clearly come from different worlds when it comes to a core belief of microservice architecture.
Too many people who complain about microservices are clearly unaware that a microservice doesnt mean remote, or distributed. It can have local transport - even a simple function call - which is the same as the "modular" approach the author dictates - but is also more flexible in that you CAN have remote transport with no additional effort because the service locator handles it for you.
Its just building your code in a modular way, with a defined interface between isolated components, a service locator (this is effectively an index of your modules (microservices) and where they are located - local, remote, etc) and some mechanism to call them based on the supported transports (thread local, RPC, whatever) of that "module".
It can all reside in a single repo. Your microservices can all be compiled in a single app, your service locator can be a switch statement, and your transport can be a function call. The only requirement is that they never perform a task that another microservice can handle - and if we were just talking programming, we would just call that "modularization" or using DRY principles! Microservice architecture is just non-monolithic DRY.
Anyone NOT writing microservices that way is just making life hard for themselves and probably hasn't read even the most archaic book on microservice architecture. Remove the remote transports, and you have a well built, domain driven monolith...
First of, I am going to disagree with the microservices.io definition the author references, because it is short-sighted and ends up contradicting itself when talking about assemblage. Chris Richardsons book is definitely worth a read, but we clearly come from different worlds when it comes to a core belief of microservice architecture.
Too many people who complain about microservices are clearly unaware that a microservice doesnt mean remote, or distributed. It can have local transport - even a simple function call - which is the same as the "modular" approach the author dictates - but is also more flexible in that you CAN have remote transport with no additional effort because the service locator handles it for you.
Its just building your code in a modular way, with a defined interface between isolated components, a service locator (this is effectively an index of your modules (microservices) and where they are located - local, remote, etc) and some mechanism to call them based on the supported transports (thread local, RPC, whatever) of that "module".
It can all reside in a single repo. Your microservices can all be compiled in a single app, your service locator can be a switch statement, and your transport can be a function call. The only requirement is that they never perform a task that another microservice can handle - and if we were just talking programming, we would just call that "modularization" or using DRY principles! Microservice architecture is just non-monolithic DRY.
Anyone NOT writing microservices that way is just making life hard for themselves and probably hasn't read even the most archaic book on microservice architecture. Remove the remote transports, and you have a well built, domain driven monolith...
Thank you to everyone who contributed insightful thoughts and feedback on the subject. Your input has been invaluable, and I've taken the time to revise and improve the article.
This parody sums up the issues with mircoservices very well: https://www.youtube.com/watch?v=y8OnoxKotPQ
Don't take design advice in a vacuum. Think about the problem at hand, the constraints and the people and organizations involved, and design something appropriate to that context.
> Self-confidence is good. On the other hand, too much self-confidence means disaster. - His holiness the Dalai Lama
I guess it's true for microservices as well.
I guess it's true for microservices as well.
I would love the day when WASM + WASI fulfill their promise and let us write our modules against an interface so you can easily swap the underlying implementation without changing something in your app, this way I can care less if I'm running a microservice or a module, let that decision to the devops/management team.
C FFI and machine code already accomplish this. The reasons you wouldn't use C FFI and machine code are the same reasons you wouldn't use WASI and WASM.
Am I the only one who uses microservices to deploy small monolithic projects?
To be frank, this is a terrible advice. I’m certain this person, given enough time, will come to the same conclusion as he/she is exposed to different projects and organizations.
As many posters here state, there are many instances where services are loosely coupled, developed by different team and/or at different time, and makes no sense to make into a monolith.
As you may not benefit from shoehorning a well-maintained monolith into a microservice, the opposite is true.
As many posters here state, there are many instances where services are loosely coupled, developed by different team and/or at different time, and makes no sense to make into a monolith.
As you may not benefit from shoehorning a well-maintained monolith into a microservice, the opposite is true.
[deleted]
Not sure if there's any other profession than software that has such a hard time with proportionality. We can seemingly with a perfectly straight face argue that yes this massive Rube Goldberg machine was required to avoid that small nuisance that feels wrong but actually isn't something that consumes time at all.
Imagine if a carpenter or plumber acted like this; having self-fulfillment as prio 1, 2, 3 in all cases, not outcome. They'd never get hired again.
Imagine if a carpenter or plumber acted like this; having self-fulfillment as prio 1, 2, 3 in all cases, not outcome. They'd never get hired again.
I lol'd when I got to 'Why would you ever want to allocate more resources to one particular part? It’s not like the other parts will eat up the extra resources. If your system needs more RAM, it needs more RAM. Why would you care about which part needs more RAM?' and stopped reading.
got it, I will not build micro services, I will build just regular one.
Yet another self righteous person on the internet teaching us the errors of our thinking. Everything wrong with micro services is an issue with micro services and everything wrong with modules is a skil issue of the person doing the modules.
> Once the interface is established, each team can operate independently, just like in microservices, as long as they adhere to the defined bounds.
Where I was lost laughing.
I'm sorry, but the hords of spaghetti monsters that I battle on a daily bases are spawning because communication is hard the more people are involved in it and the more someone is in a hurry to deploy their feature. Not even talking for the fun when you need to update a third party dependency and you discover that half of the code base depends on a now removed feature, so expect migration in 5 to 12 years time.
> Once the interface is established, each team can operate independently, just like in microservices, as long as they adhere to the defined bounds.
Where I was lost laughing.
I'm sorry, but the hords of spaghetti monsters that I battle on a daily bases are spawning because communication is hard the more people are involved in it and the more someone is in a hurry to deploy their feature. Not even talking for the fun when you need to update a third party dependency and you discover that half of the code base depends on a now removed feature, so expect migration in 5 to 12 years time.
I feel like at this point in my career I've worked in all four quadrants of the spagehtti/encapsulation plane
^
monolith
|
|
<- spaghetti ----+------ clean ->
|
|
independent services
v
If you turned that into a heatmap of all software projects, I bet there would be hotspots. Probably would need to filter by project size or you would just end up with clusters at the top-right and bottom-left though
Having an opinion on something doesn't make the speaker a self righteous person.
Regarding the topic, I assume that with "the hords of spaghetti monsters" you mean lack of software perfection. In our profession, it's mostly about how to manage the lack of software perfection, and the answer does not always have to be a microservive. They are great for controlling hardware resources in a cloud-deployment, but then I see tech-dudes dockerizing everything anytime without asking why.
It's a decision every dev team have to make for themselves.
Regarding the topic, I assume that with "the hords of spaghetti monsters" you mean lack of software perfection. In our profession, it's mostly about how to manage the lack of software perfection, and the answer does not always have to be a microservive. They are great for controlling hardware resources in a cloud-deployment, but then I see tech-dudes dockerizing everything anytime without asking why.
It's a decision every dev team have to make for themselves.
Opinions are like asses - everyone has one. Self righteous people are those who think that only their opinions matter and try to present them as facts, rolling over any reasonable objection with hand waving and insulting the other party for not understanding things.
This comment is also your opinion, which you think is the only one that matters, and you present as fact.
Microservices can destroy DevOps teams used to kludging polyglot stacks together.
The issue is hardly limited to training, but rather ignorance of ones own bad habits.
Author probably heard the buzzword off someone else's CV. =)
The issue is hardly limited to training, but rather ignorance of ones own bad habits.
Author probably heard the buzzword off someone else's CV. =)
[deleted]