You Don’t Need All That Complex/Expensive/Distracting Infrastructure(medium.com)
medium.com
You Don’t Need All That Complex/Expensive/Distracting Infrastructure
https://medium.com/@dmfutcher/you-dont-need-all-that-complex-expensive-distracting-infrastructure-a70dbe0dbccb
84 comments
Or just set up Elastic Beanstalk with AWS or something similar, hand it a zip file of your code and let AWS configure your load balancer, autoscaling, patching your OS and runtime, managing your database and backups, etc.
What you suggest may be "simple" in terms of the amount of work the user needs to do, but my comment is about reduced complexity and $$ because you don't need them yet.
When you set up such an infrastructure like you proposed, there are still too many wheels in motion(complexity), and even worse they are abstracted away from you further and further. And, I'm sure the AWS bill is not going to be pretty considering they do all those bells and whistles.
When you set up such an infrastructure like you proposed, there are still too many wheels in motion(complexity), and even worse they are abstracted away from you further and further. And, I'm sure the AWS bill is not going to be pretty considering they do all those bells and whistles.
You literally click a few buttons and it sets up all of the infrastructure. Then you give it a zip file. The EC2 instance that it creates is a regular VM.
Elastic Beanstalk itself is free.
Elastic Beanstalk itself is free.
It is true that Elastic Beanstalk is free, and is very easy for a developer to use to begin with. However, If I were to advice a new startup on their infrastructure, I would not recommend it for the following reasons. It is my personal judgement. Others may prefer to go all in on Amazon.
1. It is clearly a vendor lock in. You build your business upon all AWS services, using an AWS orchestrator - good luck ever getting out.
2. Abstraction. It is a controversial opinion. I'd rather have less magic when possible and know what goes on underneath, and possibly have control over. Now, a startup may choose to offload the duties of operating a database(RDS), storing files(S3), performing backups(S3/Glacier) in the interest of time and trade-off control, optimisation and $$. However, considering the generalisation of simpler startups, IMHO operating a small scale Postgres on a standard EC2 instance is not really much of a hassle...
3. Cost - however small it may be in terms of actual dollars, AWS managed services(S3, RDS, SES, SQS etc..) come with added cost over simply running their open source equivalent yourselves.
Again, Keeping it simple - not in terms of what the user needs to do to deploy their application, but in terms of what components do you need to get your application running in production.
1. It is clearly a vendor lock in. You build your business upon all AWS services, using an AWS orchestrator - good luck ever getting out.
2. Abstraction. It is a controversial opinion. I'd rather have less magic when possible and know what goes on underneath, and possibly have control over. Now, a startup may choose to offload the duties of operating a database(RDS), storing files(S3), performing backups(S3/Glacier) in the interest of time and trade-off control, optimisation and $$. However, considering the generalisation of simpler startups, IMHO operating a small scale Postgres on a standard EC2 instance is not really much of a hassle...
3. Cost - however small it may be in terms of actual dollars, AWS managed services(S3, RDS, SES, SQS etc..) come with added cost over simply running their open source equivalent yourselves.
Again, Keeping it simple - not in terms of what the user needs to do to deploy their application, but in terms of what components do you need to get your application running in production.
1. It is clearly a vendor lock in. You build your business upon all AWS services, using an AWS orchestrator - good luck ever getting out.
If you are starting a business, the least of your business risks is lock in to AWS.
Besides that, you're handing a regular old zip file with your code to Elastic Beanstalk, it creates a VM with the standard web server package installed depending on your language. If later on, you need to go to another environment (and in the grand scheme of things, few companies ever switch out their infrastructure no matter how many Repository classes they write to abstract their database), you create your same webserver set up in the new environment.
Every cloud vendor has a hand-me-your-code-in-a-zip-file-and-I-setup-everything-for-you service.
However, considering the generalisation of simpler startups, IMHO operating a small scale Postgres on a standard EC2 instance is not really much of a hassle...
Why would I want to manage my own Postgres database and worry about backups, patching, etc. when I can pay someone else to do it?
Cost - however small it may be in terms of actual dollars, AWS managed services(S3, RDS, SES, SQS etc..) come with added cost over simply running their open source equivalent yourselves.
And that's more for your developers to manage, I'm sure if you can find some gullible young developers to work 60+ hours per week pulling double duty as developers and infrastructure operators it will be cheaper. I for one am not willing to do that. Every hour that your developers are spending maintaining infrastructure and doing the "undifferentiated heavy lifting" they are spending time not creating features adding business value.
You're talking about adding more servers (and no redundancy) to save a few pennies. I would avoid any company managed like that like the plague.
I actually have walked away from offers where the company was "on AWS" but all they were doing was running a bunch of EC2 instances and hosting everything themselves and expected their developers to be on call for infrastructure failures.
On the other hand, I did take a job where the new manager's marching orders were to move as far away from managing any infrastructure that could be done by AWS. We have to have a serious justification to put anything new on EC2 instances instead of using Lambda or Fargate.
If you are starting a business, the least of your business risks is lock in to AWS.
Besides that, you're handing a regular old zip file with your code to Elastic Beanstalk, it creates a VM with the standard web server package installed depending on your language. If later on, you need to go to another environment (and in the grand scheme of things, few companies ever switch out their infrastructure no matter how many Repository classes they write to abstract their database), you create your same webserver set up in the new environment.
Every cloud vendor has a hand-me-your-code-in-a-zip-file-and-I-setup-everything-for-you service.
However, considering the generalisation of simpler startups, IMHO operating a small scale Postgres on a standard EC2 instance is not really much of a hassle...
Why would I want to manage my own Postgres database and worry about backups, patching, etc. when I can pay someone else to do it?
Cost - however small it may be in terms of actual dollars, AWS managed services(S3, RDS, SES, SQS etc..) come with added cost over simply running their open source equivalent yourselves.
And that's more for your developers to manage, I'm sure if you can find some gullible young developers to work 60+ hours per week pulling double duty as developers and infrastructure operators it will be cheaper. I for one am not willing to do that. Every hour that your developers are spending maintaining infrastructure and doing the "undifferentiated heavy lifting" they are spending time not creating features adding business value.
You're talking about adding more servers (and no redundancy) to save a few pennies. I would avoid any company managed like that like the plague.
I actually have walked away from offers where the company was "on AWS" but all they were doing was running a bunch of EC2 instances and hosting everything themselves and expected their developers to be on call for infrastructure failures.
On the other hand, I did take a job where the new manager's marching orders were to move as far away from managing any infrastructure that could be done by AWS. We have to have a serious justification to put anything new on EC2 instances instead of using Lambda or Fargate.
> I actually have walked away from offers where the company was "on AWS" but all they were doing was running a bunch of EC2 instances and hosting everything themselves and expected their developers to be on call for infrastructure failures.
In addition to wasting engineer time and sanity, this is bad for the bottom line. Hardware fails, and if new hardware doesn’t cause systems to auto failover, you lose revenue, customers and trust. If you don’t autoscale, your service degrades when you need it most and you lose new customers and business when you get spiky load.
With all the advances in things like fargate and Aurora Serverless, it doesn’t make sense not to use these. The baseline cost is ~$200/mo for a setup that doesn’t go down when hardware fails or get slow when you get an influx of users. And it’s straightforward to set up.
In addition to wasting engineer time and sanity, this is bad for the bottom line. Hardware fails, and if new hardware doesn’t cause systems to auto failover, you lose revenue, customers and trust. If you don’t autoscale, your service degrades when you need it most and you lose new customers and business when you get spiky load.
With all the advances in things like fargate and Aurora Serverless, it doesn’t make sense not to use these. The baseline cost is ~$200/mo for a setup that doesn’t go down when hardware fails or get slow when you get an influx of users. And it’s straightforward to set up.
> If you are starting a business, the least of your business risks is lock in to AWS.
You are locking yourself into a cloud provider that is a part of a really hostile country in terms of digital privacy. Your customers may not care right now, but they really should. If you can sell that kind of behaviour to yourself because you're "starting", then your customers deserve the kind of business they are getting from you. To me it sounds incredibly unethical.
You are locking yourself into a cloud provider that is a part of a really hostile country in terms of digital privacy. Your customers may not care right now, but they really should. If you can sell that kind of behaviour to yourself because you're "starting", then your customers deserve the kind of business they are getting from you. To me it sounds incredibly unethical.
So which provider should a US based start up use? If they use a colo which country do you think it’s going to be in?
Seeing that all of the most valuable tech companies in the world besides the ones in China are based in the US, do you suggest they host with Alibaba Cloud?
Seeing that all of the most valuable tech companies in the world besides the ones in China are based in the US, do you suggest they host with Alibaba Cloud?
There is more to the world than the US and China (both of which are terrible choices). Outside of those places there are also cloud providers that give good service at a reasonable price.
Where would that be where there is a better startup scene than the US?
All valid concerns. I expected such a response, and hence I qualified my reasoning as a personal preference :-)
> If you are starting a business, the least of your business risks is lock in to AWS.
True and I agree, for the largest part.
> Every cloud vendor has a hand-me-your-code-in-a-zip-file-and-I-setup-everything-for-you service.
This is where I'd disagree (perhaps because of my infrastructure management background). The magical abstraction of "give me your code and don't worry about what happens underneath" IMO is not a good engineering practice. It may be for some. But, you're spending time anyway reading through documentation of some AWS service setting up your business only to walk away from it all as soon as you need to scale. I'd rather spend the same time reading the docs of the FOSS software that the AWS service in question is built on. Why? - Now you know what runs your code, and how. There is an inflection point to this idea. I'm not advocating for running your own DNS, DHCP, personal firewall etc. Just the core of what your business depends on. In my example - nginx and Postgresql.
Again, if you have an idea which you want to see if it works - perhaps setting up nginx and PostgreSQL by yourself just to see feasibility is silly. But, if you are starting up a real business and expect it run seriously, it'd be a good idea understanding what runs your business.
> You're talking about adding more servers (and no redundancy) to save a few pennies. I would avoid any company managed like that like the plague.
I intended to bring up cost only as a tangential benefit for a startup in its 0th phase of life. I did not mean to suggest that a company should make infrastructure reliability decision based on saving a few pennies.
By the time you need the complexity of dependable infrastructure redundancy, failover handling, and auto scaling, your company is big enough to hire infrastructure engineers whose job it is to do those and not have developers half-assing them under stress. My point is - for a young enough startup, avoid the need for complexity in the first place.
> I actually have walked away from offers where the company was "on AWS" but all they were doing was running a bunch of EC2 instances and hosting everything themselves and expected their developers to be on call for infrastructure failures.
If a company expects their developers to do infrastructure operations as "extra load", and be on call for some $AWS_SERVICE/$INFRASTRUCTURE, then I'd avoid them like plague as well. They are setting themselves up for failure by skimping on hiring SRE and infrastructure engineers.
By the time you need on call service for infrastructure, you're simply not in the nginx+PostgreSQL realm that I was talking about anymore.
> If you are starting a business, the least of your business risks is lock in to AWS.
True and I agree, for the largest part.
> Every cloud vendor has a hand-me-your-code-in-a-zip-file-and-I-setup-everything-for-you service.
This is where I'd disagree (perhaps because of my infrastructure management background). The magical abstraction of "give me your code and don't worry about what happens underneath" IMO is not a good engineering practice. It may be for some. But, you're spending time anyway reading through documentation of some AWS service setting up your business only to walk away from it all as soon as you need to scale. I'd rather spend the same time reading the docs of the FOSS software that the AWS service in question is built on. Why? - Now you know what runs your code, and how. There is an inflection point to this idea. I'm not advocating for running your own DNS, DHCP, personal firewall etc. Just the core of what your business depends on. In my example - nginx and Postgresql.
Again, if you have an idea which you want to see if it works - perhaps setting up nginx and PostgreSQL by yourself just to see feasibility is silly. But, if you are starting up a real business and expect it run seriously, it'd be a good idea understanding what runs your business.
> You're talking about adding more servers (and no redundancy) to save a few pennies. I would avoid any company managed like that like the plague.
I intended to bring up cost only as a tangential benefit for a startup in its 0th phase of life. I did not mean to suggest that a company should make infrastructure reliability decision based on saving a few pennies.
By the time you need the complexity of dependable infrastructure redundancy, failover handling, and auto scaling, your company is big enough to hire infrastructure engineers whose job it is to do those and not have developers half-assing them under stress. My point is - for a young enough startup, avoid the need for complexity in the first place.
> I actually have walked away from offers where the company was "on AWS" but all they were doing was running a bunch of EC2 instances and hosting everything themselves and expected their developers to be on call for infrastructure failures.
If a company expects their developers to do infrastructure operations as "extra load", and be on call for some $AWS_SERVICE/$INFRASTRUCTURE, then I'd avoid them like plague as well. They are setting themselves up for failure by skimping on hiring SRE and infrastructure engineers.
By the time you need on call service for infrastructure, you're simply not in the nginx+PostgreSQL realm that I was talking about anymore.
This is where I'd disagree (perhaps because of my infrastructure management background). The magical abstraction of "give me your code and don't worry about what happens underneath" IMO is not a good engineering practice.
The context of the discussion is not needing “complex” infrastructure starting out. If you are trying to ramp up quickly, there is a way with a click of a few buttons you can have a scalable, fault tolerant solution with Blue/Green or rolling deployments etc.
It may be for some. But, you're spending time anyway reading through documentation of some AWS service setting up your business only to walk away from it all as soon as you need to scale.
Scaling your web servers with EB is just going into the console and increasing the maximum number of servers in your web farm.
I'd rather spend the same time reading the docs of the FOSS software that the AWS service in question is built on. Why? - Now you know what runs your code, and how.
Developers don’t develop on AWS. They still need to know how to set up their local machine to run their software during development.
There is an inflection point to this idea. I'm not advocating for running your own DNS, DHCP, personal firewall etc. Just the core of what your business depends on. In my example - nginx and Postgresql.
They would set up Nginx on their computer as part of their development cycle. They would need to know that anyway.
Again, if you have an idea which you want to see if it works - perhaps setting up nginx and PostgreSQL by yourself just to see feasibility is silly. But, if you are starting up a real business and expect it run seriously, it'd be a good idea understanding what runs your business.
Is setting up Postgres on my computer locally going to help me understand how Aurora/Postgres stores data redundantly across multiple availability zones with Multi-AZ redundancy, automatic failover, read replicas, automated backups, point in time recovery etc?
By the time you need the complexity of dependable infrastructure redundancy, failover handling, and auto scaling, your company is big enough to hire infrastructure engineers whose job it is to do those and not have developers half-assing them under stress. My point is - for a young enough startup, avoid the need for complexity in the first place.
Why do you assume that reliability and scalability isn’t a requirement from day one? Both small companies I’ve worked for were B2B companies that had a few large “whales” as clients. They wouldn’t have been able to keep their first client if they weren’t reliable on scalable.
If a company expects their developers to do infrastructure operations as "extra load", and be on call for some $AWS_SERVICE/$INFRASTRUCTURE, then I'd avoid them like plague as well. They are setting themselves up for failure by skimping on hiring SRE and infrastructure engineers. By the time you need on call service for infrastructure, you're simply not in the nginx+PostgreSQL realm that I was talking about anymore.
If an AWS infrastructure service goes down, there is really nothing an infrastructure engineer can do at that time. It’s dead simple to build out on AWS in a fault tolerant, scalable manner between autoscaling groups of EC2 instances, Aurora (MySQL/Postgres) with read replicas and multi AZ deployments, lambda, etc.
But you still don’t need to hire a full time infrastructure person. You can farm that stuff out to a managed service provider cheaper.
Don’t get me wrong, I personally hate Elastic Beanstalk. If I want to spin something up fast, I’ll use another service that AWS has called Codestar. It creates a code template in your chosen language, your CI/CD pipeline, an EC2 instance (or lambda pipeline for serverless) and your buildspec/appspec files (containing a bunch of shell commands) and your CloudFormation template. But to make the skeleton usable, you still have to know how everything on AWS works.
But the whole point of the submission is that you shouldn’t have to know all that from day one. At the end of the day, you still have a standard Windows or Linux based VM with EB using the same stack you would use anywhere else.
The context of the discussion is not needing “complex” infrastructure starting out. If you are trying to ramp up quickly, there is a way with a click of a few buttons you can have a scalable, fault tolerant solution with Blue/Green or rolling deployments etc.
It may be for some. But, you're spending time anyway reading through documentation of some AWS service setting up your business only to walk away from it all as soon as you need to scale.
Scaling your web servers with EB is just going into the console and increasing the maximum number of servers in your web farm.
I'd rather spend the same time reading the docs of the FOSS software that the AWS service in question is built on. Why? - Now you know what runs your code, and how.
Developers don’t develop on AWS. They still need to know how to set up their local machine to run their software during development.
There is an inflection point to this idea. I'm not advocating for running your own DNS, DHCP, personal firewall etc. Just the core of what your business depends on. In my example - nginx and Postgresql.
They would set up Nginx on their computer as part of their development cycle. They would need to know that anyway.
Again, if you have an idea which you want to see if it works - perhaps setting up nginx and PostgreSQL by yourself just to see feasibility is silly. But, if you are starting up a real business and expect it run seriously, it'd be a good idea understanding what runs your business.
Is setting up Postgres on my computer locally going to help me understand how Aurora/Postgres stores data redundantly across multiple availability zones with Multi-AZ redundancy, automatic failover, read replicas, automated backups, point in time recovery etc?
By the time you need the complexity of dependable infrastructure redundancy, failover handling, and auto scaling, your company is big enough to hire infrastructure engineers whose job it is to do those and not have developers half-assing them under stress. My point is - for a young enough startup, avoid the need for complexity in the first place.
Why do you assume that reliability and scalability isn’t a requirement from day one? Both small companies I’ve worked for were B2B companies that had a few large “whales” as clients. They wouldn’t have been able to keep their first client if they weren’t reliable on scalable.
If a company expects their developers to do infrastructure operations as "extra load", and be on call for some $AWS_SERVICE/$INFRASTRUCTURE, then I'd avoid them like plague as well. They are setting themselves up for failure by skimping on hiring SRE and infrastructure engineers. By the time you need on call service for infrastructure, you're simply not in the nginx+PostgreSQL realm that I was talking about anymore.
If an AWS infrastructure service goes down, there is really nothing an infrastructure engineer can do at that time. It’s dead simple to build out on AWS in a fault tolerant, scalable manner between autoscaling groups of EC2 instances, Aurora (MySQL/Postgres) with read replicas and multi AZ deployments, lambda, etc.
But you still don’t need to hire a full time infrastructure person. You can farm that stuff out to a managed service provider cheaper.
Don’t get me wrong, I personally hate Elastic Beanstalk. If I want to spin something up fast, I’ll use another service that AWS has called Codestar. It creates a code template in your chosen language, your CI/CD pipeline, an EC2 instance (or lambda pipeline for serverless) and your buildspec/appspec files (containing a bunch of shell commands) and your CloudFormation template. But to make the skeleton usable, you still have to know how everything on AWS works.
But the whole point of the submission is that you shouldn’t have to know all that from day one. At the end of the day, you still have a standard Windows or Linux based VM with EB using the same stack you would use anywhere else.
This. Most businesses are not megacorporations which need unlimited scalability and bleeding edge platforms. Start simple and add complexity only when it's needed. It's boring, but stability is meant to be boring.
> Kubernetes clusters? Load balancers across multiple cloud regions? Zero-click, Blue-Green deployment based on real-time metrics from a continuous deployment pipeline?
Kubernetes, multi region, canary deployment, immutable infrastructure, even infrastructure as code is overengineering for a brand new indie project, for sure.
BUT, there is a certain minimum amount of quality, availability and scalability any product needs, especially a new product that might show up on hacker news or product hunt and get 100-300 concurrent requests a second for multiple hours at unpredictable times.
To not use fargate because you have to dockerize your app and build it during a codepipeline stage is silly. That takes what, a week, max, the first time you ever do it? Once you've done it, you can do it faster for future products. To not use an autoscaling and high availability database when it costs the same and is so simple to set up makes no sense either. To not use ci/cd to test your db migrations and new code on staging before pushing to prod is just dumb and asking for pain and suffering. Why put yourself through that when products have been built to make these things _so_ easy and inexpensive now?
Kubernetes, multi region, canary deployment, immutable infrastructure, even infrastructure as code is overengineering for a brand new indie project, for sure.
BUT, there is a certain minimum amount of quality, availability and scalability any product needs, especially a new product that might show up on hacker news or product hunt and get 100-300 concurrent requests a second for multiple hours at unpredictable times.
To not use fargate because you have to dockerize your app and build it during a codepipeline stage is silly. That takes what, a week, max, the first time you ever do it? Once you've done it, you can do it faster for future products. To not use an autoscaling and high availability database when it costs the same and is so simple to set up makes no sense either. To not use ci/cd to test your db migrations and new code on staging before pushing to prod is just dumb and asking for pain and suffering. Why put yourself through that when products have been built to make these things _so_ easy and inexpensive now?
> ‘Engineers get sidetracked by things that make engineers excited, not that solve real problems for users’
The real question is why business allows them to. I find it entertaining to try to substitute another profession for engineer in the sentence above and see if it still works. Lawyers, teachers, doctors, civil architects...
The real question is why business allows them to. I find it entertaining to try to substitute another profession for engineer in the sentence above and see if it still works. Lawyers, teachers, doctors, civil architects...
The precious commodity with our work is cognitive load. We can build tooling that reduces future cognitive load. As a result, we have access to multipliers. There is no analogy to this with the other fields.
The idea that tooling "reduces" cognitive load is based on the assumption that tooling lets you work in terms of abstractions rather than worrying about what the tooling actually does. This assumption breaks down as soon as the tooling itself breaks down. At that point, not only do you have to worry about the underlying problems that the tooling is intended to solve, but you also have to worry about the implementation details of the tooling itself.
Tooling breaks down more often when it's more complicated, and you need more complexity when your tooling abstracts over a large amount of complexity, so the more that your tooling is supposed to reduce your cognitive load, the more likely it will break in such a way that it actually increases your cognitive load.
Tooling breaks down more often when it's more complicated, and you need more complexity when your tooling abstracts over a large amount of complexity, so the more that your tooling is supposed to reduce your cognitive load, the more likely it will break in such a way that it actually increases your cognitive load.
[deleted]
'CEOs get excited by things that make CEOs excited, not that solve real problems for real users...'
Still works, mostly. Certainly true of architects, partially true of lawyers and doctors, less true of teachers.
The key difference is the amount of f2f time with users/customers/clients. If you add a dumb feature to your site and you have to talk to users about it f2f - not over Twitter or Slack or Medium, but in person - you're going to give yourself a much more realistic idea of what matters to them.
Still works, mostly. Certainly true of architects, partially true of lawyers and doctors, less true of teachers.
The key difference is the amount of f2f time with users/customers/clients. If you add a dumb feature to your site and you have to talk to users about it f2f - not over Twitter or Slack or Medium, but in person - you're going to give yourself a much more realistic idea of what matters to them.
> CEOs get excited by things that make CEOs excited
They seem particularly enarmoured with re-arranging the org-charts.
They seem particularly enarmoured with re-arranging the org-charts.
> `you’re on Medium after all’
Ahhhhh! You got me!
Ahhhhh! You got me!
Pilots!
Well perhaps it’s the business doing pie in the sky thinking resulting in these solutions. It’s not without a reason the Indy dev is highlighted here.
in most companies (I know) the business stakeholders have little idea about what makes sense within and for engineering. So they rely on their 'subject matter expert' engineers to find the best decisions. (and a place that doesn't trust their engineers to come up with the right solution is doomed imo).
This problem usually indicates a weak team, or lack of leadership within engineering. But granted, selecting a weak CTO (who picks snowflake & diva engineers) should be attributed to the business.
This problem usually indicates a weak team, or lack of leadership within engineering. But granted, selecting a weak CTO (who picks snowflake & diva engineers) should be attributed to the business.
My experiences with lawyers suggests that they can have a tendency to get sidetracked that is most definitely "not that solve real problems for users" - the real difference with lawyers being that they will then charge you vast amounts of money for their sidetracks.
Usually,. the thing that you lose is scalability, availability any reliability.
You will need a remote backup and a secondary located service, just in case. Potentially a third development one to not push breaking changes directly into production.
Skipping any of this will make your service brittle in addition to not being scalable.
And it already hits the rule of three... (of "don't repeat yourself") that makes a lightweight Kubernetes deployment worthwhile. (Perhaps without all the bells and whistles.)
Remember, downtime is usually lost money. A day's worth can be thousands of dollars at least, and that's two nines already. (99.99%)
You will need a remote backup and a secondary located service, just in case. Potentially a third development one to not push breaking changes directly into production.
Skipping any of this will make your service brittle in addition to not being scalable.
And it already hits the rule of three... (of "don't repeat yourself") that makes a lightweight Kubernetes deployment worthwhile. (Perhaps without all the bells and whistles.)
Remember, downtime is usually lost money. A day's worth can be thousands of dollars at least, and that's two nines already. (99.99%)
Well if the first day you shipped your product and it's already worth 1k, yeah it makes sense to do everything the "correct" way. But for most of the side projects and startup products this makes no sense at all.
Or, you ship 2 months earlier, and the trade off being suffering a day's worth of downtime.
It makes no sense to start with a more complicated setup, until there is clear evidence of a need.
It makes no sense to start with a more complicated setup, until there is clear evidence of a need.
The main problem is that usually low quality infra bites you when you get exposure. Which is exactly when you don't want it to fail...
For a start, Heroku or AWS instance will do and they scale. (As the other commenter said)
If you're doing your own infra, you need at least basic fallbacks and safety.
For a start, Heroku or AWS instance will do and they scale. (As the other commenter said)
If you're doing your own infra, you need at least basic fallbacks and safety.
I saw so many places falling for this fallacy.
Instead on working on the core products, engineers would have "fun" adding more microservices, redundant Kubernetes clusters, istio controllers etc etc. (all of that with less than 10 users).
Instead on working on the core products, engineers would have "fun" adding more microservices, redundant Kubernetes clusters, istio controllers etc etc. (all of that with less than 10 users).
When people start adding micro services (or whatever the fad is this month) ask "why?"
"Because it scales" "But we don't have a scaling problem, and if we did it could easily be solved with some caching".
Unfortunately there is also the bullshit which is hiring in this industry, where solving things the simple way doesn't add anything interesting to your CV.
"Because it scales" "But we don't have a scaling problem, and if we did it could easily be solved with some caching".
Unfortunately there is also the bullshit which is hiring in this industry, where solving things the simple way doesn't add anything interesting to your CV.
People who underestimate the challenges adding caching can cause make me a bit nervous.
Especially when you start putting caches on top of caches. You can end up with a hunt which cache has the incorrect value problem.
Especially when you start putting caches on top of caches. You can end up with a hunt which cache has the incorrect value problem.
Just my 2c regarding ”microservices” or SOA, whatever you want to call it, is that, sure it can enable you to scale horizontally from a technical perspective, but few have to deal with those problems for real.
What it does though when organically implemented, is it allows you to scale dev teams more easilly and it will also let you build functionallity and features more rapidly for an ever evolving business.
This is my take-away from two massive undertakings at two separate medium-to-large scale enterprises that I’ve been involved in.
One was a complete failure and one an enormous success that both have had business altering implications.
What it does though when organically implemented, is it allows you to scale dev teams more easilly and it will also let you build functionallity and features more rapidly for an ever evolving business.
This is my take-away from two massive undertakings at two separate medium-to-large scale enterprises that I’ve been involved in.
One was a complete failure and one an enormous success that both have had business altering implications.
Same; one project I was at was a lot of marketing talk, but after a year and a half of development they still had no working software, a big microservices architecture with a full-time infrastructure automation team and three developer teams, but no architect or person with the big (technical) picture in mind, no monitoring, no tracing, some logging (ELK stack) but no logging standards, etc. If they had just started with a normal Java application and idk, tossed it on App Engine they would've been live within six months at a fraction of the cost.
I had an argument with a co-worker about this. He wanted to use Kubernetes, so I asked him what problem he was trying to solve. No answer- just wanted to because he could put it on his resume.
You deploy tech to solve a problem. Don't go solving problems that you don't have yet.
You deploy tech to solve a problem. Don't go solving problems that you don't have yet.
That, actually, is a bigger problem. Some people are using things not for the benefit of the thing they're building, but for their own (AKA resumes).
> No answer - just wanted to because he could put it on his resume.
Is it his fault that the incentives are borked?
Is it his fault that the incentives are borked?
there is a time and place to experiment with new technology.
i am a big fan of using proven technology + KISS, but also like for people to experiment with new technology in a low pressure/learning environment.
people using something just to put it on their resume is a huuge red flag to me.
people using something just to put it on their resume is a huuge red flag to me.
Nobody ever got promoted or a raise because the core product 'just works'.
As an engineer, you need growth in headcount or complexity if you want a raise.
It's a people management problem that flows from the CEO and form the HR department, not a technical one.
As an engineer, you need growth in headcount or complexity if you want a raise.
It's a people management problem that flows from the CEO and form the HR department, not a technical one.
One gets fired when everything „just works”. Everything works (untils it stops working) so your job is done, what are you doing even during your office hours? I wish having a „technical manager” would avoid such situations but it’s enough if this „technical manager” hates the whole domain you work in.
>Your users to don’t care how your code gets onto your server.
I don't like arguments like this. This is true for any number of tasks that a layman wouldn't care to understand. You still will do them anyway, regardless of whether or not your user cares. It doesn't take a lot of experience to see the correlation between poor engineering practices and poor user experience.
Now, I don't think the author was using this argument to pitch poor engineering, but there are good reasons for setting up resilient and consistent build deploys, as he addresses. I think the only real takeaway is "dont over-engineer your infrastructure" which is just an extension of generally "don't over-engineer."
Every time infrastructure gets talked about on HN the conversation gets so sidetracked. The real problem is that people don't think about infrastructure in the context of a holistic view of a project. The hours you spend setting up your project with resilient and consistent CI builds are many times more valuable down the road.
I think people need to not be so afraid of spending time on infrastructure. People already err on the side of spending less than more time on infrastructure. I don't think it's wrong at all that the author spent hours building an automatic build process for his project, especially because now that he's set it up he can refer to it in the future. Maybe with his newfound knowledge and experience on the problem means that he can leverage it with little effort in the future. None of this is a bad thing especially if you think about your engineering career on a macro scale instead of week to week.
I don't like arguments like this. This is true for any number of tasks that a layman wouldn't care to understand. You still will do them anyway, regardless of whether or not your user cares. It doesn't take a lot of experience to see the correlation between poor engineering practices and poor user experience.
Now, I don't think the author was using this argument to pitch poor engineering, but there are good reasons for setting up resilient and consistent build deploys, as he addresses. I think the only real takeaway is "dont over-engineer your infrastructure" which is just an extension of generally "don't over-engineer."
Every time infrastructure gets talked about on HN the conversation gets so sidetracked. The real problem is that people don't think about infrastructure in the context of a holistic view of a project. The hours you spend setting up your project with resilient and consistent CI builds are many times more valuable down the road.
I think people need to not be so afraid of spending time on infrastructure. People already err on the side of spending less than more time on infrastructure. I don't think it's wrong at all that the author spent hours building an automatic build process for his project, especially because now that he's set it up he can refer to it in the future. Maybe with his newfound knowledge and experience on the problem means that he can leverage it with little effort in the future. None of this is a bad thing especially if you think about your engineering career on a macro scale instead of week to week.
I agree that CI is a brilliant investment. Also it can initially be as simple as a git hook. However, I agree with the author’s point that you do not need a Kubernetes distributed NoSQL megagalaxy when you have a number of customers which you can count on one hand. I’ve seen so much time wasted on this sort of thing.
One obvious benefit of the whole single/simple server idea is that you can make changes to the system so quickly and easily, which is very important at the beginning.
One obvious benefit of the whole single/simple server idea is that you can make changes to the system so quickly and easily, which is very important at the beginning.
Ya nosql can be useful at scale, but it's not a great choice for most new projects. It really locks you into your schema and makes querying the data at scale difficult outside of use cases you originally designed for. I've seen aws architects recommending just starting with sql, and adding nosql later as needed. Definitely agree with this approach for most use cases.
I agree with the author : start simple, then use more complex tools when and if you need to.
I believe it is true about infrastructure, and about features and code as well.
When my team needs to release a functionality for the "product"[1] that we maintain, we have a very simple strategy : we go step by step. The users tell us what they need, and we start to deliver as soon as we can. Our first release only solves a portion of the user's problem, but the second release solves more, and so does the next, and so on. I devised this strategy with the users to make sure they are fine with it: they expect a perfect product... eventually. But in the mean time, they having only some portions of their needs met, and we avoid the feeling of being overwhelmed by a great apparent complexity.
[1] The context is a bit complicated... and irrelevant. We don't exactly maintain a product, but it's close enough for the argument here.
I believe it is true about infrastructure, and about features and code as well.
When my team needs to release a functionality for the "product"[1] that we maintain, we have a very simple strategy : we go step by step. The users tell us what they need, and we start to deliver as soon as we can. Our first release only solves a portion of the user's problem, but the second release solves more, and so does the next, and so on. I devised this strategy with the users to make sure they are fine with it: they expect a perfect product... eventually. But in the mean time, they having only some portions of their needs met, and we avoid the feeling of being overwhelmed by a great apparent complexity.
[1] The context is a bit complicated... and irrelevant. We don't exactly maintain a product, but it's close enough for the argument here.
Oops! Sorry for the huge typo in my comment. "they having only some portions of their needs met" should be read "they are fine with having only some portions of their needs met".
(It is a bit more than a typo)
(It is a bit more than a typo)
You don't need it if you are a one-man band doing a small side project.
> I spent hours setting up a simple and free continuous deployment pipeline. Commit to git, Semaphore CI builds the Docker image, uploads the image to a container registry, then pulls & restarts the container on my server. It’s really cool. I felt like a wizard once I got it working … and it was free! I actually sat down here to write all about how to do it yourself, until I realised how long I spent on it and how much value it delivered to my users
If you have a team of some size, the time you gain over the lifecycle of the project can be very significant when you have a smoothly running automated dev/test/UA/live pipeline.
> Not build the fanciest deployment pipelines, or multi-zone, multi-region, multi-cloud Nuclear Winter proof high availability setup.
If your downtime can be measured in appreciable lost revenue or reputational cost, then yes, you need all of that, as well as a solid SRE or 5 to keep it all running.
> I spent hours setting up a simple and free continuous deployment pipeline. Commit to git, Semaphore CI builds the Docker image, uploads the image to a container registry, then pulls & restarts the container on my server. It’s really cool. I felt like a wizard once I got it working … and it was free! I actually sat down here to write all about how to do it yourself, until I realised how long I spent on it and how much value it delivered to my users
If you have a team of some size, the time you gain over the lifecycle of the project can be very significant when you have a smoothly running automated dev/test/UA/live pipeline.
> Not build the fanciest deployment pipelines, or multi-zone, multi-region, multi-cloud Nuclear Winter proof high availability setup.
If your downtime can be measured in appreciable lost revenue or reputational cost, then yes, you need all of that, as well as a solid SRE or 5 to keep it all running.
> Obviously if you’re FAANG-level or some established site where that 0.1% downtime translates into vast quantities of cash disappearing from your books, this stuff is all great. You have the funds to do things “right”.
You don’t have to be a FAANG. You can be a B2B startup with one or two major clients who if they see you aren’t highly available will jump ship taking a lot of your revenue with them.
I’ve been in enough meetings working for small startups where our clients grilled us about our resiliency, availability, redundancy etc. and where they forced us to put our code in escrow as part of our contract.
I’ve been in enough meetings working for small startups where our clients grilled us about our resiliency, availability, redundancy etc. and where they forced us to put our code in escrow as part of our contract.
It doesn't have to be "vast quantities of cash", it only has to balance out the cost of the additional work. And reputational cost is notoriously difficult to turn into a cash value. I really have an issue with how the author presents this as a "Almost nobody needs this stuff", where the truth is that it is a lot more nuanced.
I typically advise my clients what uptime will cost them extra, per "9" (as in the 5-nines), and let them make their own decisions.
I typically advise my clients what uptime will cost them extra, per "9" (as in the 5-nines), and let them make their own decisions.
If I am a one man band doing a side project, I agree. I can build my C# project locally and just deploy directly from my IDE. But as soon as one other person is involved, I’m going to setup a simple hosted CI/CD pipeline with hosted build servers.
It depends on who your customer is.
If your customer is the investor, then there is no limit to the amount of hardware, infrastructure, and experts sitting behind a desk and multiple monitors in every concevable separation of concern you can think of in every major city across the globe. Your product is amazing, and its going to change the world and how people percieve things. You need everything you can throw at it. Investors want to be the best. And thats if you customer is the investor.
When you are building a startup, there is always the question, "will this scale?" or "in the event of an investment or going viral are we ready to scale automatically ?"
I think that there is a lot of worries around this, it conditions technology choices, sometimes pushing early optimisations, and it is hard because you don't have the data on how many concurrent users your product will have. The best you can do is to extrapolate from you current data or other products data.
So many people end up following a recipe, it could be Firebase recipe, AWS Fargate, Serverless... you name it. Recipes that promise limitless autoscale.
I think indeed is distracting and premature, but again is hard to avoid it, if the premise is that there will be a massive growth overnight, which is imposible to predict that will potentially kill us, and we have to be ready.
I think that there is a lot of worries around this, it conditions technology choices, sometimes pushing early optimisations, and it is hard because you don't have the data on how many concurrent users your product will have. The best you can do is to extrapolate from you current data or other products data.
So many people end up following a recipe, it could be Firebase recipe, AWS Fargate, Serverless... you name it. Recipes that promise limitless autoscale.
I think indeed is distracting and premature, but again is hard to avoid it, if the premise is that there will be a massive growth overnight, which is imposible to predict that will potentially kill us, and we have to be ready.
I've wasted a lot of time during my attempts as a solo developer to build a product with a scalable backend architecture for the hordes of users that never materialize.
Next idea I have, I swear I'm going to throw all my effort into the frontend and go with as minimal a backend as I can get away with (heck, depending on the application, human-in-the-loop might be fine early on) until I get some validation. It'll feel "wrong", but whatever.
Next idea I have, I swear I'm going to throw all my effort into the frontend and go with as minimal a backend as I can get away with (heck, depending on the application, human-in-the-loop might be fine early on) until I get some validation. It'll feel "wrong", but whatever.
While I think you can definitely get too fancy with all of that sweet infra, I feel this is missing an important point:
Once you have a working pipeline in place, you can focus on the features, that's the whole point of the exercise: enabling developers to ship features for users. If I have to SSH into some machine and restart a set of services manually in the correct order, it will keep me from releasing features. I am afraid / lazy / whatever. Spending time on this is not something that will never ever reach the user in any way.
The user does not care if something is using Heroku, K8s, or something else, yes, but the user definitely appreciates a product that can reliably iterate on things without major downtime or similar problems.
It's kind of like saying, "well unit tests are nice and stuff, but does the user ever notice?". I would strongly argue the user will notice unit tests in the long run as they ensure the software keeps being maintainable.
Once you have a working pipeline in place, you can focus on the features, that's the whole point of the exercise: enabling developers to ship features for users. If I have to SSH into some machine and restart a set of services manually in the correct order, it will keep me from releasing features. I am afraid / lazy / whatever. Spending time on this is not something that will never ever reach the user in any way.
The user does not care if something is using Heroku, K8s, or something else, yes, but the user definitely appreciates a product that can reliably iterate on things without major downtime or similar problems.
It's kind of like saying, "well unit tests are nice and stuff, but does the user ever notice?". I would strongly argue the user will notice unit tests in the long run as they ensure the software keeps being maintainable.
When you walk into a restaurant, do you notice if they have an efficiently laid-out kitchen with mise-en-place, good relationships with their suppliers, or a refrigerator at the proper temperature? No.
But you do notice when you food is slow to arrive, the food doesn't taste fresh, or the chicken gives you food poisoning.
But you do notice when you food is slow to arrive, the food doesn't taste fresh, or the chicken gives you food poisoning.
I think it is also important to remember that users are not the only consumers of a software development pipeline.
If developers don't want to work for you because they feel held back in creativity or career progression, support staff have to stay up late for downtime deployments, and managers and/or ownership feel blind to the process, then you can still have a miserable company with happy users.
Quality of the work is an important part of a compensation plan. Eventually this hits an organization's bottom line in the form of turnover, employee productivity, etc. In software development with high salary, difficult to recruit employees, this can be significant.
If developers don't want to work for you because they feel held back in creativity or career progression, support staff have to stay up late for downtime deployments, and managers and/or ownership feel blind to the process, then you can still have a miserable company with happy users.
Quality of the work is an important part of a compensation plan. Eventually this hits an organization's bottom line in the form of turnover, employee productivity, etc. In software development with high salary, difficult to recruit employees, this can be significant.
But many applications were better around 2002-2004 than now. The only thing that's better now for users who don't care about privacy at all is the opaque sign-in-with-X feature.
Email web interfaces got worse, login forms got worse, documentation got worse, search got worse (peaked around 2004).
Email web interfaces got worse, login forms got worse, documentation got worse, search got worse (peaked around 2004).
> Once you have a working pipeline in place, you can focus on the features, that's the whole point of the exercise: enabling developers to ship features for users. If I have to SSH into some machine and restart a set of services manually in the correct order, it will keep me from releasing features.
A "working pipeline" is an abstraction. Your actual pipeline will still sometimes break or misbehave in unpredictable ways, except now it takes a lot more context to debug and resolve those issues. Which is fine if you have dedicated infrastructure teams working behind the scenes to keep all that stuff stable (Heroku is an entire company that does this) but it's no silver bullet.
If you have to SSH into some machine and restart services whenever you release a feature, you will learn how to do that. And in that process, you will have a better idea how to fix it if it breaks and how to design features in an operationally stable way. If you just push to master and let the Machine take it from there, you're living in willful ignorance and are helpless when the Machine inevitably breaks down. And, in the long run, that will have a much greater impact on your feature velocity.
A "working pipeline" is an abstraction. Your actual pipeline will still sometimes break or misbehave in unpredictable ways, except now it takes a lot more context to debug and resolve those issues. Which is fine if you have dedicated infrastructure teams working behind the scenes to keep all that stuff stable (Heroku is an entire company that does this) but it's no silver bullet.
If you have to SSH into some machine and restart services whenever you release a feature, you will learn how to do that. And in that process, you will have a better idea how to fix it if it breaks and how to design features in an operationally stable way. If you just push to master and let the Machine take it from there, you're living in willful ignorance and are helpless when the Machine inevitably breaks down. And, in the long run, that will have a much greater impact on your feature velocity.
> Once you have a working pipeline in place, you can focus on the features
Bullshit, its just another layer of complexity to be managed and that may fail. My guess is that it will take more time maintaining it than it will save you in the majority of cases.
Bullshit, its just another layer of complexity to be managed and that may fail. My guess is that it will take more time maintaining it than it will save you in the majority of cases.
> My guess is that it will take more time maintaining it than it will save you in the majority of cases.
From my experience, this has almost never been the case. Exceptions were when there was infrastructure for the sake of infrastructure.
From my experience, this has almost never been the case. Exceptions were when there was infrastructure for the sake of infrastructure.
If a leader is worried about their organization having infra for the sake of infra, then they should also worry that their user-facing product has features for the sake of features—- features that complicate an interface and don’t help a user get their job done. The solution is PMs, UX designers, and engineers who know to ask “why?” when a project is proposed.
That's a good point. Many things can be overengineered/overdone, including UX design.
Given how annoying it is to SSH to a server, check out the code, restart all the relevant services, I think there’s a definite plus to having my system do that for me.
If my pipeline fails, it’s likely because of a build issue.
I have two types of pipelines both based on AWS:
1. Push code to Github -> AWS CodeBuild spins up a prebuilt or custom Docker container (on a server that I don’t have to manage) to build and run unit tests -> Code Deploy agent runs on EC2 instance that runs a script to deploy website.
2. Same as above but Code Deploy runs a CloudFormation template to deploy/update lambdas and other related resources.
The only part of either process that requires me to maintain a server at all is the web server.
It takes me about an hour to set up a new pipeline at the most. It’s basically modified a bunch of yaml/JSON files.
It’s a one time setup that pays dividends through the whole process.
I’ve also used hosted builds with Azure Devops before.
I have two types of pipelines both based on AWS:
1. Push code to Github -> AWS CodeBuild spins up a prebuilt or custom Docker container (on a server that I don’t have to manage) to build and run unit tests -> Code Deploy agent runs on EC2 instance that runs a script to deploy website.
2. Same as above but Code Deploy runs a CloudFormation template to deploy/update lambdas and other related resources.
The only part of either process that requires me to maintain a server at all is the web server.
It takes me about an hour to set up a new pipeline at the most. It’s basically modified a bunch of yaml/JSON files.
It’s a one time setup that pays dividends through the whole process.
I’ve also used hosted builds with Azure Devops before.
This reminds me of those ghost cities in China -- they built it, but nobody came.
There's an obvious and overlooked middle ground between "massively complicated infrastructure" and "a cheap VPS" - charge more and use PaaS. Heroku and Engine Yard are expensive, but they deliver a huge amount of value. You've got better things to do than patch Nginx or deal with a crashed server at 4am, just pay someone a sensible amount of money to run your infrastructure for you.
> an obvious and overlooked middle ground between "massively complicated infrastructure" and "a cheap VPS"
It's called a dedicated server. These days you can get a monster of a machine. Even on AWS you can get a machine with 2TB of RAM and 128 cores.
It's called a dedicated server. These days you can get a monster of a machine. Even on AWS you can get a machine with 2TB of RAM and 128 cores.
Heroku is fine, but their lack of an autoscaling database for spiky load and minimum $250/mo for autoscaling web servers are really very far behind what using aws directly offers these days. Heroku seems to have stopped innovating.
I agree. There are also options to make some of the things from the article relatively easy. Off the top of my head, a CI/CD pipeline can be set up using Gitlab , Azure DevOps, Bitbucket Pipelines, and I'm sure tons of others. They do some of the heavy lifting and you still get the benefit of an organized build that's easy to deploy to one of the services listed above.
And yeah, this may be layered on later when it seems like the idea has legs and you'd benefit from this. It most likely doesn't need to be the first step.
And yeah, this may be layered on later when it seems like the idea has legs and you'd benefit from this. It most likely doesn't need to be the first step.
Please at least use a managed database. Treat that linode box like it might disappear at any moment. It is so easy to shoot yourself in the foot with a database and lose everything.
Also FWIW configuring Linux machines manually over SSH and building and deploying your code too it, isn't always easier. Make sure you write everything you did down, because in 6 months you will completely forget how you built this thing, where it was deployed, etc...
Also FWIW configuring Linux machines manually over SSH and building and deploying your code too it, isn't always easier. Make sure you write everything you did down, because in 6 months you will completely forget how you built this thing, where it was deployed, etc...
Please write everything down and be kind to 'Future You'.
I am very anti-spending money on software that isn't directly related to a project but a $10 confluence wiki lifetime license is one of the best purchases I have ever made.
I am very anti-spending money on software that isn't directly related to a project but a $10 confluence wiki lifetime license is one of the best purchases I have ever made.
I have not done this for a number of years, but in the past for my own and customer projects I would just run Postgres on the same VPS as the web app but use a cronjob to write incremental backups every few hours to S3.
As other people here have said, agreeing with you, paying for multiple availability zone RDS or paying Heroku is also a good plan.
As other people here have said, agreeing with you, paying for multiple availability zone RDS or paying Heroku is also a good plan.
And whatever you choose, TEST YOUR BACKUPS. Spin up a new instance and practice recovery. This needs to be done at least quarterly, both to test your backups and train yourself and/or your people in this process.
Right on! I am enjoying the book “A Company of One” right now and the author makes the same point of only doing what it takes to provide great service to a few customers. Another point is to spend more effort servicing existing customers than spending time trying to get new customers.
For deployment: if you can tolerate rare service downtimes then it is difficult to beat Linode or Digital Ocean. I am probably in a minority, but I also really like GCP’s AppEngine: turn on billing, set the minimum instance count to one and the maximum to two or three, and you have a resilient system. That said, I run my primary web site on a single tiny GCP VPS instance with no fault tolerance.
For heavier compute requirements, I also really like a physical server from Hetzner- for about $100/month you can get a GPU, i7, and lots of RAM and bandwidth.
For deployment: if you can tolerate rare service downtimes then it is difficult to beat Linode or Digital Ocean. I am probably in a minority, but I also really like GCP’s AppEngine: turn on billing, set the minimum instance count to one and the maximum to two or three, and you have a resilient system. That said, I run my primary web site on a single tiny GCP VPS instance with no fault tolerance.
For heavier compute requirements, I also really like a physical server from Hetzner- for about $100/month you can get a GPU, i7, and lots of RAM and bandwidth.
Thanks for the "A Company of One" book recommendation. I just ordered it.
I run a small team, but we do a large number of projects. Each project is developed fast and then maintained at low intensity for a long time. Have found great use for a nifty complex DevOps pipeline.
Before, when I built the same app for two years, the nifty pipeline was overwrought.
I think the context switching inherent in what I do now makes the nifty pipeline useful.
Before, when I built the same app for two years, the nifty pipeline was overwrought.
I think the context switching inherent in what I do now makes the nifty pipeline useful.
I see this "Push for new and shiny solutions"[1] across everything from startups to large corporations. And I am genuinely surprised at how often I see people falling for this good old trap of over-optimization/engineering. I genuinely cannot fathom why people want the complexity.
It is by no means constrained to infrastructure, it is architecture, technology, management the whole lot.
[1]http://jesperreiche.com/the-push-for-new-and-shiny-solutions...
It is by no means constrained to infrastructure, it is architecture, technology, management the whole lot.
[1]http://jesperreiche.com/the-push-for-new-and-shiny-solutions...
The problem with Kubernetes (and microservices in general) is getting a team productive with it, and that requires lots of time, dedication, and tooling. It's not the right solution for most teams, but it could be if we streamline it more
This is a pretty dumb article.
Like, no shit don't build more infrastructure than you need. If you're a one or two man team you don't need anything more complicated than 'push container here, pull down here' or 'push code here' et. al. Once you get above that having a repeatable process (repeatable by someone besides the person who built it) becomes valuable.
The 'every second spent on infrastructure isn't spent on features' line makes me think I'm going to have to blackmail into working on security updates or paying down technical debt. And the picture you picked is cheeky.
I will agree with this statement, "You need the most simple, least time consuming infrastructure that gets you to that point." But that WILL become more than just 'throw it on a linode instance' after you get some customers and more than 2 people working on whatever uber for dogs project you have going on.
Like, no shit don't build more infrastructure than you need. If you're a one or two man team you don't need anything more complicated than 'push container here, pull down here' or 'push code here' et. al. Once you get above that having a repeatable process (repeatable by someone besides the person who built it) becomes valuable.
The 'every second spent on infrastructure isn't spent on features' line makes me think I'm going to have to blackmail into working on security updates or paying down technical debt. And the picture you picked is cheeky.
I will agree with this statement, "You need the most simple, least time consuming infrastructure that gets you to that point." But that WILL become more than just 'throw it on a linode instance' after you get some customers and more than 2 people working on whatever uber for dogs project you have going on.
One of my web sites is run by a "bundle exec rails s" in a tmux window on an EC2 instance. New deploy? Run "git pull" (and maybe "rake db:migrate") in tmux pane and restart the "rails s" command. Easy peasy.
I'd make it more complicated and resilient but why?
I'd make it more complicated and resilient but why?
Sure, Kubernetes, fully automated CI/CD, autoscaling, lambda functions are all cool and very useful if you have a problem and paying users in that scale.
Majority of the applications even today in 2019 would be served well by the reliable Nginx + (Python, C++, Rust, Go or other language of choice), PostgreSQL and backups. If you keep things simple, you can even have a standby mirror of the same infrastructure to take over if needed, and it would still cost you less than spinning up nodes after nodes for the complexity of running your own Kubernetes.