Why companies still want in-house data centres(economist.com)
economist.com
Why companies still want in-house data centres
https://www.economist.com/business/2023/10/05/why-companies-still-want-in-house-data-centres
91 comments
Cloud makes economic sense in two scenarios.
For startups that don't have a a clue what they will need because still searching for market fit so it doesn't make sense to provision hardware not knowing what is needed. If you suddenly find market fit and usage grows 100x+ in a few months, you totally want to be on cloud.
And in larger coporations, those that have extremely spiky usage. For example a company that has 90% of their usage at the end of each month, it doesn't make sense to own hardware to meet the peak load if it will be idle 28 days of the month.
Most corporations are not in those two categories though! If you are not, you are vastly overpaying for compute, storage and network compared to owning your own hardware. It is sad that such companies are rushing to overpay for cloud just because it's hip.
Objectively analyze your needs and usage and make a data-driven decision. Don't rush into cloud just because everyone else did.
For startups that don't have a a clue what they will need because still searching for market fit so it doesn't make sense to provision hardware not knowing what is needed. If you suddenly find market fit and usage grows 100x+ in a few months, you totally want to be on cloud.
And in larger coporations, those that have extremely spiky usage. For example a company that has 90% of their usage at the end of each month, it doesn't make sense to own hardware to meet the peak load if it will be idle 28 days of the month.
Most corporations are not in those two categories though! If you are not, you are vastly overpaying for compute, storage and network compared to owning your own hardware. It is sad that such companies are rushing to overpay for cloud just because it's hip.
Objectively analyze your needs and usage and make a data-driven decision. Don't rush into cloud just because everyone else did.
I’ve never worked at a small company when on-prem is cheaper, because having to provision for peak load is not cost effective.
> having to provision for peak load is not cost effective.
That is true, if peak load is much much higher than mean load.
For most companies, that's not the case.
On the cloud you are overpaying for cpu, network and storage by a very wide margin. That makes sense if your peak load is, say, 100x your mean load (and you have sufficienly smart autoscaling to optimize for this).
But for most companies there is not such huge multiplier from mean usage to peak usage. So those are better off with owning hardware.
That is true, if peak load is much much higher than mean load.
For most companies, that's not the case.
On the cloud you are overpaying for cpu, network and storage by a very wide margin. That makes sense if your peak load is, say, 100x your mean load (and you have sufficienly smart autoscaling to optimize for this).
But for most companies there is not such huge multiplier from mean usage to peak usage. So those are better off with owning hardware.
A lot of small companies have very stable workloads for years. Unless you meant startups?
That's a sales line. Or do you work exclusively for small companies that are selling high demand online services that're trying really hard to go viral?
The only other time I can imagine is if a company isn't going to exist in six months, such as movie productions, or companies that have so little going on that they don't need more than a $10 a month plan.
Otherwise, on premises or colocation are invariably cheaper when there's any kind of real usage.
The only other time I can imagine is if a company isn't going to exist in six months, such as movie productions, or companies that have so little going on that they don't need more than a $10 a month plan.
Otherwise, on premises or colocation are invariably cheaper when there's any kind of real usage.
I’ve worked for a small telecom where load was variable, new contracts sometimes meant double or triple the capacity, other times we had smaller contracts without bursty load.
The other company was software for automotive companies and marketing, only US based so traffic was non existent from midnight to 5:00 am, and some companies would need to ramp up for events like NFL games when they’d advertise.
The other company was software for automotive companies and marketing, only US based so traffic was non existent from midnight to 5:00 am, and some companies would need to ramp up for events like NFL games when they’d advertise.
It just depends on the business, b2b can be very consistent. Some workloads just have predictable behavior.
I worked at a medium sized place where we had plans in place to handle the workload for our entire market on-prem (if we were ever to capture that market).
We realized that horizontal scalability wasn't important to us because we had enough of the market to know that the other half of the market could be supported on our existing solution.
I worked at a medium sized place where we had plans in place to handle the workload for our entire market on-prem (if we were ever to capture that market).
We realized that horizontal scalability wasn't important to us because we had enough of the market to know that the other half of the market could be supported on our existing solution.
If your market is regional and not worldwide, you almost assuredly have diurnal load spikes and periods of time where you have almost no activity.
Some services just don't stop at night. Healthcare has trends but it never stops.
It becomes cheaper because they will run long after support has ended and cobble together a working server from the pile of broken ones, all the while rejecting getting a new server that could handle and replace dozens.
datacenter is to information what power plant is to electricity
Economically, scale makes the centralized solution more efficient. That doesn't preclude independent entities running their own stoves, or running their own internal combustion engines, or their own industrial furnaces, but confines those uses to niche areas where direct power generation is more convenient/efficient due to additional constraints.
Politically, centralization means power. Power seeking entities will eventually outright ban independent power generation, for whatever excuse du jour. See NY ban of gas stoves, or EU plans to ban ICE cars. For information, 'to prevent illegal activity' excuse is usually used.
Economically, scale makes the centralized solution more efficient. That doesn't preclude independent entities running their own stoves, or running their own internal combustion engines, or their own industrial furnaces, but confines those uses to niche areas where direct power generation is more convenient/efficient due to additional constraints.
Politically, centralization means power. Power seeking entities will eventually outright ban independent power generation, for whatever excuse du jour. See NY ban of gas stoves, or EU plans to ban ICE cars. For information, 'to prevent illegal activity' excuse is usually used.
> datacenter is to information what power plant is to electricity
Very much not.
Running a generator at home is much more expensive than paying your utility for electricity. But running a server at home is far cheaper than paying AWS if you actualy use the server.
Very much not.
Running a generator at home is much more expensive than paying your utility for electricity. But running a server at home is far cheaper than paying AWS if you actualy use the server.
I don't have much contact with cloud, but it just seems expensive and unstable in practice from business perspective.
Cloud offers standardization and support, so in some cases it might be preferable, because it meets requirements without spending additional time on setup or operations, but eventually cost might become problem, especially if cloud providers will decide to increase prices and you can't easily migrate it to on-premise.
Probably it's best to stay hybrid and be always ready to at least change cloud providers, so you need to avoid vendor lock-in. That sounds like cloud native, so why bother with cloud, if you can have some servers, use them with Rancher and create Kubernetes clusters?
There will be some overhead, but your company controls servers, can easily migrate also to cloud, if needed, provides most functionality that most teams may require and depends only on electricity, internet providers and workers required to operate some data center, so it can always choose cheapest option to operate.
Cloud offers standardization and support, so in some cases it might be preferable, because it meets requirements without spending additional time on setup or operations, but eventually cost might become problem, especially if cloud providers will decide to increase prices and you can't easily migrate it to on-premise.
Probably it's best to stay hybrid and be always ready to at least change cloud providers, so you need to avoid vendor lock-in. That sounds like cloud native, so why bother with cloud, if you can have some servers, use them with Rancher and create Kubernetes clusters?
There will be some overhead, but your company controls servers, can easily migrate also to cloud, if needed, provides most functionality that most teams may require and depends only on electricity, internet providers and workers required to operate some data center, so it can always choose cheapest option to operate.
Big problem with on-prem is lead time on expansion. Can easily slow down expansion. Hard to project out.
But if you have high utilization stable load, it can pay off to have on-prem. It's still a big risk, though. Cloud has a stable platform to build off. You can trust the 'interface' so to speak. Meanwhile you suddenly have to worry about driver incompatibilities, disk wear, stuff like that.
We have cloud infra and on-prem infra. Big problem with cloud is performance for price.
Even for small company with stable load, think about it. You have to have a guy aware of and thinking of the fact that the Mellanox SFP+ card you have has a driver regression on Linux 6.2. Maintenance nightmare. Better to have cloud. Clear interface to build against, and easy to replicate when problems occur.
Sysadmins do stuff. You can't go sysadmin-free and expect efficient spending. Someone else just ends up sysadmin and frequently someone too high paid so you're losing money on the time they play that role.
But if you have high utilization stable load, it can pay off to have on-prem. It's still a big risk, though. Cloud has a stable platform to build off. You can trust the 'interface' so to speak. Meanwhile you suddenly have to worry about driver incompatibilities, disk wear, stuff like that.
We have cloud infra and on-prem infra. Big problem with cloud is performance for price.
Even for small company with stable load, think about it. You have to have a guy aware of and thinking of the fact that the Mellanox SFP+ card you have has a driver regression on Linux 6.2. Maintenance nightmare. Better to have cloud. Clear interface to build against, and easy to replicate when problems occur.
Sysadmins do stuff. You can't go sysadmin-free and expect efficient spending. Someone else just ends up sysadmin and frequently someone too high paid so you're losing money on the time they play that role.
People say cloud is cheaper, but that's based on horrible assumptions such as the idea that companies don't have staff that know how to maintain a server, so cloud costs have to be compared with in-house plus the salary of a systems administrator.
People forget that services in the cloud require technical know how, and that the lack of technical know how can easily result in wildly insane cloud bills.
Really, though, any heavily used service will always cost more in the cloud. Transfer? Sure, bringing Internet to your location might cost an arm and a leg, but it's trivial to find colocation with transfer that's significantly cheaper than anything cloud.
CPU? Definitely. Ancient Xeon cores are a dime a dozen, but they don't compare to even an inexpensive last generation Ryzen. Storage? Absolutely, and the price differences are absolutely ridiculous. Even GPU workloads are cheaper if you have to run them for more than a handful of months.
"But you can scale in the cloud!" Right. You can scale in real life, too, AND you can scale in real life WITH the cloud, if you want, so that's a silly thing to say. It's not one or the other, exclusively.
I have yet to find a single example of a company that isn't doing really silly things saving money by moving to the cloud.
People forget that services in the cloud require technical know how, and that the lack of technical know how can easily result in wildly insane cloud bills.
Really, though, any heavily used service will always cost more in the cloud. Transfer? Sure, bringing Internet to your location might cost an arm and a leg, but it's trivial to find colocation with transfer that's significantly cheaper than anything cloud.
CPU? Definitely. Ancient Xeon cores are a dime a dozen, but they don't compare to even an inexpensive last generation Ryzen. Storage? Absolutely, and the price differences are absolutely ridiculous. Even GPU workloads are cheaper if you have to run them for more than a handful of months.
"But you can scale in the cloud!" Right. You can scale in real life, too, AND you can scale in real life WITH the cloud, if you want, so that's a silly thing to say. It's not one or the other, exclusively.
I have yet to find a single example of a company that isn't doing really silly things saving money by moving to the cloud.
You left out all of the other costs of running a datacenter. Cooling, fire suppression, electricity, DR site, etc.
No, I didn't leave them out. They're just not a significant factor because those costs are proportional to need and demand. A company is either large enough to have enough that those things are aggregated across tons of servers, or they're likely colocating. Also, datacenters don't require all of that to be datacenters.
You did leave them out because most people _want_ electricity in their datacenter, or have DR, etc. These aren't free.
And colocation is not free -- you pay for all of those things on top of the hardware you have to plan for peak cycles on. Nor what you were talking about in the OP, hence ignoring colo. Please don't move goal posts. This is HN, not reddit.
And colocation is not free -- you pay for all of those things on top of the hardware you have to plan for peak cycles on. Nor what you were talking about in the OP, hence ignoring colo. Please don't move goal posts. This is HN, not reddit.
Your colocation provider handles all that for the flat monthly fee you're paying them for the rack space.
> Cooling, fire suppression, electricity, DR site, etc.
No, unless you are particularly huge (google, facebook), you rent space in a colocation facility that handles the real estate stuff (like electricity, cooling, fire). That is the best of all worlds for most companies needs.
No, unless you are particularly huge (google, facebook), you rent space in a colocation facility that handles the real estate stuff (like electricity, cooling, fire). That is the best of all worlds for most companies needs.
The author of the article makes a claim about using 5g technology to send the data to the DC; which would mean that it can’t be far away.
Author clearly does not understand wireless networking.
Ubiquiti sells consumer grade wireless dishes that beams a signal between two dishes point-to-point at distances of a mile with speeds of 250mpbs or greater.
I remember when they released there newest dish a few years ago and one of the IT YouTubers did a test of setting up the dish on a basic lan on two different college campus that were miles across town from each other.
It was v1 of the dish, and he got near 1gbps file transfer speeds between the two dishes without much setup or tuning.
Author clearly does not understand wireless networking.
Ubiquiti sells consumer grade wireless dishes that beams a signal between two dishes point-to-point at distances of a mile with speeds of 250mpbs or greater.
I remember when they released there newest dish a few years ago and one of the IT YouTubers did a test of setting up the dish on a basic lan on two different college campus that were miles across town from each other.
It was v1 of the dish, and he got near 1gbps file transfer speeds between the two dishes without much setup or tuning.
Suspect the author is way more confused than that. The 5G distance comment, if it means anything, presumably refers to the distance between the "field-deployed" servers and the 5G tower. That does have to be quite close typically. But the mention of 5G is probably paid content because as you note, anyone with a real purpose would just deploy a fixed wireless p2p link themselves.
It’s not really a 5G tower..
It literally looks like a Wireless Access Point, and can double as one sometimes.
You plug a network cable into a power supply for dedicated power over POE, and another network cable into your LAN. On the other side where you other dish is, you do the exact same. Plug the network cable into a LAN, and boom you will magically have the devices on both sides on the same network.
The dishes just have to a clear line of sight, and you can get some insane distances.
There is a community mesh WiFi network in New York City that uses these exact ubiquiti dishes to beam wireless high speed trunks across the water to the different burrows; and to uptown.
For just $800, you can set up a pair of air fiber dishes, and be able to beam a 2gbps trunk up to 7 miles away.
https://store.ui.com/us/en/pro/category/all-60ghz-wireless/p...
A few years ago, we moved to a larger space down the street; but Verizon couldn’t get fiber installed in time of our move out date of our existing location.
We set up dishes on each building as they had line of sight fortunately. We ran a VOIP network over it for a month without any hiccups. Even in the worst storms, we didn’t see any speed slow downs or packet loss. It was very impressive.
It literally looks like a Wireless Access Point, and can double as one sometimes.
You plug a network cable into a power supply for dedicated power over POE, and another network cable into your LAN. On the other side where you other dish is, you do the exact same. Plug the network cable into a LAN, and boom you will magically have the devices on both sides on the same network.
The dishes just have to a clear line of sight, and you can get some insane distances.
There is a community mesh WiFi network in New York City that uses these exact ubiquiti dishes to beam wireless high speed trunks across the water to the different burrows; and to uptown.
For just $800, you can set up a pair of air fiber dishes, and be able to beam a 2gbps trunk up to 7 miles away.
https://store.ui.com/us/en/pro/category/all-60ghz-wireless/p...
A few years ago, we moved to a larger space down the street; but Verizon couldn’t get fiber installed in time of our move out date of our existing location.
We set up dishes on each building as they had line of sight fortunately. We ran a VOIP network over it for a month without any hiccups. Even in the worst storms, we didn’t see any speed slow downs or packet loss. It was very impressive.
If the cloud was truly cost-effective then AWS and its competitors wouldn't be the cash cows that they are.
This is partly true. The reason AWS et al ever grew in the first place was also that managing data centers well and automatically used to be a lot harder, and was assumed to be a whole org's full-time mostly manual job. AWS, Google, et al showed how well this could be done with software, showed the benefits of making it easier to do even for app engineers, and incentives shifted. Tools have significantly improved, though, and we're getting back to the point where it can be done with OSS and a team or three of engineers who really know the stacks involved.
In my opinion, the promised "simplicity" of the cloud never materialized. You've replaced physical hardware that's easy to reason about with opaque, unpleasant and eventually-consistent APIs along unnecessary complexity (VPCs, etc) only designed to vendor-lock you even more into the provider. The staff savings never materialized - every company I've been involved with had at least one "DevOps" engineer, but instead of fighting with hardware he's constantly messing with YAML/Terraform/CDK files.
The alternative isn't that different however. A company running its own centre may choose to go OpenStack/k8s. It still requires lots of messing about with awful files. The 'cost' then shifts elsewhere, and merely satisfies bean counters.
IMO the simplicity promised does exist, it's up to the company how it gets implemented. Like either toolstack, if it's misused, it will cost more than they thought (and then they'll blame the toolstack)
IMO the simplicity promised does exist, it's up to the company how it gets implemented. Like either toolstack, if it's misused, it will cost more than they thought (and then they'll blame the toolstack)
I've tried several times and never got an on-prem k8s working. I use proxmox now and never looked back.
The cloud is cost-effective when you start, because you can start using a cheap or even a free plan, and get many pre-configured services.
The cloud is cost-effective when your load is highly spiky, so you can spin up a ton of capacity during the spike, but not pay for it during the lull. (I worked in a company for which this was a principal case.)
If you are a relatively large company and want compute mostly for internal IT needs, especially for sensitive data, it may be cheaper to run your own server room and a private cloud. It won't be cheap in absolute terms though.
The cloud is cost-effective when your load is highly spiky, so you can spin up a ton of capacity during the spike, but not pay for it during the lull. (I worked in a company for which this was a principal case.)
If you are a relatively large company and want compute mostly for internal IT needs, especially for sensitive data, it may be cheaper to run your own server room and a private cloud. It won't be cheap in absolute terms though.
I work as an IT Infrastructure Architect for VMware and Windows Based Enterprise workloads and a lot of the time it's just more cost effective to have the servers in house for many of the workloads. It isn't the case for ALL workloads, for example we move almost all exchange email services to office 365, in combination with Azure AD for active directory service. As far as datacenter space and power usage, a lot of the time the server room is located in the utility room / closets, they already have the space and the building and the power infrastructure. Typically they have vendors that require VM's because the vendor software is not cloud based, many of these enterprise software can consume quite a lot of CPU/RAM/DISK and are very latency sensitive. Having a couple of powerful vmware hosts and an all NVMe SSD SAN is really really nice for performance and speeding up these applications and not paying 4K+$ a month for equivalent hardware in the cloud. You can even argue against the capital investment cost because a lot of this gear you can lease directly from DELL, and have full service contract for 7 years 24x7 4 hour SLA on hardware.
Depending on what your company does even if it starts out and gets free credits through the AWS startup program it still might not be cheaper.
But it's more flexible and a lot of startups often pivot a view times.
Through if you have high compute and somewhat longer term predictable levels of compute then e.g. renting a rack in a server center and placing your hardware in can be notifiable cheaper (when your book keeping splits the initial buying costs across the years you run the hardware)
And then there is one huge lie about cloud, that it's way less work. It's not. It's a mess. I mean seriously we have a few docent years now of experience about what is user friendly and what is not when it comes to API/interface design and I would argue ~90% of cloud design from AWS is just plain terrible. That costs time, and time is money.
But it's more flexible and a lot of startups often pivot a view times.
Through if you have high compute and somewhat longer term predictable levels of compute then e.g. renting a rack in a server center and placing your hardware in can be notifiable cheaper (when your book keeping splits the initial buying costs across the years you run the hardware)
And then there is one huge lie about cloud, that it's way less work. It's not. It's a mess. I mean seriously we have a few docent years now of experience about what is user friendly and what is not when it comes to API/interface design and I would argue ~90% of cloud design from AWS is just plain terrible. That costs time, and time is money.
Startups prefer cloud because:
1) a lot of startups were never about building a sustainable business but to live the "startup founder" lifestyle and provide cushy jobs for your friends. In this case, complexity is a feature as it gives you an excuse to raise a lot of money, hire a lot of people and give those people tons of busywork to do (which they wouldn't have with a simpler stack).
2) even for startups who do want to build a sustainable business, VCs are often close-minded and anything that doesn't fit the typical "engineering playground" mold described above may fail technical due diligence despite it being better from an engineering point of view. So cloud and overengineering it is.
1) a lot of startups were never about building a sustainable business but to live the "startup founder" lifestyle and provide cushy jobs for your friends. In this case, complexity is a feature as it gives you an excuse to raise a lot of money, hire a lot of people and give those people tons of busywork to do (which they wouldn't have with a simpler stack).
2) even for startups who do want to build a sustainable business, VCs are often close-minded and anything that doesn't fit the typical "engineering playground" mold described above may fail technical due diligence despite it being better from an engineering point of view. So cloud and overengineering it is.
> "startup founder" lifestyle and provide cushy jobs for your friends
I have yet to see a startup for which that is the case, like at all. Idk if it's a SV thing but in most of the world this won't fly.
Most important you are missing the most important reason why startup go with the cloud.
Flexibility.
Startups often pivot multiple times in various aspects before they find a way approach substainability. Like common are changes in monetization model and changes in target audience. This often also comes with major changes in requirements for compute/hardware. This isn't just about amount of resources but also kind of them. Furthermore it also often comes with changes in architecture where services provides by AWS and similar can make such changes much faster/easier.
This means that for startups setting up their own hardware is a form of self-locking which startups can not always afford, and can even lead to a startup failing.
Naturally once a company has found a promising way to reach profitability the cloud vs. non cloud decision needs to be reconsidered.
I have yet to see a startup for which that is the case, like at all. Idk if it's a SV thing but in most of the world this won't fly.
Most important you are missing the most important reason why startup go with the cloud.
Flexibility.
Startups often pivot multiple times in various aspects before they find a way approach substainability. Like common are changes in monetization model and changes in target audience. This often also comes with major changes in requirements for compute/hardware. This isn't just about amount of resources but also kind of them. Furthermore it also often comes with changes in architecture where services provides by AWS and similar can make such changes much faster/easier.
This means that for startups setting up their own hardware is a form of self-locking which startups can not always afford, and can even lead to a startup failing.
Naturally once a company has found a promising way to reach profitability the cloud vs. non cloud decision needs to be reconsidered.
A bit less cynically, a startup is a company designed for enormous growth. If not, it's just a sparkling regular business.
If your user base grows 100x in a year, there's no easy way to scale using your own hardware. But this is exactly the (rare) case the VCs are after. So a startup is required to build their tech in the cloud, even if in 99% of the cases it's an overkill and an extra expense.
If your user base grows 100x in a year, there's no easy way to scale using your own hardware. But this is exactly the (rare) case the VCs are after. So a startup is required to build their tech in the cloud, even if in 99% of the cases it's an overkill and an extra expense.
If you have enough hard technical changes due to changes in monetization model or similar it might not even have any extra expenses. I think the company I currently work for might have been bankrupt by now (after due to a marked change our original monetization strategy started to dry up) if we hadn't taken advantage of the flexibility of the cloud. (But then we still might not make it, but at least we had a chance to make it which I don't see us having had we tried to have our own dedicated hardware.)
Then there are also some highly managed/opinionated "cloud providers" (e.g. not aws), while they tend to have even more absurd prices for compute power they tend to actually safe quite a bit man power (in difference to e.g. AWS where the person time savings seam to be completely eaten up by subtle complexity and need for optimizing deployments in ways you otherwise would not need to). Which can make them attractive for very small startups which very low compute requirements. Through such startups also tend to not have calssical VC.
Then there are also some highly managed/opinionated "cloud providers" (e.g. not aws), while they tend to have even more absurd prices for compute power they tend to actually safe quite a bit man power (in difference to e.g. AWS where the person time savings seam to be completely eaten up by subtle complexity and need for optimizing deployments in ways you otherwise would not need to). Which can make them attractive for very small startups which very low compute requirements. Through such startups also tend to not have calssical VC.
> The cloud is cost-effective when you start, because you can start using a cheap or even a free plan
Is it? The marginal cost of setting up a old computer as a server is essentially zero.
I think the main benefit is that devs can spend $$$ on AWS without doing a procurement. I.e. AWS solves a beurocracy problem not a technical one.
Is it? The marginal cost of setting up a old computer as a server is essentially zero.
I think the main benefit is that devs can spend $$$ on AWS without doing a procurement. I.e. AWS solves a beurocracy problem not a technical one.
Are you going to run your production from an old desktop colocated at the cheapest DC nearby?
You must know how to run a really scrappy startup!
All of a sudden, some post on Reddit brings in a wave of visitors and a large bunch of signups; your hardware barely manages. What's your next action? (Note: `kubectl scale` is not it.)
You must know how to run a really scrappy startup!
All of a sudden, some post on Reddit brings in a wave of visitors and a large bunch of signups; your hardware barely manages. What's your next action? (Note: `kubectl scale` is not it.)
The cloud is only cost-effective if your workload fits into their loss leaders where they're actually giving you reasonably-priced services. Anything else and you're still getting scammed. It may be that the amount is too low to matter and the pragmatic decision is to keep going for the time being, but you're still getting scammed.
The fast scaling/up down is a good argument but you need to proceed with caution:
1) is your application actually structured to be able to do this? Would the overheads of developing your application in this way negate the savings (a fully distributed service with queues/etc has more development/management overhead than your basic monolithic MVC app, which may eat up all your savings)?
You need to do your own numbers here and be pragmatic. You'll have plenty of resume-driven people who see the potential of an engineering playground and will immediately tell you whatever it takes to make this happen, regardless of whether it will benefit the business.
2) how much does it actually cost to run your application at full scale outside of the cloud? I've seen places that do the whole autoscaling thing (and have lots more moving parts as a result) and yet a fully-scaled-up equivalent on bare-metal would cost less than the baseline scale on the cloud, thus removing the business case for it.
Keep in mind that the cloud effectively ate all the hardware improvements we've had in the last decade so switching to bare-metal hosting means you'll need a lot less hardware. A cloud "vCPU" is more like a pocket calculator when compared to a real CPU, and same with storage.
The fast scaling/up down is a good argument but you need to proceed with caution:
1) is your application actually structured to be able to do this? Would the overheads of developing your application in this way negate the savings (a fully distributed service with queues/etc has more development/management overhead than your basic monolithic MVC app, which may eat up all your savings)?
You need to do your own numbers here and be pragmatic. You'll have plenty of resume-driven people who see the potential of an engineering playground and will immediately tell you whatever it takes to make this happen, regardless of whether it will benefit the business.
2) how much does it actually cost to run your application at full scale outside of the cloud? I've seen places that do the whole autoscaling thing (and have lots more moving parts as a result) and yet a fully-scaled-up equivalent on bare-metal would cost less than the baseline scale on the cloud, thus removing the business case for it.
Keep in mind that the cloud effectively ate all the hardware improvements we've had in the last decade so switching to bare-metal hosting means you'll need a lot less hardware. A cloud "vCPU" is more like a pocket calculator when compared to a real CPU, and same with storage.
It's amazing how people don't see this. Using the clouds is just like renting. In the long term it's more expensive. Cloud operators are not charities.
Yet many/most companies rent their offices or facilites, because they don't want to be in the property management business. It may cost more in the end, but it's more predictable and lower risk.
I think the analogy works best for SaaS but not necessarily for PaaS in most cases.
I think the analogy works best for SaaS but not necessarily for PaaS in most cases.
It easy to move staff from the officeA to the officeB in the same city, in fact I witnessed such moves multiple times. However moving from AWS to Azure is not that easy at all!
Right and if certain times of the year you need an eight bedroom house for a week, while other times a two bedroom is fine, then renting may be way cheaper than buying an eight bedroom house.
This analogy doesn't really apply because thanks to bandwidth charges, it's uneconomical to have a 2-bed house (outside of the cloud) and occasionally rent the 8-bed in the cloud, so you're forced to also get the 2-bed one in the cloud and pay insane margins on that too.
Extending this metaphor, I would then want to own my two bedroom home, and take a vacation to the rented mansion.
Is there a well trodden way to serve most of your everyday traffic out of your own datacenter, but scale out to cloud infra when traffic peaks?
Is there a well trodden way to serve most of your everyday traffic out of your own datacenter, but scale out to cloud infra when traffic peaks?
It's doable but the egress bandwidth charges (which are there to prevent exactly this) from most clouds may negate any savings.
[deleted]
That is actually the perfect analogy. It is exactly like rent vs buy housing discourse, in that many people harbor irrational preferences for buying even when renting is demonstrably cheaper.
the buying vs renting calculus depends on the respective prices and rates. now think what happens if the entire rentable stock of housing was owned by three companies.
They also need to renew infrastructure on a continual basis
Or simply that the benefits of economies of scale are enormous in this area.
They are not.
it's complicated, but in general server centers can profite enormously from economy at scale, but also handling that many customers at scale adds additional cost compared to just having that much compute for your own use case, so does handing out tones of free credits to startups, and running a company (personal wise) at scale is often highly inefficient, too. Especial if you make things unnecessary complicated.
but mainly, AWS and similar do not have a need to lower their raw compute price to that level because even without they get enough customers, e.g. there had been cases of Microsoft Azure in some locations running out of spare compute resources leading to issues for customer.
but mainly, AWS and similar do not have a need to lower their raw compute price to that level because even without they get enough customers, e.g. there had been cases of Microsoft Azure in some locations running out of spare compute resources leading to issues for customer.
> in general server centers can profite enormously from economy at scale
That's plain false.
That's plain false.
no it's not
most things compute related once you reach a certain minimal scale (i.e. need room cooling in your server room, etc.) get cheaper with scale sometimes quite massively
to list some
- hardware cost, sometimes you can't even get the hardware you need/want on smaller scale
- electricity cost (through it's complicated, if you can't choose the place for the data center electricity cost only up to some location specific scale)
- overhead for reliability (e.g. multiple power connections from different providers, multiple wired fast internet connections from different providers etc.)
- fast low latency internet access (what apartments get is often not cutting it at all even for quite small scale)
- man power for hardware maintenance (a single person can do hardware maintenance for quite a lot of units, having a team of such people makes handling sick cases, quitting and similar much easier)
- custom software tooling (amortizes more)
- physical access control (has a pretty high fix cost if you want to get it right, but after which it scales somewhat linearish, assuming we limit ourself to a single building or small complex)
- revenue also get's more regulare with more customers, which makes expensive unepxpected revenue drops due to e.g. a customer going insolvent much easyer to handle and as such cheaper
- hardware spare part handling gets easier and cheaper on scale
- ...
And sure there is an upper limit per-data center for scaling which is dependent on locality, but there are enough locations where this limit is quite high.
And sure if you run an such a small scale that you don't even have to consider having a "proper" server room with cooling system which is more then just fans moving heat out of racks you often also can cut many other costs, like power redundancy, network redundancy, 24/7 hardware fixing support, etc. Through then if you have some B2B services with reliability guarantees you often can't cut that costs no matter how small the scale (that btw. can be a reason enough to use the cloud, providing the necessary reliability with custom hardware on a small (compute) scale is very expensive as it involves many high fix costs)
Idk. if there was a misunderstanding so just to be clear, this is about cost of a data center per-compute unit getting cheaper with scale (up to some limit). Not about running your own ad-hoc reliability nightmare datacenter vs. renting cloud resources.
most things compute related once you reach a certain minimal scale (i.e. need room cooling in your server room, etc.) get cheaper with scale sometimes quite massively
to list some
- hardware cost, sometimes you can't even get the hardware you need/want on smaller scale
- electricity cost (through it's complicated, if you can't choose the place for the data center electricity cost only up to some location specific scale)
- overhead for reliability (e.g. multiple power connections from different providers, multiple wired fast internet connections from different providers etc.)
- fast low latency internet access (what apartments get is often not cutting it at all even for quite small scale)
- man power for hardware maintenance (a single person can do hardware maintenance for quite a lot of units, having a team of such people makes handling sick cases, quitting and similar much easier)
- custom software tooling (amortizes more)
- physical access control (has a pretty high fix cost if you want to get it right, but after which it scales somewhat linearish, assuming we limit ourself to a single building or small complex)
- revenue also get's more regulare with more customers, which makes expensive unepxpected revenue drops due to e.g. a customer going insolvent much easyer to handle and as such cheaper
- hardware spare part handling gets easier and cheaper on scale
- ...
And sure there is an upper limit per-data center for scaling which is dependent on locality, but there are enough locations where this limit is quite high.
And sure if you run an such a small scale that you don't even have to consider having a "proper" server room with cooling system which is more then just fans moving heat out of racks you often also can cut many other costs, like power redundancy, network redundancy, 24/7 hardware fixing support, etc. Through then if you have some B2B services with reliability guarantees you often can't cut that costs no matter how small the scale (that btw. can be a reason enough to use the cloud, providing the necessary reliability with custom hardware on a small (compute) scale is very expensive as it involves many high fix costs)
Idk. if there was a misunderstanding so just to be clear, this is about cost of a data center per-compute unit getting cheaper with scale (up to some limit). Not about running your own ad-hoc reliability nightmare datacenter vs. renting cloud resources.
The expected argument is probably something like “it's economy of scale”, but in practice it absolutely looks the other way round: Amazon is much less profitable than it is more expensive than the alternatives[1], so if anything their seems to be negative economies of scales.
[1] if you're twice as expensive than a company that has say 10% margin, then with equal costs you should be with a 55% margin. And Amazon is more often than not more than twice as expensive as the alternatives.
[1] if you're twice as expensive than a company that has say 10% margin, then with equal costs you should be with a 55% margin. And Amazon is more often than not more than twice as expensive as the alternatives.
That is a bogus economic argument. There’s plenty of room in the equation for cloud operators to be profitable offering a product that is cheaper than on-prem.
yes but current AWS and similar mainly win through flexibility and services they provide not on cost for a immutable compute capacity. For that nearly always getting your own hardware is cheaper when you look at a time span of multiple years (but running it in a rented rack in a server center, there are services like that).
Which in practice doesn't seem to be happening though. The cloud marketing/misinformation machine is working well enough that there's no point actually becoming cheaper than on-prem since the marketing is successfully convincing people to overpay.
Economies of scale negate this reasoning. (Among several other factors).
AWS charges 90x more for bandwidth than conventional hosting companies. The economies of scale are in Jeff Bezos' pocket instead of being passed to customers.
> The decision to build or rent may depend on a user’s data intensity.
Think of brick-and-mortars real estate (e.g. office space) which for most companies is completely outside their core value proposition hence they would be most inclined to lease. Accurate statistics are surprisingly non-trivial to find but it seems like the company owned segment is circa 20% (versus 80% leasing their office space needs).
Now when you think of the role data centers play in a growing digital economy, for many companies increasingly their value proposition is linked to their digital infrastructure. So that would push the fraction of "owned" versus "leased" substantially higher. This would leave still a eye-wateringly massive amount to be served by "hyperscalers". This is really what the "cloud revolution" (renting somebody elses linux server) is mostly about. There is a certain naturalness to this state of affairs.
But there is the major catch. Imagine if practially all the real estate in the world is owned by just a handful of mega-corps and anybody needing office space needs to rent from them. Companies small and large will quickly realize what the term "rent-seeking" [1] means... when the seller holds all the cards and the buyer holds none.
Rationally a hyperscaler oligopoly would attempt to keep the price of digital infrastructure at just below the ownership cost, though greed and various frictions in migrating back to in-house might temporarily allow milking higher rents.
In any case its a most interesting parameter to observe in these rapidly changing times
[1] https://en.wikipedia.org/wiki/Rent-seeking
Think of brick-and-mortars real estate (e.g. office space) which for most companies is completely outside their core value proposition hence they would be most inclined to lease. Accurate statistics are surprisingly non-trivial to find but it seems like the company owned segment is circa 20% (versus 80% leasing their office space needs).
Now when you think of the role data centers play in a growing digital economy, for many companies increasingly their value proposition is linked to their digital infrastructure. So that would push the fraction of "owned" versus "leased" substantially higher. This would leave still a eye-wateringly massive amount to be served by "hyperscalers". This is really what the "cloud revolution" (renting somebody elses linux server) is mostly about. There is a certain naturalness to this state of affairs.
But there is the major catch. Imagine if practially all the real estate in the world is owned by just a handful of mega-corps and anybody needing office space needs to rent from them. Companies small and large will quickly realize what the term "rent-seeking" [1] means... when the seller holds all the cards and the buyer holds none.
Rationally a hyperscaler oligopoly would attempt to keep the price of digital infrastructure at just below the ownership cost, though greed and various frictions in migrating back to in-house might temporarily allow milking higher rents.
In any case its a most interesting parameter to observe in these rapidly changing times
[1] https://en.wikipedia.org/wiki/Rent-seeking
I literally built billions of dollars of equipment for my employer deploying new datacenters. Even with a mandate to pivot to cloud-first, that kind of investment still needs to be used. And predictably after suffering enough pain from cloud-preferred thinking, we're settling on a mix, in no small part due to both monetary and migration cost.
Are you doing show/charge back for both on premise and cloud? Do different teams manage each environment?
>Are you doing show/charge back for both on premise and cloud?
Yes, you're billed in either case. But it's the difference between blue and green dollars: money that exits the firm is much more strictly budgeted.
> Do different teams manage each environment?
Sort of. Knowing how to deploy to AWS vs an internally-hosted kube farm vs managing a fleet of virtual machines are all different skills, not that you can't learn them all. At one time our org structure split those functions but now if you go far up enough the management chain is the same.
Yes, you're billed in either case. But it's the difference between blue and green dollars: money that exits the firm is much more strictly budgeted.
> Do different teams manage each environment?
Sort of. Knowing how to deploy to AWS vs an internally-hosted kube farm vs managing a fleet of virtual machines are all different skills, not that you can't learn them all. At one time our org structure split those functions but now if you go far up enough the management chain is the same.
> deploy to AWS vs an internally-hosted kube
This is IMHO one of the absurdities of current could technology.
We have the technology and know-how to not make this different skills, to make it mostly uniform and abstract away all the compute hosting specific parts in the same way that compilers mostly abstract away most ISA/hardware details (at least for most languages and many use-cases, not all)
This is IMHO one of the absurdities of current could technology.
We have the technology and know-how to not make this different skills, to make it mostly uniform and abstract away all the compute hosting specific parts in the same way that compilers mostly abstract away most ISA/hardware details (at least for most languages and many use-cases, not all)
I was disappointed (not that I'm picking on them) to learn Hashicorp's terraform didn't do exactly the sort of abstraction you're talking about. You can cobble together layers of automation, but I share your disappointment that there's not a single solution. Although I will say rackn's digital rebar gets a lot of the way there. You can deploy workloads in a pretty technology-agnostic fashion for the things it does support, and it can be integrated/extended with ansible, terraform and other scripting tools (including plain old bash) or other languages as needed.
Can’t get high utilization modern cpu on cloud for cheap. Ryzen 7950x single core very fast and very cheap.
You want a consumer desktop chip in your server? Consider the I/O.
If he can buy 2 (or more) consumer-grade chips for the price of one enterprise-grade one and split his workload across them then it still makes more sense to go for the consumer version.
Consumer hardware gets undeserved bad rap despite being a good fit for many of today's "cloud-native" and fault-tolerant workloads.
Don't run critical infra on it that can't be fault tolerant (conventional RDBMS), but for your cluster of queue workers? Go right ahead since the queue is designed to handle failure of any nodes gracefully.
Consumer hardware gets undeserved bad rap despite being a good fit for many of today's "cloud-native" and fault-tolerant workloads.
Don't run critical infra on it that can't be fault tolerant (conventional RDBMS), but for your cluster of queue workers? Go right ahead since the queue is designed to handle failure of any nodes gracefully.
I think it's a great choice for a lot of workloads, especially CI.
Some places performance/price matters most. Some places raw performance. 7950x outperforms cloud processor and cheap. But use Epyc in other places. Just empirical optimum.
That article is a bit of a mishmash of factors, pros, cons, and trends
On prem requires a capital investment and a cast iron workload prediction.
If you have a stable business this works. But you may still want to go to cloud if IT is not your business. An on prem infrastructure requires capital and capital can be used to change your core business, which could bring you sufficient bottom line to make cloud costs irrelevant.
If you are trying for growth then a cloud infrastructure enables that.
If you have a stable business this works. But you may still want to go to cloud if IT is not your business. An on prem infrastructure requires capital and capital can be used to change your core business, which could bring you sufficient bottom line to make cloud costs irrelevant.
If you are trying for growth then a cloud infrastructure enables that.
Business isn't just about managing costs.
I still want an in house datacentre. Our move to cloud has a 5x cost increase. We need more staff to run it as well (!).
ah yes -- you need less people -- is probably the biggest lie of modern cloud
through technically it should be that way
so I would argue the current cloud infrastructure is seriously messed up
through technically it should be that way
so I would argue the current cloud infrastructure is seriously messed up
With cloud you can have less staff, but you need to pay more per headcount as they need to know how to manage tens if not hundreds of different services.
Traditional on-prem has silos where the DBA just knows databases and their salary is capped because of that.
I guess in theory you could have a "cloud DBA", but in my experience every business unit wants to keep autonomy over their cloud account/project and do it all in house to get rid of the delays that occur with centralized resources.
Traditional on-prem has silos where the DBA just knows databases and their salary is capped because of that.
I guess in theory you could have a "cloud DBA", but in my experience every business unit wants to keep autonomy over their cloud account/project and do it all in house to get rid of the delays that occur with centralized resources.
> With cloud you can have less staff, but you need to pay more per headcount
I would disagree
in my experience this isn't the case at all
through having less more experienced (and in turn well payed) personal is always a thing no matter weather you speak about on-prem or cloud
and there really is no reason you need to do on-primise like the siloed "Traditional on-prem" as you describe.
I would even say that especially for smaller companies they stopped doing on-prem that way even before the cloud became widely relevant. I mean it's just not feasible and there really isn't a fundamental reason why you have to silo yourself in that way just because you are on-prem.
I would disagree
in my experience this isn't the case at all
through having less more experienced (and in turn well payed) personal is always a thing no matter weather you speak about on-prem or cloud
and there really is no reason you need to do on-primise like the siloed "Traditional on-prem" as you describe.
I would even say that especially for smaller companies they stopped doing on-prem that way even before the cloud became widely relevant. I mean it's just not feasible and there really isn't a fundamental reason why you have to silo yourself in that way just because you are on-prem.
Are data privacy and security really nowhere in the list of considerations?
I find it hard to believe that cloud providers aren't mining data and pulling every possible metric they can from usage. AWS alone has been responsible* for countless breeches and data leaks. Wanting to keep your data out of the hands of third parties for the protection of your own data or that of your users seems like it ought to be a factor in there somewhere when deciding to keep your data in house.
* You can argue that it's mostly been the customers who fail to configure the service correctly at fault, and you wouldn't be wrong, but any time that many people are "doing it wrong" the problem isn't just them, its the service they're using (bad defaults, hidden footguns, unclear documentation, etc).
* You can argue that it's mostly been the customers who fail to configure the service correctly at fault, and you wouldn't be wrong, but any time that many people are "doing it wrong" the problem isn't just them, its the service they're using (bad defaults, hidden footguns, unclear documentation, etc).
Simply put: it makes sense. When you are big enough, why not?
It's not solely a total cost decision but it's also OpEx vs. CapEx, what can be depreciated based on that, etc.
The main reason is a whole generation of IT executives who don’t want to look superfluous.
[deleted]
It's paywalled but...
if someone says it's "unlikely" it either is a stylistic method to goat people or they don't understand much about the technology, because there is nothing unlikely about it
if someone says it's "unlikely" it either is a stylistic method to goat people or they don't understand much about the technology, because there is nothing unlikely about it
The following paragraph is the most illuminating: "The decision to build or rent may depend on a user’s data intensity. Consider a firm in America with a medium-sized data centre which thinks its computing load will rise roughly four-fold over a decade. In that case, building its own breaks even in seven years or so, and ends up being 5% cheaper overall than leasing, according to data from Schneider Electric. If the load remains stable, renting is the less expensive option, by a similar amount (assuming a flat cost of capital)."
This assumes that cloud costs are stagnant and reliable. This is a very naive assumption - it can be seen that if the cloud providers think they can get away with cost increases, they will. On-premises works out cheaper in the majority of cases, even including the cost of a sysadmin team.