We stand to save $7M over five years from our cloud exit(world.hey.com)
world.hey.com
We stand to save $7M over five years from our cloud exit
https://world.hey.com/dhh/we-stand-to-save-7m-over-five-years-from-our-cloud-exit-53996caa
18 comments
Very happy to see this. I have been a silent advocate for bringing back/investing into in-house "cloud" infrastructure for the many SME I've done some work for. I have already seen corporate (non-tech/startup/saas) do this but have yet to see true adoption in-house for more-tech oriented companies
Good idea, however remember to calculate power requirements in the DC and get at least 100% more space than you currently need!
I've seen cases where you want to provision the new rack but unfortunately DC is full and you have to end up running infrastructure spanning across different buildings or even different DCs.
With Zen4 I would go with single CPU option, it's hard to saturate 256 threads.
Main cost in the cloud is always the database server, because it needs to be on 100% of the time, cannot be on spot instance, and needs lots of memory and CPU just in case of a spike of traffic.
I've seen cases where you want to provision the new rack but unfortunately DC is full and you have to end up running infrastructure spanning across different buildings or even different DCs.
With Zen4 I would go with single CPU option, it's hard to saturate 256 threads.
Main cost in the cloud is always the database server, because it needs to be on 100% of the time, cannot be on spot instance, and needs lots of memory and CPU just in case of a spike of traffic.
When companies migrate back to colo/on-prem from cloud are they disabling CPU vuln mitigations for non public facing servers?
On some server generations the performance uplift is huge.
They're using Zen4. Zen4 should have those issues fixed in the hardware.
> Just under a million of that was on storing 8 petabytes of files in S3, fully replicated across several regions
Hol' up. They're replicating data across regions -- across /several/ regions -- on top of a service (S3) which already has 99.999999999% data durability in a single region? ...Why?
Hol' up. They're replicating data across regions -- across /several/ regions -- on top of a service (S3) which already has 99.999999999% data durability in a single region? ...Why?
Latency?
I think it'll be interesting to see.
It seems like they won't move their 8PB of data off the cloud. This seems smart because you can always spin up servers when one goes down, but if your storage system loses important user emails, that's the end of your service.
It seems like they won't move their 8PB of data off the cloud. This seems smart because you can always spin up servers when one goes down, but if your storage system loses important user emails, that's the end of your service.
What do you use when you decide to host 8PB of data yourself?
Now what do you run on top of that, CEPH? NFS shares? Present it as iSCSI?
60x18TB - 1PB raw
$10,000 for a 60 drive chassis
$250/drive = $15,0000
$25,000 total
Now what do you run on top of that, CEPH? NFS shares? Present it as iSCSI?
Ceph, Minio, or some other S3-compatible system.
Minio is dual licensed: AGPS or their enterprise license. Both can be a big no-no for companies. I was wondering what could be the alternative to MinIO, in addition to Ceph.
IMO "never pay for software no matter what" is a red flag to me. Buying a license can be the right choice if it saves you money on ops or hardware.
I see two issues with licensed systems for large enough workload. One is scale of cost. Enterprise software charges by some normalized server configuration (CPU, memory, and etc. MinIO seems an exception, though). As a result, the cost of the license can grow to be prohibitive for large workload. Another is that enterprise licenses usually do not allow users to modify the software easily, while for any significant workload, we almost always need to add code for smooth operations.
> It seems like they won't move their 8PB of data off the cloud.
For now. He wrote they will do that next year.
For now. He wrote they will do that next year.
Good luck to them. I'm sure they can do it but I wouldn't want to be working on that project.
The real question is, how much cloud computing cost would DHH save if he'd stopped using Ruby
That is not the real question. Choosing another language won't fix any of the issues he is addressing with the moving to on-prem servers.
It is irrelevant for the topic.
He wrote a blog post about that topic. IIRC around 10% of their cost comes from Ruby. Meanwhile they're aiming to save ~50% by moving off AWS.
I've been saying for years that cloud is more cost but you end up needing the same number of people if you have a large enough scope.
Scalability isn't actually all that hard, you solve it in cloud or you solve it in hardware.
People are always very quick to say that running on bare metal means you have to start working in nuclear power plants or CPU lithography from sand. It's intellectually dishonest of course, but we need new players in the game that can't be accused of being old and actually weighing the benefits and drawbacks publicly. So, Bravo.