Amazon has more than half of all Arm server CPUs in the world(theregister.com)
theregister.com
Amazon has more than half of all Arm server CPUs in the world
https://www.theregister.com/2023/08/08/amazon_arm_servers/
143 comments
> None of the other cloud providers are using ARM at this scale yet. They’re using Ampere chips, whereas Amazon invested in building their own - it tells you the level of commitment Amazon have here.
James Hamilton was adamant AWS build ARM expertise in-house [0] (one which they accelerated by acquiring Annapurna Labs), and he's been proven right [1] despite the obstacles he himself foresaw [2].
[0] On the Origins of AWS Custom Silicon, https://archive.is/4k5ul
[1] Low cost, low power servers for Internet-scale services, https://mvdirona.com/jrh/talksandpapers/JamesHamilton_CEMS.p...
[2] AWS Designed Processor: Graviton, https://archive.is/WLFBW
James Hamilton was adamant AWS build ARM expertise in-house [0] (one which they accelerated by acquiring Annapurna Labs), and he's been proven right [1] despite the obstacles he himself foresaw [2].
[0] On the Origins of AWS Custom Silicon, https://archive.is/4k5ul
[1] Low cost, low power servers for Internet-scale services, https://mvdirona.com/jrh/talksandpapers/JamesHamilton_CEMS.p...
[2] AWS Designed Processor: Graviton, https://archive.is/WLFBW
> and he's been proven right despite the obstacles he himself foresaw
I wouldn't trust anybody to be right about the benefits of something this big if they don't also foresee the obstacles you'll face to realise those benefits.
I wouldn't trust anybody to be right about the benefits of something this big if they don't also foresee the obstacles you'll face to realise those benefits.
> and our shiny new M1/M2 Macs run the same binaries
Does that mean you're all running asahi linux, or are you somehow running linux elf binaries on macOS?
Or are you using VMs, just like the x86 + docker desktop folks are doing to run the same binaries on their laptops and servers?
Does that mean you're all running asahi linux, or are you somehow running linux elf binaries on macOS?
Or are you using VMs, just like the x86 + docker desktop folks are doing to run the same binaries on their laptops and servers?
You still need VM to run Linux ELF on macOS, but you don't need to use full-blown emulation like QEMU if the Linux ELF is for AARCH64, unlike when you try to run x86 ELF. It's much faster this way.
VMs through things like Docker for Mac, in our case.
And yes - it’s exactly the same as you can do if you use x86 servers and dev machines.
It’s just the fact that we can also do this for ARM on hardware that doesn’t suck that makes using ARM in production more practical for us.
And yes - it’s exactly the same as you can do if you use x86 servers and dev machines.
It’s just the fact that we can also do this for ARM on hardware that doesn’t suck that makes using ARM in production more practical for us.
Just as an aside, the same process works painlessly with podman, too, if your employer is allergic to Docker. Maybe once or twice a year I get reminded that I’m not using x86 - and it sure is nice to work all day and finish with 60% battery.
Is this battery savings from using podman instead of docker for mac or using docker for max with M1/M2?
Running on ARM itself is a big power savings. Docker Desktop also had issues with CPU load which they might have improved since I stopped using it a couple of years ago. There were a bunch of GitHub issues starting around 2018 which were closed, moved, reopened, and it looks like it’s still a concern whereas with Podman I’ve been able to use it without thinking about it.
Sorry I missed the details:
"Does that mean you're all running asahi linux or are you somehow running linux elf binaries on macOS?" So you're running whatever linux districtuion you want or some particular flavor through Docker on Mac? And you don't have ELF binaries because you're using precompiled packages for ARM and don't need weirdo prebuilt ELFs, or you have a workaround?
"Does that mean you're all running asahi linux or are you somehow running linux elf binaries on macOS?" So you're running whatever linux districtuion you want or some particular flavor through Docker on Mac? And you don't have ELF binaries because you're using precompiled packages for ARM and don't need weirdo prebuilt ELFs, or you have a workaround?
Docker for Mac runs a full-blown Linux kernel last time I checked (Docker for Windows can use Windows containers). So whatever OS they run on the cloud (with OCI) these containers are going to work on the (very fast) M1/M2. The OS the Docker images are using is irrelevant. The OS running in the cloud is irrelevant. Only relevant factor is Docker for Mac as it means Linux kernel overhead. But this is offset by M1/M2. Clever setup!
The interesting thing is, how is Docker for Mac able to run x86 containers at speed when Apple's Developer Documentation for Rosetta says it can't be used for virtual machines?
Or have they just embedded the QEMU on-demand translator with binfmt_misc in the ARM virtual machine and the M1/M2 is just powerful enough to make users not notice what's going on?
[1] https://developer.apple.com/documentation/apple-silicon/abou...
Or have they just embedded the QEMU on-demand translator with binfmt_misc in the ARM virtual machine and the M1/M2 is just powerful enough to make users not notice what's going on?
[1] https://developer.apple.com/documentation/apple-silicon/abou...
Rosetta can be used to emulate individual processes inside a linux-based virtual machine - Apple provide tooling for this: https://developer.apple.com/documentation/virtualization/run...
> Docker for Windows can use Windows containers
I thought Windows containers never went anywhere. Interesting to hear that they are still a thing.
I thought Windows containers never went anywhere. Interesting to hear that they are still a thing.
Yes, but pretty much in the same sad state as they were a few years ago. We're running a few in production and next time we touch it we're replacing them with Windows Server Core EC2 instances built from baked images.
Note on Linux containers:
Running Linux containers on Windows requires the use of LinuxKit or WSL. Docker Desktop for Windows requires you switch modes between Linux container mode and Windows container mode as you can't run both simultaneously. A workaround is to install an additional Docker daemon inside the WSL environment. Most people are going to install Docker Desktop and use WSL in Linux mode. It's fine. Hopefully my facts are up-to-date.
Note on Linux containers:
Running Linux containers on Windows requires the use of LinuxKit or WSL. Docker Desktop for Windows requires you switch modes between Linux container mode and Windows container mode as you can't run both simultaneously. A workaround is to install an additional Docker daemon inside the WSL environment. Most people are going to install Docker Desktop and use WSL in Linux mode. It's fine. Hopefully my facts are up-to-date.
Windows containers have improved quite a lot in Windows 11.
Exactly this.
Plus as the Apple virtualisation framework now implements pretty standard virtual hardware - it uses virtio - we are finding things work vastly better in Docker for Mac - particularly as we can use virtiofs etc to pass files through from the host.
Plus as the Apple virtualisation framework now implements pretty standard virtual hardware - it uses virtio - we are finding things work vastly better in Docker for Mac - particularly as we can use virtiofs etc to pass files through from the host.
We deploy containers to production, and use variants of the same containers for local development - mostly Debian.
Those containers contain a mix of custom built software and packages from the distro.
Those containers contain a mix of custom built software and packages from the distro.
asahi linux is only needed if you are running on the bare metal. A VM (which abstracts the hardware) means you can run existing ARM64 binaries.
I assume he just meant running binaries compiled for the same ISA / processor architecture.
I think ARM and AMD both are starting to cut deep into Intel's market share. I do think Ampere is a bit higher priced than I'd be comfortable with if I were setting up servers. The AMD 128-core Zen 4c server chip is very impressive in terms of per-core power use and on par on power usage with ARM even.
Incredible amount of computing capacity per u in the racks at this point on both fronts. Intel is betting on custom accelerators, but I don't know how well that fits on a rental (cloud) model unless you get a full server/cpu.
Incredible amount of computing capacity per u in the racks at this point on both fronts. Intel is betting on custom accelerators, but I don't know how well that fits on a rental (cloud) model unless you get a full server/cpu.
>My employer is moving workloads to ARM on AWS as it’s much better for price/perf
It's a fool's errand to try to reason about Amazon's pricing. We don't have figures about how much a Graviton costs to build or to run, but we do know AWS has staggeringly high margins across the board. They have huge headroom to make price cuts if they want their homegrown solution to win.
And perhaps it is better! I have no doubt it's a good CPU. But we can't infer that from pricing.
It's a fool's errand to try to reason about Amazon's pricing. We don't have figures about how much a Graviton costs to build or to run, but we do know AWS has staggeringly high margins across the board. They have huge headroom to make price cuts if they want their homegrown solution to win.
And perhaps it is better! I have no doubt it's a good CPU. But we can't infer that from pricing.
It could also be simultaneously true that it's cheaper to rent ARM servers from AWS than it is to rent Intel servers from them, and that AWS makes more money renting you the ARM servers. In fact, given they're developed in house, it might even be likely that the ARM margins are higher for them than their Intel margins.
Quite right - you can't infer performance from pricing.
We did some assessments of how well our applications ran on ARM and x86 based AWS instances, and quite simply it costs us less to serve the same quantities of end user traffic using Amazon's ARM instances compared to the x86 ones.
We did some assessments of how well our applications ran on ARM and x86 based AWS instances, and quite simply it costs us less to serve the same quantities of end user traffic using Amazon's ARM instances compared to the x86 ones.
Smaller dent then selling trampolin patches that take a already sold good and multiply the performance with a factor of 0.75.
Graviton server and serverless functions have shown such a better performance to cost ratio, it's unsurprising Amazon is doubling down on those.
The real question is, why are other cloud providers not working on similar chips?
The real question is, why are other cloud providers not working on similar chips?
I'm sure they are. But Amazon has a multi year lead.
Amazon took a gamble on server ARM chips. Not a bad or blind gamble, but they had no idea if their customers would take to them at all. I'm sure if they hadn't, we'd all be sitting here saying what an obviously dumb waste of money.
Amazon took a gamble on server ARM chips. Not a bad or blind gamble, but they had no idea if their customers would take to them at all. I'm sure if they hadn't, we'd all be sitting here saying what an obviously dumb waste of money.
AWS has a massive economic hand in ARM support. By offering cheaper compute on the ARM platform, they gave enough reason for everybody to make things work on ARM. I think they had a pretty good idea that customers would make it work for the price
> The real question is, why are other cloud providers not working on similar chips?
I suspect the answer is simple; they are nowhere near as flush with cash to buy up a CPU company and invest years to make their own chips. It's no small endeavor that AWS has gone through to get here.
I suspect the answer is simple; they are nowhere near as flush with cash to buy up a CPU company and invest years to make their own chips. It's no small endeavor that AWS has gone through to get here.
Microsoft and Google are far more flush with cash than Amazon is. Out of the smaller cloud providers, IBM and Oracle wouldn't have needed to buy a CPU company, they'd both been designing top tier CPUs for decades.
Here's an alternative equally simple explanation: ARM doesn't actually have anywhere near the level of advantage in perf/$ that Amazon's pricing would imply. The price differential on AWS isn't coming from any kind of intrinsic advantage, but from Amazon subsidizing their ARM-based compute for strategic reasons.
Here's an alternative equally simple explanation: ARM doesn't actually have anywhere near the level of advantage in perf/$ that Amazon's pricing would imply. The price differential on AWS isn't coming from any kind of intrinsic advantage, but from Amazon subsidizing their ARM-based compute for strategic reasons.
If ARM wasn't more efficient for Amazon then they wouldn't be aggressively moving their entire internal AWS infrastructure to gravitons. I've heard from folks in the know there that every single new service big or small internally has to work on graviton and by default is expected to launch on it unless the product goes through hoops to justify x86 and its expense.
Even in 2021, most of Amazon Retail’s workloads were “serverless” and the entire platform was actively flipping to Graviton one cluster at a time.
How do you know? Is there a link I could read about this? I've heard Amazon.com doesn't even use AWS — was this ever acurate?
I worked there for 10 years.
Sure, when AWS was created Amazon.com didn’t use it.
However today, Amazon.com is built entirely on AWS. It has been using EC2 and S3 since at least around 2010 (by 2013 “legacy hardware” was almost non-existent). Until 2020, it was all “hidden” behind Amazon’s internal tooling. Around 2019 there started to be a big push to just use AWS directly ie teams get AWS accounts owned by the Amazon.com org. There is still internal tooling to hook the code repo, build systems, and “pipelines” to directly deploy to AWS resources.
Meanwhile, the “serverless” I was talking about in the previous post was an internal service that hosts a Lisp dialect called Datapath. As a developer, you write some Datapath and give them the money and it executes as much as you need it to. This service is the core of Amazon.com and was actively being migrated to Graviton.
Sure, when AWS was created Amazon.com didn’t use it.
However today, Amazon.com is built entirely on AWS. It has been using EC2 and S3 since at least around 2010 (by 2013 “legacy hardware” was almost non-existent). Until 2020, it was all “hidden” behind Amazon’s internal tooling. Around 2019 there started to be a big push to just use AWS directly ie teams get AWS accounts owned by the Amazon.com org. There is still internal tooling to hook the code repo, build systems, and “pipelines” to directly deploy to AWS resources.
Meanwhile, the “serverless” I was talking about in the previous post was an internal service that hosts a Lisp dialect called Datapath. As a developer, you write some Datapath and give them the money and it executes as much as you need it to. This service is the core of Amazon.com and was actively being migrated to Graviton.
>I've heard Amazon.com doesn't even use AWS — was this ever acurate?
What? AWS spun out of Amazon.com having extra servers/infa after the holiday season rush and pushed for an internal way to monetize their extra compute/storage power.
https://en.wikipedia.org/wiki/Amazon_Web_Services
https://aws.amazon.com/solutions/case-studies/amazon/
What? AWS spun out of Amazon.com having extra servers/infa after the holiday season rush and pushed for an internal way to monetize their extra compute/storage power.
https://en.wikipedia.org/wiki/Amazon_Web_Services
https://aws.amazon.com/solutions/case-studies/amazon/
I've read that they did use it for some time then stopped. People then took AWS to be "Amazon Scale" despite not being true/misleading.[0] If I'm reading it right, even your own link only says they are using s3 for backing up their database.
See also: https://news.ycombinator.com/item?id=4181014
[0] https://news.ycombinator.com/item?id=33024415
See also: https://news.ycombinator.com/item?id=4181014
[0] https://news.ycombinator.com/item?id=33024415
It's a creation myth/legend IIRC
But is ARM necessarily that more efficient or is it just cheaper than x86 because Amazon designed it inhouse and doesn't need to pay AMD/Intel (who have high margins)?
Web workloads aren't super CPU intensive on their own. You take some bytes from the network, parse them, make some calls to services on an internal network, then serialize and send network bytes. Most of that workload is waiting for the network bytes to come and go.
For that kind of work it's all about power efficiency and Amazon is able to specifically design their gravitons to have just enough cache, CPU speed, management components, etc. to get the job done and nothing more. A bog standard Intel CPU is designed for any and every workload so it's going to be idling and wasting power on components and transistors you never use or care about.
For that kind of work it's all about power efficiency and Amazon is able to specifically design their gravitons to have just enough cache, CPU speed, management components, etc. to get the job done and nothing more. A bog standard Intel CPU is designed for any and every workload so it's going to be idling and wasting power on components and transistors you never use or care about.
[deleted]
There's a reason it took over the smartphone industry decade before AWS started using them.
Arm on the server isn't a radical idea and been proposed for a long time. Smartphones and servers share a lot of similar requirements like especially performance per watt
Arm on the server isn't a radical idea and been proposed for a long time. Smartphones and servers share a lot of similar requirements like especially performance per watt
> There's a reason it took over the smartphone industry
Intel wasn't interested in "low-margin" markets and was generally incompetent?
Intel wasn't interested in "low-margin" markets and was generally incompetent?
Well, yeah, no way I'm debating that.
Weird thing is they had StrongArm in the early 2000s.
Weird thing is they had StrongArm in the early 2000s.
Ampere ARM chips still have a similar price advantage (check out Hetzner cloud pricing, for example).
> IBM and Oracle wouldn't have needed to buy a CPU company, they'd both been designing top tier CPUs for decades
you HAVE TO HAVE those CPUs on devs laptops. Otherwise it just won't work. Macs with M1/M2 are pretty much ideal setup to develop for gravitons
you HAVE TO HAVE those CPUs on devs laptops. Otherwise it just won't work. Macs with M1/M2 are pretty much ideal setup to develop for gravitons
> Out of the smaller cloud providers, IBM and Oracle wouldn't have needed to buy a CPU company, they'd both been designing top tier CPUs for decades.
CPUs that need an insane amount of power and have barely any applications in mainstream computing. Not exactly a good fit to provide to customers.
> The price differential on AWS isn't coming from any kind of intrinsic advantage, but from Amazon subsidizing their ARM-based compute for strategic reasons.
By running their own CPUs, AWS doesn't have to rely on Intel's ability and willingness to ship stuff and they don't have to pay Intel's margin either.
CPUs that need an insane amount of power and have barely any applications in mainstream computing. Not exactly a good fit to provide to customers.
> The price differential on AWS isn't coming from any kind of intrinsic advantage, but from Amazon subsidizing their ARM-based compute for strategic reasons.
By running their own CPUs, AWS doesn't have to rely on Intel's ability and willingness to ship stuff and they don't have to pay Intel's margin either.
> CPUs that need an insane amount of power and have barely any applications in mainstream computing. Not exactly a good fit to provide to customers.
Obviously there's no real market for Sparc or POWER in the public cloud, and there probably never was a time window for creating a viable market either.
But if they'd wanted to get into the server ARM market, they would have had the expertise in-house, there was no need to buy out some startup for the team. That their expertise was on CPUs with different microarchitectures, but that shouldn't matter all that much (didn't matter when Apple bought PA Semi to work on ARM instead of PowerPC).
> By running their own CPUs, AWS doesn't have to rely on Intel's ability and willingness to ship stuff and they don't have to pay Intel's margin either.
Right, being in control of your own destiny rather than being tied to a single supplier is a big deal, and that's one example of what I meant by the strategic reasons. Having more leverage over Intel and AMD on pricing is another. Making it harder for some customers to move their workloads away (because they're bought into ARM and other companies don't have compelling ARM offerings) is a potential third.
Not having to pay Intel's margins is a potential advantage, but requires reaching sufficient scale to offset the costs of running your own CPU team.
But for any of this to make sense, they need to successfully move a large proportion of their customers over to ARM. Just having the servers available and nobody using them does nothing. That's exactly the kind of situation where you'd want to subsidize the prices for the ARM servers and build up the customer base. That's the case even if ARM has some intrinsic perf/$ advantage; you'd still want to subsidize it to the point where you're able to saturate your ability to manufacture and install ARM servers.
Obviously there's no real market for Sparc or POWER in the public cloud, and there probably never was a time window for creating a viable market either.
But if they'd wanted to get into the server ARM market, they would have had the expertise in-house, there was no need to buy out some startup for the team. That their expertise was on CPUs with different microarchitectures, but that shouldn't matter all that much (didn't matter when Apple bought PA Semi to work on ARM instead of PowerPC).
> By running their own CPUs, AWS doesn't have to rely on Intel's ability and willingness to ship stuff and they don't have to pay Intel's margin either.
Right, being in control of your own destiny rather than being tied to a single supplier is a big deal, and that's one example of what I meant by the strategic reasons. Having more leverage over Intel and AMD on pricing is another. Making it harder for some customers to move their workloads away (because they're bought into ARM and other companies don't have compelling ARM offerings) is a potential third.
Not having to pay Intel's margins is a potential advantage, but requires reaching sufficient scale to offset the costs of running your own CPU team.
But for any of this to make sense, they need to successfully move a large proportion of their customers over to ARM. Just having the servers available and nobody using them does nothing. That's exactly the kind of situation where you'd want to subsidize the prices for the ARM servers and build up the customer base. That's the case even if ARM has some intrinsic perf/$ advantage; you'd still want to subsidize it to the point where you're able to saturate your ability to manufacture and install ARM servers.
> but that shouldn't matter all that much (didn't matter when Apple bought PA Semi to work on ARM instead of PowerPC).
Apple is designing their own cores while Amazon is just using ARM Neoverse.
And Oracle is offering Ampere instance which IIRC are quite close to Gravirton cost/performance wise.
Apple is designing their own cores while Amazon is just using ARM Neoverse.
And Oracle is offering Ampere instance which IIRC are quite close to Gravirton cost/performance wise.
Not just cash but also willing to make investments which don’t show returns within a year. Amazon has a history of very lean profits due to reinvestment and something like this might not be possible with Google’s promotion culture, especially adoption takes time so the initial numbers won’t be compelling.
So far they seem to be betting on Ampere which seems to be developing their own cores (which is much more complex/expensive than what Amazon is doing) so it's interesting how they will do. If Ampere manages to get a moat AWS might end up behind everyone else in a couple of years.
I got an ARM Windows 11 laptop (Lenovo X13s) because I wanted a lightweight travel laptop with long batter life. I was expecting a big compromise, but it's actually wonderful!
I'm a giant Intel fanboy, but I think they have a real challenge here.
I'm a giant Intel fanboy, but I think they have a real challenge here.
Intel definitely has a challenge – having gone through both of Apple’s PowerPC migrations, this was the smoothest yet because we’re in a much better place with open source software dominating and internet distribution being the norm. I remember things being held up by proprietary compilers or needing to port hand rolled assembly but this time around we’re using higher level languages & frameworks a lot more and it’s been mostly transparent. Similarly, back then getting hardware to build and test was a chore; now it’s a checkbox in the EC2 launch options.
Thinking about it, I knew that Intel not landing the first iPhone contract was a big loss on mobile but I don’t think anyone expected it to savage their traditional strongholds like this because that paved the way for so many tools not just being ported but heavily optimized for ARM. They’re certainly still doing a ton of business but now there’s a cap on how much profit they can take even if AMD completely founders.
Thinking about it, I knew that Intel not landing the first iPhone contract was a big loss on mobile but I don’t think anyone expected it to savage their traditional strongholds like this because that paved the way for so many tools not just being ported but heavily optimized for ARM. They’re certainly still doing a ton of business but now there’s a cap on how much profit they can take even if AMD completely founders.
The funny thing is that roughly a decade ago AMD literally was prototyping an ARM CPU presumably for server use. I recall hearing about
2014 - AMD Opteron A1100: https://www.google.com/url?q=https://www.anandtech.com/show/...
The startup Calxeda (2008-2013) - https://en.m.wikipedia.org/wiki/Calxeda
Intel also had the 'xscale' ARM chips back in the late 2000s
2014 - AMD Opteron A1100: https://www.google.com/url?q=https://www.anandtech.com/show/...
The startup Calxeda (2008-2013) - https://en.m.wikipedia.org/wiki/Calxeda
Intel also had the 'xscale' ARM chips back in the late 2000s
I don’t want to say it’s all the iPhone but I do think it’s not a coincidence that things are very different now that we have robust tools like Clang and over a decade of compatibility patches throughout the open source world.
In 2008 even Java developers tended to be antsy about running on non-x86 hardware. A decade later it’s close to a non-issue for a staggering amount of stuff.
In 2008 even Java developers tended to be antsy about running on non-x86 hardware. A decade later it’s close to a non-issue for a staggering amount of stuff.
I'm not a backend expert, but I'd also speculate abother big change was containerization becoming popular - Docker came out in 2013.
Definitely - both for making it easy to manage isolated services independently and for forcing everything to have a known build process.
x86 lock-in benefits AMD greatly, just as it does Intel. It is my belief that they started looking at ARM cores when they thought they would never be able to be competitive with Intel in x86 (pre-Zen was a bad time for AMD). And when it became clear that actually their new cores would do very well against Intel, they dropped ARM plans. They don't want to move off x86 if they don't have to.
Intel Xscale were a different beast. They got that through StrongARM from DEC as part of some law suit wheeling and dealing. They never seemed to really want to push it, preferring instead to try competing in mobile with a string of failed x86 cores.
Intel Xscale were a different beast. They got that through StrongARM from DEC as part of some law suit wheeling and dealing. They never seemed to really want to push it, preferring instead to try competing in mobile with a string of failed x86 cores.
> I knew that Intel not landing the first iPhone contract was a big loss on mobile
Funnily enough Intel also had the best/most competitive ARM cores with XScale in the early 2000s which seemed like the default option for most high-end devices at the time. I wouldn't have been surprised if Apple would've just went with that (even if not x86) if Intel hadn't decided to can it without actually having any viable x86 SoC to replace it...
Funnily enough Intel also had the best/most competitive ARM cores with XScale in the early 2000s which seemed like the default option for most high-end devices at the time. I wouldn't have been surprised if Apple would've just went with that (even if not x86) if Intel hadn't decided to can it without actually having any viable x86 SoC to replace it...
Why would Amazon develop a custom Arm chip and not RISC-V?
Don't they have to pay a fee to Arm for this, even though it is being developed by them?
And would Google (which is allegedly developing their own Arm Chip) and Apple (which also has their own chip) not also benefit from backing Risc-V as well?
I have not been able to get clear in my head the relationship between Risc-V and Arm, and even more so exactly what responsibility down stream each user has.
Would appreciate the some ELI5 from knowledgeable HNers.
---
Slightly related: is it more or less a given that eventually no one will use x64 chips (since Arm/Risc-V is more efficient)?
If not for servers, is it at least a given for desktop computers, as more software is written for ARM, and re-compilers like Rosseta improve?
If so, I know that Intel is investing in RISC-V, but has AMD given up the space entirely?
Don't they have to pay a fee to Arm for this, even though it is being developed by them?
And would Google (which is allegedly developing their own Arm Chip) and Apple (which also has their own chip) not also benefit from backing Risc-V as well?
I have not been able to get clear in my head the relationship between Risc-V and Arm, and even more so exactly what responsibility down stream each user has.
Would appreciate the some ELI5 from knowledgeable HNers.
---
Slightly related: is it more or less a given that eventually no one will use x64 chips (since Arm/Risc-V is more efficient)?
If not for servers, is it at least a given for desktop computers, as more software is written for ARM, and re-compilers like Rosseta improve?
If so, I know that Intel is investing in RISC-V, but has AMD given up the space entirely?
Because by paying a fee to Arm, they get decades of architecture experience, support, peripherals (including a well-thought-out internal BUS), etc.
RISC-V has a good ISA, but there currently isn't a lot of silicon experience in the real world (relative to Arm and Intel, for example). And Amazon isn't going to roll their own to that degree. It's not worth the meager licensing costs.
It's like the difference between deploying Red Hat Enterprise Linux and Arch Linux. Each has it's place, but right now its advantages are in much different areas.
RISC-V has a good ISA, but there currently isn't a lot of silicon experience in the real world (relative to Arm and Intel, for example). And Amazon isn't going to roll their own to that degree. It's not worth the meager licensing costs.
It's like the difference between deploying Red Hat Enterprise Linux and Arch Linux. Each has it's place, but right now its advantages are in much different areas.
The most common use case is where ARM provide fully functional CPU core IP that you can put into your own SoC - in AWS Graviton's case that seems to be the ARM Neoverse line of cores.
You can also get architecture licenses where you can implement your own CPU core (from scratch, or by making tweaks to an ARM design) - Apple has one of these, but Apple is a special case in that they were one of the original partners when ARM was originally founded in 1990.
RISC-V on the other hand is an architecture that is open for anyone to implement, but the RISC-V foundation doesn't sell you a core design - you need to create your own, or find a partner to sell you one. It's also much newer, so there's less of an ecosystem around it compared to ARM.
You can also get architecture licenses where you can implement your own CPU core (from scratch, or by making tweaks to an ARM design) - Apple has one of these, but Apple is a special case in that they were one of the original partners when ARM was originally founded in 1990.
RISC-V on the other hand is an architecture that is open for anyone to implement, but the RISC-V foundation doesn't sell you a core design - you need to create your own, or find a partner to sell you one. It's also much newer, so there's less of an ecosystem around it compared to ARM.
> Don't they have to pay a fee to Arm for this
Because it's way cheaper than to do everything ARM provides on your own?
Gravitron3 is just using ARM Neoverse-V1 (which AFAIK are years ahead of any RISC-V cores). So they aren't different from Qualcomm/Ampere have been doing so far.
I think Ampere is planning to release a CPU with their own custom cores in the near future though. It's interesting whether Amazon will follow suite or not. It would be very bizarre if they tried switching to RISC-V in the near future though (I do find this semi -fanatical obsession with RISC-V a bit fascinating though but I don't think it's shared by many large corporations outside China..)
Because it's way cheaper than to do everything ARM provides on your own?
Gravitron3 is just using ARM Neoverse-V1 (which AFAIK are years ahead of any RISC-V cores). So they aren't different from Qualcomm/Ampere have been doing so far.
I think Ampere is planning to release a CPU with their own custom cores in the near future though. It's interesting whether Amazon will follow suite or not. It would be very bizarre if they tried switching to RISC-V in the near future though (I do find this semi -fanatical obsession with RISC-V a bit fascinating though but I don't think it's shared by many large corporations outside China..)
> Why would Amazon develop a custom Arm chip and not RISC-V?
The first Graviton was launched in 2018, which means it was in development and manufacturing even earlier. You couldn't build a high performance RISC-V chip back then. Even in 2023, RISC-V is significantly behind ARM/x86 in performance and software support isnt great either (for example, Debian has supported Arm for over a decade, but RISC-V support only arrived a few weeks ago).
The first Graviton was launched in 2018, which means it was in development and manufacturing even earlier. You couldn't build a high performance RISC-V chip back then. Even in 2023, RISC-V is significantly behind ARM/x86 in performance and software support isnt great either (for example, Debian has supported Arm for over a decade, but RISC-V support only arrived a few weeks ago).
[deleted]
I don't doubt it, as a private individual buying Ampere Altra CPU's has been pretty painful -- at least in Europe.
I could only buy 4 in the end.
Sidenote: has anyone noticed that there's no price difference for GCP to use ARM?
I could only buy 4 in the end.
Sidenote: has anyone noticed that there's no price difference for GCP to use ARM?
The new HPC Graviton3E instances (https://aws.amazon.com/ec2/instance-types/hpc7g/) are pretty great for CFD and simulation workloads.
Article says Amazon accounts for >50% while China accounts for 40%, so Amazon actually has >80% outside China. I wonder why Microsoft and Google aren’t as eager to stop paying the x86 tax.
Google is doing their own server ARM chips: https://www.theregister.com/2023/02/14/google_prepares_its_o...
They are but it takes 2 to 3 years to develop a chip like this. Someone already posted a link to the Google chip. Microsoft got a lot of CPU designers from Qualcomm's team after Qualcomm got rid of them a few years ago. Now Microsoft has laid off their custom ARM core team so anything they do will be a licensed ARM core.
Amazon serves traffic to China, they have an AWS region there I think.
Three. Two operated by other companies (in Beijing and Ningxia) that are only available in China, and Hong Kong which appears to be just another AWS region available globally.
Very little traffic for AWS China regions
Microsoft has been abysmally slow to move towards ARM. They hitched their wagon to Qualcomm and snapdragon... which does not seem like a smart decision now (especially as Qualcomm seems to be moving to RISC-V lol). Windows on ARM for servers has been a bit of a pipe dream and joke--I think they finally have it available but I stopped looking or caring about it years ago. Amazon is the defacto ARM in the cloud/server leader right now and I don't see that changing anytime soon.
>which does not seem like a smart decision now (especially as Qualcomm seems to be moving to RISC-V lol).
I politely disagree. We know for a fact, from December's RISC-V Summit, that Microsoft is influencing RISC-V to ease the burden on their own Windows for RISC-V effort.
Qualcomm was a partner for ARM, and they are already used to working with them. They will likely be partners again, with RISC-V devices this time around.
And Windows for ARM will go the same way as Windows for Alpha. Just an historical footnote.
RISC-V is inevitable.
I politely disagree. We know for a fact, from December's RISC-V Summit, that Microsoft is influencing RISC-V to ease the burden on their own Windows for RISC-V effort.
Qualcomm was a partner for ARM, and they are already used to working with them. They will likely be partners again, with RISC-V devices this time around.
And Windows for ARM will go the same way as Windows for Alpha. Just an historical footnote.
RISC-V is inevitable.
> RISC-V is inevitable.
Why? Some people here seem to be weirdly obsessed with RISC-V when it's likely to end up being even more closed/proprietary than ARM (at least on the high-end). Why would you give away your competitive advantage to everyone else by licensing your core designs?
ARM at least provides more or less an even playing field to everyone. It's much cheaper for competitors to catch-up with Gravitron since they can just license the exactly same core it's built on. If it was a RISC-V CPU and as much ahead it would be way easier for AWS to maintain it's moat.
Why? Some people here seem to be weirdly obsessed with RISC-V when it's likely to end up being even more closed/proprietary than ARM (at least on the high-end). Why would you give away your competitive advantage to everyone else by licensing your core designs?
ARM at least provides more or less an even playing field to everyone. It's much cheaper for competitors to catch-up with Gravitron since they can just license the exactly same core it's built on. If it was a RISC-V CPU and as much ahead it would be way easier for AWS to maintain it's moat.
> Windows on ARM for servers has been a bit of a pipe dream and joke
Arguably Windows for servers is a joke in itself. What can it do, besides Active Directory and Exchange, that Linux can't?
(I know LDAP exists, but let's not pretend that AD isn't superior in almost every way imaginable)
Arguably Windows for servers is a joke in itself. What can it do, besides Active Directory and Exchange, that Linux can't?
(I know LDAP exists, but let's not pretend that AD isn't superior in almost every way imaginable)
Lots of stuff in the VMS line of programming model.
For years GNU/Linux lacked server capabilities at the level of IO Completion ports, grated, it finally catched up.
The security model, that on GNU/Linux would require to enable SELinux, seccomp and several other knobs.
A proper ABI for drivers, with an userspace driver model for most scenarios, increasing server security.
Sharing files that aren't stuck in the NFS model.
Capabilities based security, ability to allow only execution of signed applications and group policies for executions.
Ah, and Azure runs on Windows, not Linux.
https://techcommunity.microsoft.com/t5/windows-os-platform-b...
For years GNU/Linux lacked server capabilities at the level of IO Completion ports, grated, it finally catched up.
The security model, that on GNU/Linux would require to enable SELinux, seccomp and several other knobs.
A proper ABI for drivers, with an userspace driver model for most scenarios, increasing server security.
Sharing files that aren't stuck in the NFS model.
Capabilities based security, ability to allow only execution of signed applications and group policies for executions.
Ah, and Azure runs on Windows, not Linux.
https://techcommunity.microsoft.com/t5/windows-os-platform-b...
Run software with byzantine licensing/security models tethered into Windows security features.
Not that it's a good thing, but it's been my experience with some enterprise software libraries in practice.
Not that it's a good thing, but it's been my experience with some enterprise software libraries in practice.
Out of curiosity is the timing of this Bernstein Research report and the upcoming Arm IPO a coincidence?
Where are people (that aren't Amazon) buying ARM servers to put in their datacenters? They seem hard to find or actually order.
The big hyperscale datacenter companies either design it themselves or partner with another large semiconductor company to help them design it for their particular needs.
These companies are not interested in selling to small companies that just want 1 to 100 machines. The only company that might is Ampere.
https://www.asacomputers.com/arm-servers.html
Oracle is using the ARM server chip from Ampere but they would be buying tens of thousands or more and getting volume discount pricing. I've heard rumors that Ampere customized their chip for Oracle's needs.
https://www.oracle.com/cloud/compute/arm/
Qualcomm was designing an ARM server chip about 8 years ago but killed the project. Most of those engineers went to Microsoft and now ARM itself.
I've heard stories about Chinese hyperscale datacenters wanting to design their own ARM servers but I haven't heard anything in a while.
These companies are not interested in selling to small companies that just want 1 to 100 machines. The only company that might is Ampere.
https://www.asacomputers.com/arm-servers.html
Oracle is using the ARM server chip from Ampere but they would be buying tens of thousands or more and getting volume discount pricing. I've heard rumors that Ampere customized their chip for Oracle's needs.
https://www.oracle.com/cloud/compute/arm/
Qualcomm was designing an ARM server chip about 8 years ago but killed the project. Most of those engineers went to Microsoft and now ARM itself.
I've heard stories about Chinese hyperscale datacenters wanting to design their own ARM servers but I haven't heard anything in a while.
I tried Arm on EC2 but random reboots and hangs sent me back to Intel.
I moved RDS and Elasticache to ARM with no issues for over a year ago
For EC2 only used it for a single instance where I could redeploy the app easier by picking an ARM image
For EC2 only used it for a single instance where I could redeploy the app easier by picking an ARM image
Interesting. I never even considered the instance stability to be a risk when using ARM CPUs on Amazon. Anybody else can share their experience?
On a mix of Graviton2 and 3 based instances, we’ve found no significant difference between ARM, Intel, and AMD instances for weird problems/instance replacement, etc.
Can't corroborate. We have not had any issues with Graviton (mix of r7g and t4g today, previously we used r6g and that was fine too).
[deleted]
My ARM server on EC2 work better than the intel one.
They are low—end, low—load servers, if that matter.
They are low—end, low—load servers, if that matter.
I tried Ampere Altra on Hetzner, it was unparallel price/performance wise. But it kept falling into low frequency state (1 GHz on all 80 cores). That's either something in the motherboard or in the processor, couldn't figure it out. Guess not mature enough yet.
That's gonna be fun when Amazon switches to RISC-V for their next (or next+n) iteration of Graviton.
ARM presence in the server market will go away like poof. More than half of it.
ARM presence in the server market will go away like poof. More than half of it.
What indication exists that Amazon is considering RISC?
It’s mere existence. If Amazon isn’t considering it I’d be surprised.
Wouldn't designing your own cores be on a whole other level compared to what Amazon is doing now (licensing Neoverse)? If they don't feel confident they could build a competitive ARM core on their own (unlike Ampere which seems to be trying that) why would it be different with RISC-V which is still years behind ARM (on the high-end)?
If not designing their own cores, RISC-V has far more options than ARM to license from.
From a business continuity perspective, Amazon doesn't benefit from being locked to a single vendor for its CPU core IP.
From a business continuity perspective, Amazon doesn't benefit from being locked to a single vendor for its CPU core IP.
> RISC-V has far more options
Like what? Are there any high-end RISC-V cores that proven to be competitive with Neoverse you could license?
It all just seem hypothetical to me at this point and I really struggle trying to understand who and why would design cores to license them to others? Looking at ARM it just doesen't seem like a great business model compared to making them yourself..
Like what? Are there any high-end RISC-V cores that proven to be competitive with Neoverse you could license?
It all just seem hypothetical to me at this point and I really struggle trying to understand who and why would design cores to license them to others? Looking at ARM it just doesen't seem like a great business model compared to making them yourself..
>Are there any high-end RISC-V cores that proven to be competitive with Neoverse you could license?
Look into Tenstorrent Ascalon, which is supposedly competitive with projected Zen5 performance.
Ascalon has, by the way, already been licensed to LG[0].
RISC-V is inevitable.
0. https://www.eetasia.com/jim-keller-on-ai-risc-v-tenstorrents...
Look into Tenstorrent Ascalon, which is supposedly competitive with projected Zen5 performance.
Ascalon has, by the way, already been licensed to LG[0].
RISC-V is inevitable.
0. https://www.eetasia.com/jim-keller-on-ai-risc-v-tenstorrents...
Sounds good. Are there any benchmarks? Can you but it? How much does it cost?
> RISC-V is inevitable.
I'm sorry but you sound a bit like the "Crypto bros" from a few years ago.
> RISC-V is inevitable.
I'm sorry but you sound a bit like the "Crypto bros" from a few years ago.
None, currently.
> That's gonna be fun when Amazon switches to RISC-V for their next (or next+n) iteration of Graviton.
> ARM presence in the server market will go away like poof. More than half of it.
This is a borderline fetishistic delusion to think that this could happen immediately in the first place. The changeover from x86/64 to ARM for AWS customers didn't even go this fast, and that's WITH the help & stability provided by the extensive tooling & ecosystem around ARM.
RISC-V is nowhere near to being able to provide the same sort of tooling for years to come, and this comes from someone that loves the architecture's openness.
> ARM presence in the server market will go away like poof. More than half of it.
This is a borderline fetishistic delusion to think that this could happen immediately in the first place. The changeover from x86/64 to ARM for AWS customers didn't even go this fast, and that's WITH the help & stability provided by the extensive tooling & ecosystem around ARM.
RISC-V is nowhere near to being able to provide the same sort of tooling for years to come, and this comes from someone that loves the architecture's openness.
Well it's certainly going to be fun for their competitors when Amazon ends up being 5-10 years behind AMD/Intel/Ampere when they do that.
Is there any scientific comparison between AWS amd64 and arm instances for performance and cost? I tried running some stuff on an arm instance but it seemed to similar in both price and performance to amd64. Just with added complexity to make it work nicely with Github actions built container images.
If Amazon's heralding something more general, I'm curious what a move to ARM will do to server side async and sync/async trade-off with waiting tasks/io. Less potential point if 20 threads/cores can be had for the price of 1?
Or, what they've changed to take advantage of the difference.
Or, what they've changed to take advantage of the difference.
The Register does not report any absolute numbers and the source appears to be a proprietary market research report. Is the actual number of ARM or x86 CPUs at Amazon publically available?
What would a CPU instruction set look like to minimise energy costs for server workloads? And do spinning hard drives use more energy than SSDs?
What exactly happens to old hardware? Is it easily recyclable ?
As far as I understand it, electronics are recycled by crushing them and extracting a handful of minerals - e.g. gold, there's more gold in circuit boards than the rocks they dig out of gold mines I've heard.
The rest - which by mass, of course, is something approaching "all of it" - is mostly toxic fibre glass, epoxy and plastic that we don't really know what to do with. It's bad, even before you start talking about all the electricity and heat that was used to turn it into electronics in the first place.
The rest - which by mass, of course, is something approaching "all of it" - is mostly toxic fibre glass, epoxy and plastic that we don't really know what to do with. It's bad, even before you start talking about all the electricity and heat that was used to turn it into electronics in the first place.
My significant other worked in a recycling plant. To elaborate on the "rest" and how it is done:
PCBs get delivered, sometimes without big components (coolers, screws/brackets fans removed), but also sometimes with those. They are shredded as a whole, including all soldered components, to a particle size of 2mm. Iron and aluminium are separated magnetically (in case of aluminium with an eddy current separator, iron with a static field). All three components, mostly-iron, mostly-aluminium and the rest are then melted down separately. Adherent plastics, electrolytes, epoxy, etc burn off during this process. Gas and ash are lead through separators that filter out the fly-ash as much as possible, which is buried somewhere as toxic waste, the rest is entered into the atmosphere after cleaning (afaik various liquids to bubble through to wash out soluble chemicals, which are then dried and buried). All three metalllic liquids are then metall alloys of various purities which are then cleaned and separated further by skimming off the slag, adding flux, electrolysis and other chemical separation methods. Most of the "other" fraction is copper, which is the main product overall, other rare metals such as gold occur in far smaller amounts and seemingly add only a little to the overall earnings in the process.
Please note that this is the process used in a relatively new plant in a Western European nation. I guess the "advantage" over the rest of the world is that at least there is a fly-ash separator and exhaust cleaning.
PCBs get delivered, sometimes without big components (coolers, screws/brackets fans removed), but also sometimes with those. They are shredded as a whole, including all soldered components, to a particle size of 2mm. Iron and aluminium are separated magnetically (in case of aluminium with an eddy current separator, iron with a static field). All three components, mostly-iron, mostly-aluminium and the rest are then melted down separately. Adherent plastics, electrolytes, epoxy, etc burn off during this process. Gas and ash are lead through separators that filter out the fly-ash as much as possible, which is buried somewhere as toxic waste, the rest is entered into the atmosphere after cleaning (afaik various liquids to bubble through to wash out soluble chemicals, which are then dried and buried). All three metalllic liquids are then metall alloys of various purities which are then cleaned and separated further by skimming off the slag, adding flux, electrolysis and other chemical separation methods. Most of the "other" fraction is copper, which is the main product overall, other rare metals such as gold occur in far smaller amounts and seemingly add only a little to the overall earnings in the process.
Please note that this is the process used in a relatively new plant in a Western European nation. I guess the "advantage" over the rest of the world is that at least there is a fly-ash separator and exhaust cleaning.
Thanks for this. Really interesting.
>toxic fibre glass, epoxy and plastic
I’m guessing there are no environmentally friendly alternatives to these materials than can be easily swapped in factories without much hassle.
I’m guessing there are no environmentally friendly alternatives to these materials than can be easily swapped in factories without much hassle.
Actually, there are companies working on PCBs based on organic, water-soluble substrates [1] - to recycle a board, you boil it in some hot water and get a liquid stream containing the substrate and a solid stream containing the electronic components and copper traces.
[1] https://interestingengineering.com/innovation/new-biodegrada...
[1] https://interestingengineering.com/innovation/new-biodegrada...
Which is a researach-stage idea with obvious drawbacks: Moisture in air will also slowly decompose PCBs, making them age much faster than traditional epoxy-based PCBs. So then you either limit the maximum age of all your products severely (not very green) or you need additional materials to moisture-proof all your casings (also not very green).
Do you really think that they haven’t spent at least as much time thinking about this? The question would be whether those devices are replaced in less than the time it takes for natural degradation to cause problems, and both of your “not very green” dismissals sound entirely too pat. The entire reason we’re talking about this is that the industry has a huge problem with things having a service life measured in single digit years and a landfill life measured in centuries – making those easier to recycle would be a huge win because there’s no way that the current model is going to get the service life significantly closer to that kind of timeframe.
From the article that was linked:
> The research will also provide Infineon with a fundamental understanding of the design and reliability challenges customers face with the new material in their core applications.
That is corporate-speak for: they have no idea.
> The research will also provide Infineon with a fundamental understanding of the design and reliability challenges customers face with the new material in their core applications.
That is corporate-speak for: they have no idea.
It sounds more like they know that this is a hard problem requiring foundational research, and suggests that rather more thought has gone into it than a quick HN comment dismissing their work.
If I understand it correctly, their idea is to use a binding polymer that needs heat to dissolve.
Amazon has a reuse program for old hardware[1]. Disks will be shredded on-site and then recycled. [1] https://www.aboutamazon.com/news/aws/how-aws-data-centers-re...
While there are probably more modern servers around now than ever before, from browsing serverhunter.com there seem to be a couple of cliffs where older CPUs gets rarer, one at 2017 and one at 2014.
If by recyclable you meant buying old racks of datacenter servers to reuse elsewhere, even if possible, it's not very practical. Servers are super noisy and may have exotic power requirements.
r/homelab would like a word.
Time to change the name to Armazon.
I wonder if Google, Microsoft, Facebook aren't considering using ARM in their datacenters.
Both Google and Microsoft have ARM in their VM offer.
Facebook have been flip-flop’ing on their ARM server: https://www.extremetech.com/extreme/146850-facebook-arm-x86-...
Facebook have been flip-flop’ing on their ARM server: https://www.extremetech.com/extreme/146850-facebook-arm-x86-...
That’s a 10 year old article
What could go wrong?
https://www.investopedia.com/terms/1/51-attack.asp
https://www.investopedia.com/terms/1/51-attack.asp
The X86 vs ARM makes for such a great story.
One prefers locking and complicated instructions, the other is designed with open and simplicity.
At one point, Intel seemed to be dominant, and proven right with their proprietary tech, where as ARM chugged along keeping a low profile, and keep doing the good work. Now, finally the reality has caught up with them, and turns out openness and simplicity is finally taking a lead.
Stick to your principles FTW!!!
One prefers locking and complicated instructions, the other is designed with open and simplicity.
At one point, Intel seemed to be dominant, and proven right with their proprietary tech, where as ARM chugged along keeping a low profile, and keep doing the good work. Now, finally the reality has caught up with them, and turns out openness and simplicity is finally taking a lead.
Stick to your principles FTW!!!
How is ARM open? And simplicity is also subjective… x86 goes way back thats why it seems more complicated or “crufty”.
ARM is a huge corp like anyone else.
ARM got a huge boost now since everything is switching to mobile and their chips are more energy efficient than Intels. Added to that Intel was also asleep at the wheel and missed the mobile boat…
ARM is a huge corp like anyone else.
ARM got a huge boost now since everything is switching to mobile and their chips are more energy efficient than Intels. Added to that Intel was also asleep at the wheel and missed the mobile boat…
ARM chips are more energy efficient because they're simpler. Less gates/instructions = less power usage.
AMD's 128-core Zen4c chips are on par with ARM in per-core power usage, while offering higher performance per core.
Pretty sure ARM is proprietary.
I don't think it can get more proprietary than running your code on a machine that isn't accessible to the general public.
>running your code on a machine that isn't accessible to the general public.
What machine is this?
What machine is this?
>One prefers locking and complicated instructions, the other is designed with open and simplicity.
It seems you're confusing ARM and RISC-V here.
ARM isn't simple, nor open.
It seems you're confusing ARM and RISC-V here.
ARM isn't simple, nor open.
None of the other cloud providers are using ARM at this scale yet. They’re using Ampere chips, whereas Amazon invested in building their own - it tells you the level of commitment Amazon have here.
My employer is moving workloads to ARM on AWS as it’s much better for price/perf, and our shiny new M1/M2 Macs run the same binaries. We’ve had very little issue doing the shift - Amazon Linux and Debian both support ARM well.
Must be really putting a dent in Intel’s data centre sales though.