Ask HN: What features does a chip require to beat Nvidia in the AI race?
Nvidia has a lot of strengths that puts them over other competitors. I'm curious to hear from some technical experts in data center, AI labs, and enterprise. I think there's a lot of nuance here beyond just "faster and more energy efficient".
14 comments
Nvidia is reliably flexible, so, by the time you are investing hundreds of thousands of compute hours for training a new model, your lowest risk effort is to do this with Nvidia chips. Do you want to risk the slowdowns and hiccups that could be caused by being one of the first to train huge models on alternative software/hardware? If you aren't using Nvidia for the cutting edge research you are actually using your research for two things. 1) Normal cost of training 2) proof of concept for training huge new models on new to market hardware.
Nvidia doesn't really have a definite lead for 'faster and more energy efficient', there are competitors that look like good alternatives.
Nvidia doesn't really have a definite lead for 'faster and more energy efficient', there are competitors that look like good alternatives.
I think this is valid for training but Nvidia also dominates inference, which is a bit of a different argument
The whole cost of performing a given LLM computation has to be at least an order of magnitude less than the state of the art if you want to make a dent in the market.
A large part of that cost is transferring weights and values to the compute fabric. I've long thought that compute in memory could solve this, but the die areas go through the roof for the same amount of compute, even though the power is far less.
I think smaller process nodes are the only way forward.
A large part of that cost is transferring weights and values to the compute fabric. I've long thought that compute in memory could solve this, but the die areas go through the roof for the same amount of compute, even though the power is far less.
I think smaller process nodes are the only way forward.
A jacket cooler than black leather. So, probably denim.
Realistically, pretty much anything.
Realistically, pretty much anything.
To "beat" them, just don't play the game. The fad will die.
Why do you think AI is going to die?
You're asking the wrong question.
Most of Nvidia's "strength" is the heavy lifting done by the folks over at TSMC.
Apple and AMD both have access to the same TSMC technology, but neither of them are willing to invest any more into AI than they are currently. Apple ships an inference solution that is a combination of GPU and NPU and mostly okay for inference at home, and AMD ships a high performance enterprise compute solution that scales the whole way up to the needs of supercomputers without needing to make a purely AI-only product line.
If this was the next big money making frontier, then AMD would be chasing after it too. Instead, they sell a pretty damned good enterprise compute product that actually has a life outside of the AI bubble, and will keep selling it to happy customers long after the AI bubble pops.
The question you need to ask is how bad will Nvidia's collapse be. Like, I'm pretty sure some form of the company will still exist, but they have nothing stopping their valuation going from $4T back down to something more reasonable like $250m, they don't do anything any better than anyone else, its all 100% Jensen wooing people with his leather jacket collection.
Most of Nvidia's "strength" is the heavy lifting done by the folks over at TSMC.
Apple and AMD both have access to the same TSMC technology, but neither of them are willing to invest any more into AI than they are currently. Apple ships an inference solution that is a combination of GPU and NPU and mostly okay for inference at home, and AMD ships a high performance enterprise compute solution that scales the whole way up to the needs of supercomputers without needing to make a purely AI-only product line.
If this was the next big money making frontier, then AMD would be chasing after it too. Instead, they sell a pretty damned good enterprise compute product that actually has a life outside of the AI bubble, and will keep selling it to happy customers long after the AI bubble pops.
The question you need to ask is how bad will Nvidia's collapse be. Like, I'm pretty sure some form of the company will still exist, but they have nothing stopping their valuation going from $4T back down to something more reasonable like $250m, they don't do anything any better than anyone else, its all 100% Jensen wooing people with his leather jacket collection.
Nvidia have completely dominated in consumer hardware for several generations, this seems like a bunch of claims that's at complete odds with reality?
It's far more believable that AMD aren't competing because they can't.
It's far more believable that AMD aren't competing because they can't.
Nvidia lost the contract for now two generations of consoles.
AMD was XBone, XSX, PS4, and PS5.
Nvidia was only able to get in on the Switch, a console that "sold well" because it stretched across two generations, and in a units/yr basis, sold less than either of the four. Nvidia sold the Tegra X1 to Nintendo at a break-even just to get the console down to $300, else it was a no-go for consumers.
The Switch 2? Basically DOA, I'm not sure how either Nvidia or Nintendo is going to downplay this.
AMD was XBone, XSX, PS4, and PS5.
Nvidia was only able to get in on the Switch, a console that "sold well" because it stretched across two generations, and in a units/yr basis, sold less than either of the four. Nvidia sold the Tegra X1 to Nintendo at a break-even just to get the console down to $300, else it was a no-go for consumers.
The Switch 2? Basically DOA, I'm not sure how either Nvidia or Nintendo is going to downplay this.
> Most of Nvidia's "strength" is the heavy lifting done by the folks over at TSMC.
The software side is also important. No discussion of Nvidia's moat is complete without also mentioning CUDA.
The software side is also important. No discussion of Nvidia's moat is complete without also mentioning CUDA.
I don't really see it worth mentioning CUDA.
It does nothing that any other compute API uses, and the majority of enterprise compute software doesn't use it and/or works on ROCm HIP with minimal performance loss.
A lot of research projects (such as all the early LLM research, given the topic) are written in Python and use libraries to shim all of that as well; PyTorch and ONNX both run natively on AMD and is covered under AMD's commercial support.
And then we come to the case of llama.cpp, which supports more APIs than any other inference engine... not only does it run on Nvidia/CUDA, it runs on AMD/HIP, Vulkan on at least 4 different vendors, SYCL on at least Intel ARC, BLAS/BLIS, Apple/Metal, Snapdragon's quasi-NPU, and Moore Threads (that new Chinese startup for domestic GPUs).
There is no reason to write greenfield code with CUDA today, and most people aren't.
It does nothing that any other compute API uses, and the majority of enterprise compute software doesn't use it and/or works on ROCm HIP with minimal performance loss.
A lot of research projects (such as all the early LLM research, given the topic) are written in Python and use libraries to shim all of that as well; PyTorch and ONNX both run natively on AMD and is covered under AMD's commercial support.
And then we come to the case of llama.cpp, which supports more APIs than any other inference engine... not only does it run on Nvidia/CUDA, it runs on AMD/HIP, Vulkan on at least 4 different vendors, SYCL on at least Intel ARC, BLAS/BLIS, Apple/Metal, Snapdragon's quasi-NPU, and Moore Threads (that new Chinese startup for domestic GPUs).
There is no reason to write greenfield code with CUDA today, and most people aren't.
Nvidia's products have been the biggest cash grab of all time. I don't think it's a matter of other companies believing it is just a bubble and therefore not attempting to compete, but that Nvidia hasn't left them room to compete effectively. That's what I'm trying to get to the bottom of.
Run CUDA