Nvidia Announces Financial Results for Fourth Quarter and Fiscal 2024(nvidianews.nvidia.com)
nvidianews.nvidia.com
Nvidia Announces Financial Results for Fourth Quarter and Fiscal 2024
https://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-fourth-quarter-and-fiscal-2024
96 コメント
This is like playing a game of poker where you already think the guy has the best possible hand he could have and somehow when he lays his cards down, it’s even better.
I remember a time when NVIDIA, ATI and 3dFX were neck and neck.
What ever happened?
It shows that the most important move is to simply stay in the game, not just maximize profits
Edit: added ATI which I just remembered!
What ever happened?
It shows that the most important move is to simply stay in the game, not just maximize profits
Edit: added ATI which I just remembered!
>What ever happened?
3dfx stagnated and fell behind due to bad management decisions. They seemed stuck on designing 3d accelerators like it was still 1996, which were just drawing triangles and doing texture mapping, while banking on higher clocks and adding multiple chips on board and even multiple cards in SLI to boost performance, while having the CPU take care of the rest in SW.
But it was 1999 already and Nvidia moved to graphics processing units (GPUs) which could also offload the CPU and do transform, lighting, triangle setup/clipping, MPEG2 motion compensation and a multi-pipeline rendering engine for more polygon thruput at same clocks, along with being fully DX7 compliant, all in a single chip.
The market decided that Nvidia's new direction is the future and not what 3dfx was doing. 3df tried to change course and catch up but it was too late, the GPU race was too cutthroat in those days. Nvidia was launching a new GPU generation every 6 months(!) instead of every 3 years like today, so a 6-12 month setback was a guaranteed death sentence no matter what you did.
Also, one of 3dfx biggest blunders was trying to become a board maker themselves by spending a lot of cash buying a board manufacturer, which emptied their bank accounts and angered their other board partners who now saw them as unfair competitors so they moved to making ATI/Nvidia boards instead.
>It shows that the most important move is to simply stay in the game, not just maximize profits
How do you stay in the game if you go bankrupt because your products aren't competitive?
3dfx stagnated and fell behind due to bad management decisions. They seemed stuck on designing 3d accelerators like it was still 1996, which were just drawing triangles and doing texture mapping, while banking on higher clocks and adding multiple chips on board and even multiple cards in SLI to boost performance, while having the CPU take care of the rest in SW.
But it was 1999 already and Nvidia moved to graphics processing units (GPUs) which could also offload the CPU and do transform, lighting, triangle setup/clipping, MPEG2 motion compensation and a multi-pipeline rendering engine for more polygon thruput at same clocks, along with being fully DX7 compliant, all in a single chip.
The market decided that Nvidia's new direction is the future and not what 3dfx was doing. 3df tried to change course and catch up but it was too late, the GPU race was too cutthroat in those days. Nvidia was launching a new GPU generation every 6 months(!) instead of every 3 years like today, so a 6-12 month setback was a guaranteed death sentence no matter what you did.
Also, one of 3dfx biggest blunders was trying to become a board maker themselves by spending a lot of cash buying a board manufacturer, which emptied their bank accounts and angered their other board partners who now saw them as unfair competitors so they moved to making ATI/Nvidia boards instead.
>It shows that the most important move is to simply stay in the game, not just maximize profits
How do you stay in the game if you go bankrupt because your products aren't competitive?
Just to add to what you're saying, the Acquired podcast has a couple long-form episodes on Nvidia and its competition:
https://www.acquired.fm/episodes/nvidia-the-gpu-company-1993...
https://www.acquired.fm/episodes/nvidia-the-machine-learning...
CUDA itself is also a non-negligible factor in their success.
https://www.acquired.fm/episodes/nvidia-the-gpu-company-1993...
https://www.acquired.fm/episodes/nvidia-the-machine-learning...
CUDA itself is also a non-negligible factor in their success.
CUDA wasn't even a thing when 3dfx went bust. Work on what was to become CUDA started at Berkely by a phd student sometime in 2000. CUDA first released to the general public all the way in mid-2007.
It was Ian Buck @ Stanford, I think. I remember a professor from Stanford gave a talk at my grad school around 2000 about GPUs for scientific computing. Blew me away. One of the few talks I remember.
You’re right. 3dfx is an earlier story.
Nvidia invested in CUDA for decades! Following research papers in computer graphics or physics for example, you'd find a transition from Fortran to CUDA over many years. That helped quite a lot when ML came back and CUDA made it easy. They ported many basics like FFT, BLAS, etc. to CUDA so the building blocks were there already when AI grew too. Of course, it helped that many Python ML library backends also supported GPU via CUDA.
If you look at the same landscape from AMD's side, it's lackluster to say the least. And they're only now trying to catch up... And they have to make it CUDA compatible!
Nvidia won on the tools side, the GPUs aren't necessarily faster, but the only way to use them easily is CUDA.
If you look at the same landscape from AMD's side, it's lackluster to say the least. And they're only now trying to catch up... And they have to make it CUDA compatible!
Nvidia won on the tools side, the GPUs aren't necessarily faster, but the only way to use them easily is CUDA.
> What ever happened?
For 3dfx, the Voodoo 3 happened.
Whereas previous Voodoo cards were sold by a large variety of OEMs (like Diamond or ELSA), 3dfx decided to make the Voodoo 3 graphic card on their own. It was a perfectly fast and capable card, but also expensive (for its time), not as easy to get as OEM cards were, couldn't do 32-Bit textures (16-Bit only), and didn't support AGP Texturing.
nVidia's TNT2 was a bit slower, but not by much, and thanks to the many OEMs making cards, you could pick one up anywhere and usually pretty cheap. (Just make sure you're not getting the M64 by accident. But even then, that one might've been a good way to get a capable 3D graphics card for cheap).
3dfx was still competitive here and probably the best gaming card, but they had clearly bitten off more than they could chew, and it looks to me that their R&D just didn't produce anything else than "Voodoo 2 but a bit faster and delayed multiple times".
When nVidia released the Geforce 256 (Hardware Transform and Lighting!), that was it. Both the Voodoo 4 and 5 were duds (though seeing 3dfx's complete brute force approach with the never-released Voodoo 5 6000 still brings a smile to my face), they couldn't get other products to the market quick enough, and went bankrupt starting in mid 2000.
I don't even think it was about "GPU" stuff: The first programmable shader card (and thus "GPU") was the Geforce 3 in 2001, at which point 3dfx was already out of money and without perspective.
I'm not sure if they could have been saved if Sega would have chosen them for the Dreamcast (seeing how the Dreamcast sunk Sega's console business) and it might have been possible to save 3dfx still, but realistically, the one-two punch of the Voodoo 3 and Geforce 256 killed them.
For 3dfx, the Voodoo 3 happened.
Whereas previous Voodoo cards were sold by a large variety of OEMs (like Diamond or ELSA), 3dfx decided to make the Voodoo 3 graphic card on their own. It was a perfectly fast and capable card, but also expensive (for its time), not as easy to get as OEM cards were, couldn't do 32-Bit textures (16-Bit only), and didn't support AGP Texturing.
nVidia's TNT2 was a bit slower, but not by much, and thanks to the many OEMs making cards, you could pick one up anywhere and usually pretty cheap. (Just make sure you're not getting the M64 by accident. But even then, that one might've been a good way to get a capable 3D graphics card for cheap).
3dfx was still competitive here and probably the best gaming card, but they had clearly bitten off more than they could chew, and it looks to me that their R&D just didn't produce anything else than "Voodoo 2 but a bit faster and delayed multiple times".
When nVidia released the Geforce 256 (Hardware Transform and Lighting!), that was it. Both the Voodoo 4 and 5 were duds (though seeing 3dfx's complete brute force approach with the never-released Voodoo 5 6000 still brings a smile to my face), they couldn't get other products to the market quick enough, and went bankrupt starting in mid 2000.
I don't even think it was about "GPU" stuff: The first programmable shader card (and thus "GPU") was the Geforce 3 in 2001, at which point 3dfx was already out of money and without perspective.
I'm not sure if they could have been saved if Sega would have chosen them for the Dreamcast (seeing how the Dreamcast sunk Sega's console business) and it might have been possible to save 3dfx still, but realistically, the one-two punch of the Voodoo 3 and Geforce 256 killed them.
>Just make sure you're not getting the M64 by accident.
That's how my parents got scammed when I asked them for a PC for Christmas. Yes, I got one with a TNT2 M64 because that was on sales in Christmas 2000/2001. Childhood ruined.
That's how my parents got scammed when I asked them for a PC for Christmas. Yes, I got one with a TNT2 M64 because that was on sales in Christmas 2000/2001. Childhood ruined.
I'd say Nvidia's release of Cg differentiated them, winning wide developer favor, with a positive constructive feedback.
Cg made programmability of shaders widely accessible, since C was (20 years ago) widely known?
https://en.wikipedia.org/wiki/Cg_(programming_language)
Cg made programmability of shaders widely accessible, since C was (20 years ago) widely known?
https://en.wikipedia.org/wiki/Cg_(programming_language)
Well ATI is still around as AMD. But definitely behind the lead.
I remember when SLI was two graphics cards, each handling alternate scan lines of a vga monitor.
CFO: "In Q4 FY 24, large cloud providers represented more than half of our Data Center revenue..Strong demand was driven by enterprise software and consumer internet applications & multiple industry verticals"
Not clear if Meta is included in the "cloud providers", if not that would push the share of revenue from Big Tech even further to 60-70% considering the 300k H100 order from Meta.
Not clear if Meta is included in the "cloud providers", if not that would push the share of revenue from Big Tech even further to 60-70% considering the 300k H100 order from Meta.
Random, but a serious question. I've just received an offer from Nvidia as a Senior SWE, and another from [insert FAANG].
Should I be scared of the bubble-ness of Nvidia? I have a lot of faith in the company and its vision, but reading about the bubble always scares me.
Should I be scared of the bubble-ness of Nvidia? I have a lot of faith in the company and its vision, but reading about the bubble always scares me.
You should ABSOLUTELY take the Nvidia offer over other FAANG. The share price of Nvidia is not speculative hype. It's very reasonable given their growth. This is not some random crypto coin. Nvidia's share price really does reflect economic reality and their position in the market. As an employee there, you'll have the opportunity to purchase shares at a reduced price through their Employee Purchasing Program. Also, any RSUs or options you get will appreciate tremendously. Financially, the best decision here is to work at Nvidia.
Jensen also seems to be playing the long game and not letting go of that focus. This seems unique to me given that so many companies are hyper focused on the next quarter. I think Nvidia recognizes that there are more things than money (which was one of the things that set Google apart in the early days). Things like free days go a long way (company wide long weekend per quarter). To me the growth is more Nvidia being in the right place at the right time and taking full advantage of every leg up that they have. I don't think speculation because they aren't just celebrating the luck, but rather acting like they are seizing a lucky opportunity are are aware that it requires a lot of work to maintain.
I don't know what the future will bring but the entire rally around AI and especially nvidia is the purest definition of speculation...
I'm highly critical of AI, even being a ML researcher myself. I'm happy to criticize GPT, explain, and collect many downvotes for these views. I say this not to act victim, but to give context for what I'm about to say. AI isn't going away. The hype is likely to die and I think we're seeing the cracks that come from over promising. But just because people are putting a polished turd on top of something doesn't mean that something isn't a golden nugget. Actually, that's my main complaint at what's going on. We got something special and very powerful. It is weird to cover it in shit and try to convince everyone it's a fancy polish.
If scale is all you need, then we'll get there. But I'm not remotely convinced and see this as the bitter lesson's bitter lesson (a misinterpretation of the bitter lesson). But if we fund alternative pathways and hedge our bets, it's possible we get there without missing a step. But imo it looks like we've created a railroad and are just going all in on laying more tracks. Doesn't seem like the right move to me, but hey, I don't know the future any more than anyone else.
If scale is all you need, then we'll get there. But I'm not remotely convinced and see this as the bitter lesson's bitter lesson (a misinterpretation of the bitter lesson). But if we fund alternative pathways and hedge our bets, it's possible we get there without missing a step. But imo it looks like we've created a railroad and are just going all in on laying more tracks. Doesn't seem like the right move to me, but hey, I don't know the future any more than anyone else.
> The share price of Nvidia is not speculative hype.
The share price of nvidia will collapse as soon as anyone else releases a competitive datacenter gpu. End of the day GPU compute is fully commoditized. Massive margins in commodity markets dont exist because competition leads to a race to the bottom in pricing. Hard for me to imagine that doesnt happen eventually, but wouldnt be surprised if it took 5 years.
The share price of nvidia will collapse as soon as anyone else releases a competitive datacenter gpu. End of the day GPU compute is fully commoditized. Massive margins in commodity markets dont exist because competition leads to a race to the bottom in pricing. Hard for me to imagine that doesnt happen eventually, but wouldnt be surprised if it took 5 years.
What makes you think GPU compute is acting like a commodity right now? AMD has comparable hardware but nowhere near the same margins or demand for their chips.
CUDA is stopping GPUs from being commoditized today. But how can that last when every big tech company can save billions of dollars by propping up a competitor? Sure intel is incompetent and AMD might not have the software know how but they can work on their open source offering with armies of developers from other companies at their sides.
AMD really isn’t as good at the high end (by quite a decent percentage depending on the tasks) and in perf per watt term they are far ahead.
Even on the gaming side the only thing somewhat saving AMD is their lower prices for traditional game rendering (without the AI utilities on top) ; they don’t even have competitive ray tracing yet.
I don’t what are you on about…
I don’t what are you on about…
I'm sorry but this must be one of the worst advice imaginable.
The company itself is great, and is making a ton of money. But the share price of Nvidia is absolutely pure speculative hype !
The math - in the past year, during which Nvidia suddenly found itself in a monopoly situation during a huge spike in AI interest (or / and hype), they made 60B$ in revenue (x2 from 2023), and 30B$ in net profit (x10 ! from 2023). Both numbers are huge. But ... the company is valued at 1800 B$ currently. There is this pesky little ratio that is sometimes useful, called price / earning ratio, and it gives a number around 60 here.
So in short - great company, perfect positioning, but way overpriced.
The company itself is great, and is making a ton of money. But the share price of Nvidia is absolutely pure speculative hype !
The math - in the past year, during which Nvidia suddenly found itself in a monopoly situation during a huge spike in AI interest (or / and hype), they made 60B$ in revenue (x2 from 2023), and 30B$ in net profit (x10 ! from 2023). Both numbers are huge. But ... the company is valued at 1800 B$ currently. There is this pesky little ratio that is sometimes useful, called price / earning ratio, and it gives a number around 60 here.
So in short - great company, perfect positioning, but way overpriced.
For whatever it's worth it to you, I've been with NVIDIA for over 10 years and I wouldn't switch to F,A,A,N nor G for even 2x my total comp. Feel free to shoot any questions to [email protected] if it'll help you decide.
Or, just talk to your hiring manager(s) about the dilemma. For both companies, it'd be better to talk through any concerns before joining. If you accept one offer and then regret it a few months later, everyone loses: you, NVIDIA and [FAANG].
Or, just talk to your hiring manager(s) about the dilemma. For both companies, it'd be better to talk through any concerns before joining. If you accept one offer and then regret it a few months later, everyone loses: you, NVIDIA and [FAANG].
It is already a highly speculated bubble, the share price is swinging drastically within the same day.
That being said, Nvidia will have a good 2-3 years in the future, until the cloud providers start mass replace their chips with in-house ones.
For money, unfortunately, it is as good as anyone's guess, but if you join now, double your share, it will be close to 4T. I personally didn't think that is sustainable amount of money if market/competition does their duty.
That being said, Nvidia will have a good 2-3 years in the future, until the cloud providers start mass replace their chips with in-house ones.
For money, unfortunately, it is as good as anyone's guess, but if you join now, double your share, it will be close to 4T. I personally didn't think that is sustainable amount of money if market/competition does their duty.
> Nvidia will have a good 2-3 years in the future, until the cloud providers start mass replace their chips with in-house ones
How do you imagine that happening on such a rapid timeline?
IMO, the biggest threat to Nvidia is radical innovation on the software side of AI that allows common use cases (training and inference) to run on clusters of commodity non-GPU hardware for cheaper price than GPU.
A similar threat is posed from ASIC chips that are dedicated to LLM tasks, but at least in that case Nvidia can still compete for a specific hardware design rather than overcoming an entire category of commodity hardware.
How do you imagine that happening on such a rapid timeline?
IMO, the biggest threat to Nvidia is radical innovation on the software side of AI that allows common use cases (training and inference) to run on clusters of commodity non-GPU hardware for cheaper price than GPU.
A similar threat is posed from ASIC chips that are dedicated to LLM tasks, but at least in that case Nvidia can still compete for a specific hardware design rather than overcoming an entire category of commodity hardware.
Simply because of there is too much money on the table, news from today:
https://qz.com/intel-microsoft-new-chip-semiconductor-nvidia...
NVDA's margin is too high, it attracts competitors to eat into that.
https://qz.com/intel-microsoft-new-chip-semiconductor-nvidia...
NVDA's margin is too high, it attracts competitors to eat into that.
Considering how long it took apple to design a competent chip I think Nvidia will be fine for quite a while. Really their current hardware design is unparalleled and they even have some software lock-in with CUDA…
But how are competitors going to magically start creating market leading specialized hardware in 2-3 years? Manufacturing chips is not a core competency of any of the cloud providers, and to the extent they've been trying to make it one, they're focused on CPU.
Well there is TPU, and it is at least as good as GPU for training, all Google's models are trained TPU, it is the best class of training chips in industry.
Second, there is no need to be market leading, it just needs to be cost efficiently to be successful, even if it is slower, as long as it is cheaper. All cloud providers have huge chip design/manufacturing history, AWS/Azure and Google.
I would argue it IS indeed the core competency for those cloud providers, because of price. The inhouse chips exist because they can reduce the cost.
I know for a fact, there is huge momentum in some big clouds to replace Nvidia GPUs with their own chips, at least for their internal compute needs at first.
Second, there is no need to be market leading, it just needs to be cost efficiently to be successful, even if it is slower, as long as it is cheaper. All cloud providers have huge chip design/manufacturing history, AWS/Azure and Google.
I would argue it IS indeed the core competency for those cloud providers, because of price. The inhouse chips exist because they can reduce the cost.
I know for a fact, there is huge momentum in some big clouds to replace Nvidia GPUs with their own chips, at least for their internal compute needs at first.
All the cloud megasclaers already have in house chip design teams that have released successful products. Designing a GPU is easier than a cpu, they just need to figure out CUDA.
They have some small manufacturing operations that are producing a minority of the chips they actually use in their servers. Maybe they'll be able to produce enough GPUs for their use cases eventually, but there is no way they'll get there in "2-3 years" as OP suggests.
Also, if GPUs are easier to make than CPUs, then why isn't Intel leading the market in GPUs by now? Surely it's been clear they should compete with Nvidia there for at least the past half a decade. Or do they need more than 2-3 years to catch up?
Also, if GPUs are easier to make than CPUs, then why isn't Intel leading the market in GPUs by now? Surely it's been clear they should compete with Nvidia there for at least the past half a decade. Or do they need more than 2-3 years to catch up?
They dont manufacture themselves so ramping up is as simple as "here's another billion build 10 times as many." If they design GPUs that they think have significant cost advantages without downsides they will absolutely make that call.
I think you're making a lot of assumptions about specialist employees and hardware suddenly materializing in the place where they're needed to manufacture the stuff.
I'm not disagreeing they can catch up within 10 years. But 2-3 years is an insane estimate IMO.
I'm not disagreeing they can catch up within 10 years. But 2-3 years is an insane estimate IMO.
> specialist employees and hardware suddenly materializing in the place where they're needed to manufacture the stuff.
The beauty of being fabless is that they dont need to worry about that. When theyve got a design they like they just call TSMC and buy out all their capacity, they could pay double what nvidia does and still save a boatload. I agree that 2 years seems impossible and 3 a stretch, but I do think its possible for at least one cloud company to have something by then.
The beauty of being fabless is that they dont need to worry about that. When theyve got a design they like they just call TSMC and buy out all their capacity, they could pay double what nvidia does and still save a boatload. I agree that 2 years seems impossible and 3 a stretch, but I do think its possible for at least one cloud company to have something by then.
> A similar threat is posed from ASIC chips that are dedicated to LLM tasks, but at least in that case Nvidia can still compete for a specific hardware design rather than overcoming an entire category of commodity hardware.
But also consider the reason Nvidia is doing so well in the first place. The reason is because they were already developing hardware that specialized at performing tensor operations. Maybe not at the size or always the same types we see in ML, but not very far either. There's reasons why they aren't filling cards with tensor cores. But they are clearly developing specialized hardware for ML tasks. I'm not sure betting on LLMs sticking around is the best bet though. I'll be very impressed if scale is all you need.
But also consider the reason Nvidia is doing so well in the first place. The reason is because they were already developing hardware that specialized at performing tensor operations. Maybe not at the size or always the same types we see in ML, but not very far either. There's reasons why they aren't filling cards with tensor cores. But they are clearly developing specialized hardware for ML tasks. I'm not sure betting on LLMs sticking around is the best bet though. I'll be very impressed if scale is all you need.
Don't think of it like that. Go where you think you'll be the most productive and learn the most. Nvidia could grow substantially from here(probably less than 50% chance of that) or they might not, but your RSUs are going to be based on the current stock price so you'll be getting less of them. The ESPP is capped so it's not like you can dump 100% of your salary into it and use a free money glitch.
If NVDA has a position that fits you and you like the team, go for it! If you're just chasing money.. please don't.
If NVDA has a position that fits you and you like the team, go for it! If you're just chasing money.. please don't.
My impression is they still have a fairly efficient culture, the main problem with joining a bubble company would be if they are overhiring/losing focus.
Just sell shares when they vest to reduce your risk.
Just sell shares when they vest to reduce your risk.
Think of Sam Altman's ask: $7T. 7 trillion dollars is going to flow into a company that looks a lot like Nvidia whether its directed by Sam or not.
https://www.astralcodexten.com/p/sam-altman-wants-7-trillion
https://www.astralcodexten.com/p/sam-altman-wants-7-trillion
Take it and RUN with it.
Nvidia stock is extraordinarily stretched, even by it's own historical standards and it has been really stretched twice before (2017, 2021).
It's a consensus Wall Street favorite. Retail traders who barely know what they do are now being lured in. I think the stock is close to a major top.
Bull markets are born on pessimism, grow on skepticism, mature on optimism and die on euphoria. Judging by general consensus I follow here, on Twitter, and in the financial press, I'd say we are basically at euphoria. Euphoria can last for a bit but not for long. Not only do I not see a ton of upside in the share price here, I think it's a fantastic short opportunity, as when the tide goes out on a stock like this that is totally driven by the idea that it is supply-constrained, if that looks to be in doubt for any reason whatsoever, it is going to fall very far. I'm talking back to the double digits given the amount of supply they are currently producing.
Keep in mind that there are only a handful of companies that can afford these chips at scale and the business model for what they are doing is far from proven. At some point, likely soon given where interest rates are, they are going to need to show a ROI or they are going to be forced to cut spending.
It's a very unique positive feedback loop. Here you have the wealthiest most profitable companies in history that all have a vested interest in not being viewed as waving off the AI hype train. Therefore they basically have to show the street that they're investing which drives up demand for the chips and consequently the price. But you must understand that this process can play out in reverse at 10x the speed. All you need is consensus that spending on AI is suddenly not such a great investment.
I suspect that what you might see happen is earnings for Microsoft, Alphabet, Meta, and Amazon start to suffer. This will require them to cut spending on data center development (unless they are able to show profit growth coming from that investment- unlikely on a short time scale). At that point Nvidia is going to have to cut costs and likely cut back on supply. This is the identical cycle that played out during the last hype cycle, which hilariously, centered on the Metaverse.
It's a consensus Wall Street favorite. Retail traders who barely know what they do are now being lured in. I think the stock is close to a major top.
Bull markets are born on pessimism, grow on skepticism, mature on optimism and die on euphoria. Judging by general consensus I follow here, on Twitter, and in the financial press, I'd say we are basically at euphoria. Euphoria can last for a bit but not for long. Not only do I not see a ton of upside in the share price here, I think it's a fantastic short opportunity, as when the tide goes out on a stock like this that is totally driven by the idea that it is supply-constrained, if that looks to be in doubt for any reason whatsoever, it is going to fall very far. I'm talking back to the double digits given the amount of supply they are currently producing.
Keep in mind that there are only a handful of companies that can afford these chips at scale and the business model for what they are doing is far from proven. At some point, likely soon given where interest rates are, they are going to need to show a ROI or they are going to be forced to cut spending.
It's a very unique positive feedback loop. Here you have the wealthiest most profitable companies in history that all have a vested interest in not being viewed as waving off the AI hype train. Therefore they basically have to show the street that they're investing which drives up demand for the chips and consequently the price. But you must understand that this process can play out in reverse at 10x the speed. All you need is consensus that spending on AI is suddenly not such a great investment.
I suspect that what you might see happen is earnings for Microsoft, Alphabet, Meta, and Amazon start to suffer. This will require them to cut spending on data center development (unless they are able to show profit growth coming from that investment- unlikely on a short time scale). At that point Nvidia is going to have to cut costs and likely cut back on supply. This is the identical cycle that played out during the last hype cycle, which hilariously, centered on the Metaverse.
How much of it is zuck buying up all available nvidia product at any price ? AI is a more credible obsession than the metaverse. So he could just buy all nvidia output for the next few years…
Is there a limit to the number of GPU's a company may need? I mean for example, assume no one can compete with Nvidia. At what point does it become more feasible to rewrite your architecture or use less GPU's, since it seems Nvidia can charge, and does charge, what ever they want for a GPU. And can charge more if they so wish, which they might. And how long would that take?
> At what point does it become more feasible to rewrite your architecture or use less GPU
They are but it takes a lot of time.
Most of the big players - Google, Meta, OpenAI, Amazon, and Microsoft all are actively developing TPU/NPUs that would be used instead of the H100/A100's everyone is using for machine learning.
Google (tensorflow/jax), Meta(pytorch), Microsoft(onyx), Openai(triton) and Apple (mlx) each have software stacks for optimizing models for multiple platforms.
It takes a lot of time to develop the silicon and software stack. As a result everyone is using H100's in the interum until the hardware/software catches up. Google has been using their TPUs already.
There's other companies like Groq that are also developing NPU/TPU like devices.
They are but it takes a lot of time.
Most of the big players - Google, Meta, OpenAI, Amazon, and Microsoft all are actively developing TPU/NPUs that would be used instead of the H100/A100's everyone is using for machine learning.
Google (tensorflow/jax), Meta(pytorch), Microsoft(onyx), Openai(triton) and Apple (mlx) each have software stacks for optimizing models for multiple platforms.
It takes a lot of time to develop the silicon and software stack. As a result everyone is using H100's in the interum until the hardware/software catches up. Google has been using their TPUs already.
There's other companies like Groq that are also developing NPU/TPU like devices.
Is there an Open (source) TPU project?
There is SIMT on RISC V: https://www.semanticscholar.org/paper/Simty-%3A-generalized-...
And some PoC work: https://vortex.cc.gatech.edu/publications/hotchips-poster.pd...
Neither are specialized TPU/NPUs but they do fast vector operations.
And some PoC work: https://vortex.cc.gatech.edu/publications/hotchips-poster.pd...
Neither are specialized TPU/NPUs but they do fast vector operations.
There are only a few companies making competitors at scale. It's very interesting that Google doesn't sell TPUs. They don't because they need every TPU they can get fabricated.
Using the underlying cost, it is not feasible to create a general purpose parallel processor for cheaper than Nvidia. Especially considering the economies of scale. So you basically need to make a bet on a less general purpose accelerator, as Nvidia loses its economies of scale when you do that.
When you include Nvidia's huge margins, this bet is less risky, but it's still only one that the largest companies can make.
Software alone cannot do it, you need to make a bet on hardware.
When you include Nvidia's huge margins, this bet is less risky, but it's still only one that the largest companies can make.
Software alone cannot do it, you need to make a bet on hardware.
Another angle is that it is highly unlikely to be worth doing since we're so early in the AI cycle.
It's so challenging, capital intensive, and takes so long to bring these things online (especially if it's not your core competency), that but by the time you got something working in this paradigm, it's possible that a better paradigm/approach will have emerged.
It's so challenging, capital intensive, and takes so long to bring these things online (especially if it's not your core competency), that but by the time you got something working in this paradigm, it's possible that a better paradigm/approach will have emerged.
each generation of models will require larger training clusters... at least from the majors.
Google is already more reliant on their own stuff, their Nvidia addiction is just because cloud customers want them, but internal use which is probably much bigger than cloud use they are all set up to do with their own chips.
Minimum 3 to 4 years.
Whatever $7T can get you?
Joking aside, it is quite expensive and I'm pretty sure you'd run into legal issues due to vertical integration, depending how you perform it. I'll put it this way, a few years ago the headlines were about how China was poaching TSMC workers and offering huge salaries[0,1]. I haven't seen Chinese chips become competitive yet, so it looks like >5 years.
Beyond that, it's more than the chip. AMD has caught up to Nvidia in hardware. But no one is rushing to buy AMD cards because they are still not as good. Nvidia's secret sauce is Cuda (MKL is still an advantage to Intel). The naivity of the Tiny Corp was thinking that everything could be resolved in a few weekends of hacking. But Cuda is deep in a lot of projects. Like a project's backend's backend's backend deep. You got decades of engineers using Cuda and the huge momentum around it. While most programmers will never touch Cuda or GPU code, almost everyone touches things that connect with them (and this is every single day for many people. We could also say the same thing about optimization libraries like MKL). It is something that looks simple because we don't talk about it much or haven't had the experience with. But I assure you, GPU programming is a whole other world. Optimized programming is a very different style of programming and you need a different framework of thinking and problem solving. GPUs add another lay of complexity on top of that. It's why you'll hear so many people complain about writing kernels, but damn, the results speak for themselves.
So you gotta match on the hardware. Then you got to develop great software. Then you got to get that software into the other software that everyone else is using. And you gotta convert people along the way, getting them to turn from a thing they already know and have experience with to a completely new thing.
```edit
The disadvantage of being a first mover is you got to invent everything yourself and page the path, letting others follow. But the disadvantage of being a follower is that to get people to use your road or lane you can't just be equal, you have to be *better*. And you usually have to be significantly so. Momentum is a really powerful force and I think it is highly undervalued.
I think Nvidia is safe for the next few years. They aren't slacking and relying on their momentum. They're still pushing very hard, which only makes it harder for competitors. It's hard to displace a sleeping giant. It's even harder when that giant is fighting back.
```
> And how long would that take?
A long fucking time.
[0] https://news.ycombinator.com/item?id=24129861
[1] https://www.reuters.com/technology/taiwan-raids-chinese-firm...
Joking aside, it is quite expensive and I'm pretty sure you'd run into legal issues due to vertical integration, depending how you perform it. I'll put it this way, a few years ago the headlines were about how China was poaching TSMC workers and offering huge salaries[0,1]. I haven't seen Chinese chips become competitive yet, so it looks like >5 years.
Beyond that, it's more than the chip. AMD has caught up to Nvidia in hardware. But no one is rushing to buy AMD cards because they are still not as good. Nvidia's secret sauce is Cuda (MKL is still an advantage to Intel). The naivity of the Tiny Corp was thinking that everything could be resolved in a few weekends of hacking. But Cuda is deep in a lot of projects. Like a project's backend's backend's backend deep. You got decades of engineers using Cuda and the huge momentum around it. While most programmers will never touch Cuda or GPU code, almost everyone touches things that connect with them (and this is every single day for many people. We could also say the same thing about optimization libraries like MKL). It is something that looks simple because we don't talk about it much or haven't had the experience with. But I assure you, GPU programming is a whole other world. Optimized programming is a very different style of programming and you need a different framework of thinking and problem solving. GPUs add another lay of complexity on top of that. It's why you'll hear so many people complain about writing kernels, but damn, the results speak for themselves.
So you gotta match on the hardware. Then you got to develop great software. Then you got to get that software into the other software that everyone else is using. And you gotta convert people along the way, getting them to turn from a thing they already know and have experience with to a completely new thing.
```edit
The disadvantage of being a first mover is you got to invent everything yourself and page the path, letting others follow. But the disadvantage of being a follower is that to get people to use your road or lane you can't just be equal, you have to be *better*. And you usually have to be significantly so. Momentum is a really powerful force and I think it is highly undervalued.
I think Nvidia is safe for the next few years. They aren't slacking and relying on their momentum. They're still pushing very hard, which only makes it harder for competitors. It's hard to displace a sleeping giant. It's even harder when that giant is fighting back.
```
> And how long would that take?
A long fucking time.
[0] https://news.ycombinator.com/item?id=24129861
[1] https://www.reuters.com/technology/taiwan-raids-chinese-firm...
> But no one is rushing to buy AMD cards because they are still not as good.
MI300x is excellent and I'm buying them. =)
MI300x is excellent and I'm buying them. =)
Unless Sam gets his 7T I can’t see the data center growth continuing on pace.
I still would love to see a consumer card with 128GB of VRAM
I still would love to see a consumer card with 128GB of VRAM
I would love to see a rethinking of desktop hardware. GPU's are huge. They used to be dinky little add-on cards ... but at this point we almost need to have a dedicated mainboard for them.
Imagine this - instead of a motherboard mounted to the back of the case, it is mounted in the center. Behind it would be another motherboard, with a socket for a GPU and ram slots for memory. Kinda like a right/left brain connected with a high-speed interconnect. You open the left side of the case to access CPU and its memory, you open the right side of the case to access the GPU and its memory.
You would be able to upgrade your GPU chip independently of everything else. It wouldn't be mounted sideways in a PCI slot where additional brackets and support arms are needed. You could upgrade the RAM independently. Cooler, etc.
Imagine this - instead of a motherboard mounted to the back of the case, it is mounted in the center. Behind it would be another motherboard, with a socket for a GPU and ram slots for memory. Kinda like a right/left brain connected with a high-speed interconnect. You open the left side of the case to access CPU and its memory, you open the right side of the case to access the GPU and its memory.
You would be able to upgrade your GPU chip independently of everything else. It wouldn't be mounted sideways in a PCI slot where additional brackets and support arms are needed. You could upgrade the RAM independently. Cooler, etc.
What you're describing is fairly close to Intel's Beast Canion NUC [0] from a couple of years back. The Mobo and GFX essentially sit back to back. Though intel seemed to envisage upgrading the whole "CPU side" as a single piece (including ram) which probably, along with its high price, put off buyers.
Following on from that though, there are a number of miniitx cases that followed on from that type of layout such as the fractal terra [1] which utilize a pcie riser to pass it rearward to the graphics card behind the motherboard. ATX/ITX wasn't entirely designed for it, but the layout does seem to be quite elegant.
[0] https://www.tomshardware.com/reviews/nuc-11-extreme-kit-beas...
[1] https://www.fractal-design.com/products/cases/terra/terra/te...
Following on from that though, there are a number of miniitx cases that followed on from that type of layout such as the fractal terra [1] which utilize a pcie riser to pass it rearward to the graphics card behind the motherboard. ATX/ITX wasn't entirely designed for it, but the layout does seem to be quite elegant.
[0] https://www.tomshardware.com/reviews/nuc-11-extreme-kit-beas...
[1] https://www.fractal-design.com/products/cases/terra/terra/te...
There is no consumer use case for that so basically inexistent market. It would be rather dumb of Nvidia (or any other for that matter) to willingly cut its entreprise margins for such a non product.
The only one doing lots of VRAM are Apple and nobody really care because performance is not that competitive and price is about as bad as entreprise stuff.
I still think modular arch will stay for consumers products there much more benefits to that than chasing a bit more perf for low volume products. Apple can get away wit their shenanigans because they subsidize their high-end chips with the iPhone. If they could only sell Macs it would extremely unsustainable (like the PowerPC venture showed).
The only one doing lots of VRAM are Apple and nobody really care because performance is not that competitive and price is about as bad as entreprise stuff.
I still think modular arch will stay for consumers products there much more benefits to that than chasing a bit more perf for low volume products. Apple can get away wit their shenanigans because they subsidize their high-end chips with the iPhone. If they could only sell Macs it would extremely unsustainable (like the PowerPC venture showed).
I would also love to see a consumer GPU with a lot of VRAM. I think 128 GB would be too expensive BUT I'm hoping the Nvidia 50 series comes with 32+ GB VRAM for at least the top tier cards of the family (today they max out at 24 GB VRAM in the 3090 and 4090.)
imagine how impossibly sold out those cards would be.
4090s are only sold out because of the China sanctions and are moving back towards MSRP. Fundamentally even a 48GB desktop card would be no substitute for a B100.
yeah but if you had 4090s with H100 equiv VRAM (prob don't even need HBM equiv) that wouldn't require quantizing larger models you would end up with a ton of LLM workstations at a fraction of what they charge for H100s. The demand would be high. What China was doing with the 4090s requires a great deal of technical/manufacturing prowess that doesn't apply to most.
There is one. Get Apple M3 with 128GB ram :)
based on what I've seen from IDC and Gartner in fact it will keep growing for another 2-3 years in insane fashion at least.
Beat of course but on a percentage basis, this is the lowest EPS and revenue beat in the last 4 quarters.
Any thoughts on why that might be?
Partly because analysts are learning their lesson in underestimating the AI demand and starting to correct for it.
The other complication is that we don't have intuitions for AI, so it's easy to underestimate how astronomically large the market will be.
And from an execution standpoint, Jensen (Nvidua CEO) has been preparing for a very long time (albeit unintentially, so nobody could predict AI to this extent). He also tends to go all-in on things, which has caused problems in the past (e.g. crypto crash), but just happened to be incredibly prescient in this moment.
The other complication is that we don't have intuitions for AI, so it's easy to underestimate how astronomically large the market will be.
And from an execution standpoint, Jensen (Nvidua CEO) has been preparing for a very long time (albeit unintentially, so nobody could predict AI to this extent). He also tends to go all-in on things, which has caused problems in the past (e.g. crypto crash), but just happened to be incredibly prescient in this moment.
So, how many GPUs are needed to replace all the white collar workers? Maybe that’s the potential of NVIDIA.
After software have eaten the world, now AI is eating everything.
After software have eaten the world, now AI is eating everything.
So far it's mostly licking everything
How high can they go? The price is inflated for sure...but their margins are absolutely insane, and their quarterly growth % is just mind-blowing.
If you look at their earnings, they are growing at 3Bish a quarter. Assuming they maintain just that, feb 2025 they should be worth as much as apple is now, since they will be making that amount per quarter.
Cisco in 2000: Gross margin in fiscal 2000 was 64.4%, compared with 65.0% in fiscal 1999. The following table shows the standard
76% margins is absolutely bonkers. At what point does the government use that as evidence in an anti trust case?
It keeps happening.
Imagine shorting Nvidia in July 2023 last year at $500 because it seemed overvalued or overextended. now $700 like nothing. It goes to show how markets are more rational than assumed , in terms of correctly pricing in future earnings. It's not irrational exuberance, but rational. The huge rally this year was in anticipation of the blow-out earnings on Ai demand, which materialized . The 1997-2000 period in which stock prices wildly departed from fundamentals was more of an anomaly than the norm, yet people assume that it's the norm. The late 90s and tech boom and early 2000s crash was an outlier that made people overly pessimistic in subsequent decades.
Nvidia is a hard to value stock though. The revenue growth, while insanely high (120% annualized quarter over quarter), is decelerating. But the net income levels are insane (55% net income margin on revenue).
This suggests huge space for competition to drive earnings growth down. But it's unclear when that will happen.
This suggests huge space for competition to drive earnings growth down. But it's unclear when that will happen.
They're producing hardware...there are physical limits to growth rate. Is growth decelerating because there's only so much chipmaking capacity available, or is it because of a demand slowdown? The net income levels suggest maybe it's the former.
Totally open to being wrong here!
Totally open to being wrong here!
There is this whole section in the 10-k about government regulation too. Some of that has to be causing growth deceleration vs what it could be.
Given what we currently know, it is hard to say this valuation is irrational or crazy as some are saying in this thread.
What is crazy is Palantir up 4% after hours because of NVDA earnings.
We are certainly selling a ton of pickaxes, pans and shovels here. Not sure about how much gold we are really finding.
Given what we currently know, it is hard to say this valuation is irrational or crazy as some are saying in this thread.
What is crazy is Palantir up 4% after hours because of NVDA earnings.
We are certainly selling a ton of pickaxes, pans and shovels here. Not sure about how much gold we are really finding.
Who would be the one to actually provide competition? AMD, Intel, someone else?
Yes.
And more you've heard about.
And yet more you haven't, since they're in stealth.
And more you've heard about.
And yet more you haven't, since they're in stealth.
I'd predict Amazon and MS. A huge portion of Nvidia's revenue is from data centers run by amazon/MS, and they both have at least some experience in designing their own chips (more on Amazon's end with Graviton). I'd expect that they are both motivated to try to design something in house that is more suited to their needs and cuts out the nvidia profit margin.
Yes, AMD (which has great GPUs, but sucks at software), Intel (that's improving rapidly on GPU side), and possibly Google with their Tensor. Also, AWS has their dedicated AI chips, aptly named Trainium and Inferentia.
Theoretically even Apple could start producing AI chips based on their Neural Engine.
The better NVIDIA's results are now the more that encourages the competitors to try and take some of the cake for themselves.
Theoretically even Apple could start producing AI chips based on their Neural Engine.
The better NVIDIA's results are now the more that encourages the competitors to try and take some of the cake for themselves.
My impression is that AMD has always had technically competitive products, but lagged in software and ecosystem and marketing. (It had a long line of dud CEOs.)
Lisa Su (Jensen's cousin, btw) seems to have turned the ship around.
Lisa Su (Jensen's cousin, btw) seems to have turned the ship around.
> Lisa Su (Jensen's cousin, btw)
TIL! Wow, that's wild.
TIL! Wow, that's wild.
they're a fast follow that competes for the middle and low end of the market. they won't produce anything that competes with NVIDIA's topend any time soon. Will take some outside competitor like LPUs or something that comes and changes the game to make a dent, that will take a while though I reckon. Hardware revolutions take years to scale.
Model inference can be done with any GPU, AMD, Nvidia, Intel. Can even be done with CPU in many cases.
Training is largely done on Nvidia cards, but there's nothing mandating that.
Google trained Gemini on their own in-house TPUs, and according to their published stats it exceeds ChatGPTs performance.
There's a collective delusion right now that somehow CUDA entitles Nvidia to a forever monopoly on GPU compute. There's no way they will maintain ~90% gross margins on their hardware sales. It's far too economically inefficient for purchasers in the long run.
The ones who figure out how to use competitor cards at less than half the cost will have a huge advantage
Right now it's fevered money pouring into what they see as the fastest way to get their feet into the game. Nvidia will do well, but that's already more than priced in right now.
Training is largely done on Nvidia cards, but there's nothing mandating that.
Google trained Gemini on their own in-house TPUs, and according to their published stats it exceeds ChatGPTs performance.
There's a collective delusion right now that somehow CUDA entitles Nvidia to a forever monopoly on GPU compute. There's no way they will maintain ~90% gross margins on their hardware sales. It's far too economically inefficient for purchasers in the long run.
The ones who figure out how to use competitor cards at less than half the cost will have a huge advantage
Right now it's fevered money pouring into what they see as the fastest way to get their feet into the game. Nvidia will do well, but that's already more than priced in right now.
Their gross margin for Q4 is 76% (was 63% a year ago).
Across all products. Margin is likely higher for the newer product lines.
But anyway, good to pull an exact figure!
But anyway, good to pull an exact figure!
When margins are above 75% everyone and their mother wants a piece of the pie.
Am I understanding this correctly, that you judge market rationality on NVidia over the period of 1 year, absolute bubble rally ?
Even with those materialized earnings, as you say, the market valuation is still at >30 times those earnings (and I'm being very generous by extrapolating this quarter earnings, not the past year).
There is a reason for the saying "the market can stay irrationnal for longer than you can stay solvent". One should not short bubbles. But it's still a bubble !
EDIT: It's a bubble in NVidia valuation. Per se, it's a very good business, generating huge amounts of $.
Even with those materialized earnings, as you say, the market valuation is still at >30 times those earnings (and I'm being very generous by extrapolating this quarter earnings, not the past year).
There is a reason for the saying "the market can stay irrationnal for longer than you can stay solvent". One should not short bubbles. But it's still a bubble !
EDIT: It's a bubble in NVidia valuation. Per se, it's a very good business, generating huge amounts of $.
There's nothing rational about this valuation. Sir Isaac Newton would like to have a word. There was a certain radio company, a switch manufacturer, a video game retailer and a car company. Money inflows dominate the price and money outflows will dominate it in the future, but will it be tomorrow, in 6 months or 3 years is a different question.