Nvidia pursues $30B custom chip opportunity with new unit(reuters.com)
reuters.com
Nvidia pursues $30B custom chip opportunity with new unit
https://www.reuters.com/technology/nvidia-chases-30-billion-custom-chip-market-with-new-unit-sources-2024-02-09/
115 comments
From Nvidia's history of working with AIBs, Sony, Apple, the Linux community, and probably many more, they seem to be a very hard company to work with. They have an idea of what the product looks like and it's their way or the highway. I wonder if this new department will change that. If it doesn't, it won't amount to much.
I don't think AIBs had a hard time working with Nvidia. If you're referring to Evga, they just wanted to exit the GPU business after the crypto boom.
For Sony, I don't recall any problems. I think Sony and Microsoft just wanted a supplier that can provide an APU and only AMD could do it at the time. AMD was also on the verge of bankruptcy so they gave Sony and Microsoft favorable terms.
For Apple, it was a case who was to blame for the GPU failures. Ultimately, I think it would have been better for Macs to use Nvidia GPUs over AMD GPUs.
Not sure about Linux. But I assume most big Nvidia server deployments run on Linux.
I personally think Nvidia did not do much custom chip is because the margins weren't there and they wanted to devote their resources to AI. Obviously they were correct in their choice. Their market cap is closing in on Apple. Let that sink it for a bit.
The only exception is the Nintendo Switch and I have a feeling Nvidia just wanted to be in at least one gaming device to say they're still in it.
For Sony, I don't recall any problems. I think Sony and Microsoft just wanted a supplier that can provide an APU and only AMD could do it at the time. AMD was also on the verge of bankruptcy so they gave Sony and Microsoft favorable terms.
For Apple, it was a case who was to blame for the GPU failures. Ultimately, I think it would have been better for Macs to use Nvidia GPUs over AMD GPUs.
Not sure about Linux. But I assume most big Nvidia server deployments run on Linux.
I personally think Nvidia did not do much custom chip is because the margins weren't there and they wanted to devote their resources to AI. Obviously they were correct in their choice. Their market cap is closing in on Apple. Let that sink it for a bit.
The only exception is the Nintendo Switch and I have a feeling Nvidia just wanted to be in at least one gaming device to say they're still in it.
I think your recollection of the Evga exit is wrong.
They already had their 4000 series boards designed. They were all set. Prototype boards have made their way into the public sphere now. Then, whilst the launch was prepping Evga [cited](https://youtube.com/watch?v=cV9QES-FUAM) that they were fed-up of working with them, and we're basically being undercut by their own supplier.
They already had their 4000 series boards designed. They were all set. Prototype boards have made their way into the public sphere now. Then, whilst the launch was prepping Evga [cited](https://youtube.com/watch?v=cV9QES-FUAM) that they were fed-up of working with them, and we're basically being undercut by their own supplier.
paulmd(1)
To be fair, it seems like all the game consoles that shipped with NV chips in them have been fairly successful, or their failures were explicitly not related to the parts of the silicon that NV designed. Other than perhaps the jailbreak issues the launch Switch had...
And Geforce mobile parts are still quite popular in laptops. It makes me wonder how much of the difficulty is "hard company to work with" and how much is just "weird constraints make integration a pain" - I still don't know if they don't want to open their drivers, or if they can't for IP reasons.
And Geforce mobile parts are still quite popular in laptops. It makes me wonder how much of the difficulty is "hard company to work with" and how much is just "weird constraints make integration a pain" - I still don't know if they don't want to open their drivers, or if they can't for IP reasons.
> all the game consoles
For Nintendo Switch. they picked it up was mostly due to Nvidia being very desperate for a win, willing to sell it for cheap, using older technology and node all while having very little driver support. ( Also I remember Jensen loves Nintendo )
> I still don't know if they don't want to open their drivers
Drivers for GPU is pretty much like CUDA for GPGPU. It is where 99% of the value comes from.
For Nintendo Switch. they picked it up was mostly due to Nvidia being very desperate for a win, willing to sell it for cheap, using older technology and node all while having very little driver support. ( Also I remember Jensen loves Nintendo )
> I still don't know if they don't want to open their drivers
Drivers for GPU is pretty much like CUDA for GPGPU. It is where 99% of the value comes from.
Just today I was reading the article about OpenAI wanting $7T to develop their own AI chips. In the comments were a bunch of people talking about all the startups in the last 18 months trying to make bespoke AI chips.
This makes a lot of sense for NVIDIA. They have the expertise, the money, the scale, and the experience already. They can probably do it cheaper than any startup and then either pass on that savings or make more profit.
This makes a lot of sense for NVIDIA. They have the expertise, the money, the scale, and the experience already. They can probably do it cheaper than any startup and then either pass on that savings or make more profit.
> either pass on that savings or make more profit.
Has Nvidia ever passed on savings? It seems to be the opposite to how they operate.
Has Nvidia ever passed on savings? It seems to be the opposite to how they operate.
GeForce Now is an incredible value for gamers. For $20/mo you get a 4080 streamed from the cloud, 6 hours at a time (and you can just relaunch after that). It has a smallish library, but the games it supports run very well. This first party service is miles ahead of their competitors, many of who also use their cards. Shadow, for example, costs 3x as much for a worse experience.
I don't really understand how they can make the economics of it work. which worries me, especially as their AI work is just printing money. I wonder if it's just a matter of time before they abandon the gaming market.
I don't really understand how they can make the economics of it work. which worries me, especially as their AI work is just printing money. I wonder if it's just a matter of time before they abandon the gaming market.
Hmmm, at $20/mo (for 6 hours at a time) then each of those 4080's is able to bring in $80/mo of revenue. That's about $1k per year.
Seems potentially reasonable if that's amortised over a few years.
Seems potentially reasonable if that's amortised over a few years.
You're not limited to 6 hours per day (though that's a lot of gaming anyway). But I doubt people are playing in optimally spread out times, i.e. it's usually a lot busier in the evenings, and on weekends and holidays. So they have to plan for bursty peaks and a lot of bandwidth that can't be cached or CDNed (it's all generated in real-time), heat management (bunch of GPUs running at full blast), virtualizing and isolating every user's Steam instance, etc. And of course all the staff it takes to manage the whole thing, test games, fix issues, etc.
The other cloud gaming providers need to do the same thing, and they either cost more and/or have worse hardware and/or have publisher relationships. Xbox and PS streaming use actual consoles and are much less powerful, for example, Shadow costs a lot more for worse hardware, Luna works with publishers, Stadia just gave up before ever really trying (lol), etc.
Maybe in Nvidia's case, at least they can get the GPUs at cost, and maybe use them for non gaming async workloads during off peak times? I dunno. I just hope they keep GFN going; it's completely replaced my desktop PC.
The other cloud gaming providers need to do the same thing, and they either cost more and/or have worse hardware and/or have publisher relationships. Xbox and PS streaming use actual consoles and are much less powerful, for example, Shadow costs a lot more for worse hardware, Luna works with publishers, Stadia just gave up before ever really trying (lol), etc.
Maybe in Nvidia's case, at least they can get the GPUs at cost, and maybe use them for non gaming async workloads during off peak times? I dunno. I just hope they keep GFN going; it's completely replaced my desktop PC.
3080, 3060 ti, maxwell...
Not sure what you're meaning there?
those are high-value products/gens where nvidia did indeed “pass along the savings”.
Turing and ampere as a whole were already below the price increase curve due to the trailing node. Ampere certainly was a massive attempt at passing the savings through - 3060 ti and 3080 were both really aggressively priced and were great overall products. Notoriously, 3060 ti was so aggressively priced that partners didn’t even want to make it at msrp…
And Turing is another example of the “AMD is subject to the same overall industry price trends/not able to drastically beat nvidia pricing either” too - sure, AMD leapt ahead to 7nm early with 5700xt… and this was more expensive than Nvidia’s solution with Turing, such that AMD was not able to undercut Nvidia’s pricing. Those cost increases have been eating up the density gains for a long time now.
Nvidia is really actually doing the things that keep the costs down, and ampere and Turing indeed bent the cost curve below where it would otherwise have been. It’s obviously not going to stay fixed, the era of $100 and $150 gaming gpus being serious contenders has obviously long since come to a close, and now it’s happening for $200-300 gpus too, $500-600 is where a $300 gpu was 10 years ago.
People just don’t have any way to objectively determine what “true” prices should be, and refuse to believe the true price is rising because it offends their “gamer” worldview. It has to be gouging, otherwise I don’t get a new Sony GameStation X every 2 years at the exact same price, and that can’t be possible.
https://www.pcgamer.com/amd-moores-law-aint-dead-its-just-a-...
Turing and ampere as a whole were already below the price increase curve due to the trailing node. Ampere certainly was a massive attempt at passing the savings through - 3060 ti and 3080 were both really aggressively priced and were great overall products. Notoriously, 3060 ti was so aggressively priced that partners didn’t even want to make it at msrp…
And Turing is another example of the “AMD is subject to the same overall industry price trends/not able to drastically beat nvidia pricing either” too - sure, AMD leapt ahead to 7nm early with 5700xt… and this was more expensive than Nvidia’s solution with Turing, such that AMD was not able to undercut Nvidia’s pricing. Those cost increases have been eating up the density gains for a long time now.
Nvidia is really actually doing the things that keep the costs down, and ampere and Turing indeed bent the cost curve below where it would otherwise have been. It’s obviously not going to stay fixed, the era of $100 and $150 gaming gpus being serious contenders has obviously long since come to a close, and now it’s happening for $200-300 gpus too, $500-600 is where a $300 gpu was 10 years ago.
People just don’t have any way to objectively determine what “true” prices should be, and refuse to believe the true price is rising because it offends their “gamer” worldview. It has to be gouging, otherwise I don’t get a new Sony GameStation X every 2 years at the exact same price, and that can’t be possible.
https://www.pcgamer.com/amd-moores-law-aint-dead-its-just-a-...
> those are high-value products/gens where nvidia did indeed “pass along the savings”.
You've got to be joking. The 3000 series cards weren't any kind of "pass along savings", and this generation after it have pricing best called "taking the piss". Widely regarded as "price gouging".
You're probably the very first person on earth to try telling people those cards are cheap (with a straight face). I don't think you're going to be able to convince anyone that you're right.
You've got to be joking. The 3000 series cards weren't any kind of "pass along savings", and this generation after it have pricing best called "taking the piss". Widely regarded as "price gouging".
You're probably the very first person on earth to try telling people those cards are cheap (with a straight face). I don't think you're going to be able to convince anyone that you're right.
if you don't think 3060 ti and 3080 and 970/980 ti were cheap products in the overall sense of the cost-trend of the market... you might be a fanboy.
I've said it before and it's true today: a lot of people don't think of themselves as fanboys, they just constantly say all the same things fanboys say and think all the things fanboys think. The ayymd discourse permeates the online discussions of all these things such that people can no longer determine what's biased and what's not.
Moreover, as I've said before, a lot of people's sense on pricing today is just miscalibrated and often they're outright hallucinating on past prices. GTX 970 was the cheaper x70 ever released - the segment has bounced between $350 and $400 since it was introduced, and $349 in 2010 was a lot more money than it is now. And people don't have a good sense of just how much costs have grown and are growing - most people don't even bother accounting for CPI in these discussions.
https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_proces...
GTX 670: $533 in 2023 dollars (cutdown of 294mm2 die)
I've said it before and it's true today: a lot of people don't think of themselves as fanboys, they just constantly say all the same things fanboys say and think all the things fanboys think. The ayymd discourse permeates the online discussions of all these things such that people can no longer determine what's biased and what's not.
Moreover, as I've said before, a lot of people's sense on pricing today is just miscalibrated and often they're outright hallucinating on past prices. GTX 970 was the cheaper x70 ever released - the segment has bounced between $350 and $400 since it was introduced, and $349 in 2010 was a lot more money than it is now. And people don't have a good sense of just how much costs have grown and are growing - most people don't even bother accounting for CPI in these discussions.
https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_proces...
GTX 670: $533 in 2023 dollars (cutdown of 294mm2 die)
No worries. You certainly have a unique perspective. ;)
Ugh. 7 trillion dollars for AI chips? Honestly that sounds divorced from reality. Why? Because it is absurdly easy to make a memory starved AI chip with too much TFLOPS.
What innovation are they going to bring to the market? 3D XPoint combined with AI engine chiplets for petabyte scale AI models? That wouldn't cost more than a hundred billion.
What innovation are they going to bring to the market? 3D XPoint combined with AI engine chiplets for petabyte scale AI models? That wouldn't cost more than a hundred billion.
$7T is more than all of TSMCs revenue ever. Last year was $69B. That was their second biggest year, and they haven't existed for 100 years yet.
It is obviously a nonsense number for headlines IMO.
The entire Abu Dhabi Investment Authority sovereign wealth fund isn't even 1 trillion.
7 belt and road initiatives?
The simple explanation is that our media is mind numbing trash that has little to do with reality other than we wouldn't be talking about this if it wasn't for the fake 7 trillion dollar number.
The entire Abu Dhabi Investment Authority sovereign wealth fund isn't even 1 trillion.
7 belt and road initiatives?
The simple explanation is that our media is mind numbing trash that has little to do with reality other than we wouldn't be talking about this if it wasn't for the fake 7 trillion dollar number.
The problem is raising a huge amount of money, and this is the solution, which is something Sam is very good at. The utility seems purely financial.
The only interesting development in "bespoke" AI chips are the ones that put human neurons on a chip (feel free to get the "you want Skynet?..." and Jurassic Park comments out of your system because they was my first thoughts, too). Those have very different characteristics than just running attention algorithms better. All the others seem to be grasping at marginal improvements for training and inference.
There's a company out there actually doing this: https://corticallabs.com/
There's a company out there actually doing this: https://corticallabs.com/
Don't forget the tooling, ROCm still hasn't taken off very well.
ROCm runs PyTorch and TensorFlow. It seems to have more or less caught up on the technical capability front.
There are outstanding problems, particularly I've found it very crash prone on a consumer desktop and wouldn't recommend an AMD card for research compute tasks where you are also running an X server using the same card. But there aren't $30 billion opportunities for custom chips on the consumer desktop right now - I'm guessing these will be for SaaS businesses where AMD are focusing. IE, it won't matter that they can't X.org and multiply matrices at the same time because servers won't use the cards for graphics.
There are outstanding problems, particularly I've found it very crash prone on a consumer desktop and wouldn't recommend an AMD card for research compute tasks where you are also running an X server using the same card. But there aren't $30 billion opportunities for custom chips on the consumer desktop right now - I'm guessing these will be for SaaS businesses where AMD are focusing. IE, it won't matter that they can't X.org and multiply matrices at the same time because servers won't use the cards for graphics.
People don't seem to understand that running neural network inference is very easy. It's not the machine learning frameworks and libraries that are difficult to get right. Those are the trivial part.
The hard part is getting a culture that gives a damn about developing software that works and designing the hardware to support the features that the software needs.
AMD has not figured out how to run both graphics and compute on the same GPU. There can be many reasons for that, but honestly it is probably because they either don't have the necessary virtualization hardware or because two different drivers are conflicting with one another.
The hard part is getting a culture that gives a damn about developing software that works and designing the hardware to support the features that the software needs.
AMD has not figured out how to run both graphics and compute on the same GPU. There can be many reasons for that, but honestly it is probably because they either don't have the necessary virtualization hardware or because two different drivers are conflicting with one another.
> The hard part is getting a culture that gives a damn about developing software that works and designing the hardware to support the features that the software needs.
NVIDIA isn't missing the mark on the programming model and toolkit framework (PTX and forward/backward compat) either. They have a good, lean gpu design with a lot of features and a good programming model and ecosystem etc.
You're right, it's not just the matrix math, that's not rocket science, but there's a ton of little glue code around it. And you need something GPU-like for that anyway, plus a bunch of scheduler and shader-execution-reordering stuff for your tensor threads and glue code, etc. You end up with something broadly similar to a GPU anyway.
It's the ProgPOW theorem, right? That there is not some major gain to be squeezed by implementing a smaller/different machine on the instruction set. That GPUs are relatively close to some kind of computational optimum for parallel workloads (in terms of programmability/flexibility and performance).
NVIDIA's model isn't far off the global optimum imo, it's certainly in a great local minimum, and that's really true of a lot of their designs these days. It is always a little wild how everyone trivializes the idea that AMD/etc are going to catch up with some 80% solution in RT or tensor etc... like just maybe NVIDIA did the math and figured out what they think a reasonable ray performance level is, and how much they'd need to upscale, and what parts of the pipeline make sense to have accelerated by units vs emulated on shaders/etc, and there's not some massive gain to be squeezed by just putting a handful of devs on a project for a year?
Same thing for prices too. Everyone wants to assume that AMD is just choosing to follow them in gouging or whatever. The null hypothesis is that both nvidia and AMD are subject to the same industry cost trends and can’t actually do significantly better (not like 2x perf/$ or whatever), and that nvidia is in some kind of reasonable price structure after all. People are going to find that a lot of electronics prices are going to go up in the coming years. There’s no more 1600AF for $85 or 3600 for $160 either, or Radeon 7850 for $150 etc.
Not talking about original 4080 pricing etc but actually 4070 and 4060 are fairly reasonable products, and 4070 quickly fell even further below msrp. 7800xt and 7900xt and 7600xt are all fine as well. That’s about what the price increases have been since the last leasing-edge products.
NVIDIA isn't missing the mark on the programming model and toolkit framework (PTX and forward/backward compat) either. They have a good, lean gpu design with a lot of features and a good programming model and ecosystem etc.
You're right, it's not just the matrix math, that's not rocket science, but there's a ton of little glue code around it. And you need something GPU-like for that anyway, plus a bunch of scheduler and shader-execution-reordering stuff for your tensor threads and glue code, etc. You end up with something broadly similar to a GPU anyway.
It's the ProgPOW theorem, right? That there is not some major gain to be squeezed by implementing a smaller/different machine on the instruction set. That GPUs are relatively close to some kind of computational optimum for parallel workloads (in terms of programmability/flexibility and performance).
NVIDIA's model isn't far off the global optimum imo, it's certainly in a great local minimum, and that's really true of a lot of their designs these days. It is always a little wild how everyone trivializes the idea that AMD/etc are going to catch up with some 80% solution in RT or tensor etc... like just maybe NVIDIA did the math and figured out what they think a reasonable ray performance level is, and how much they'd need to upscale, and what parts of the pipeline make sense to have accelerated by units vs emulated on shaders/etc, and there's not some massive gain to be squeezed by just putting a handful of devs on a project for a year?
Same thing for prices too. Everyone wants to assume that AMD is just choosing to follow them in gouging or whatever. The null hypothesis is that both nvidia and AMD are subject to the same industry cost trends and can’t actually do significantly better (not like 2x perf/$ or whatever), and that nvidia is in some kind of reasonable price structure after all. People are going to find that a lot of electronics prices are going to go up in the coming years. There’s no more 1600AF for $85 or 3600 for $160 either, or Radeon 7850 for $150 etc.
Not talking about original 4080 pricing etc but actually 4070 and 4060 are fairly reasonable products, and 4070 quickly fell even further below msrp. 7800xt and 7900xt and 7600xt are all fine as well. That’s about what the price increases have been since the last leasing-edge products.
CUDA runs on essentially all NVIDIA consumer gpus. ROCm is supported on a single 5 year old model and two 2 year old models.
https://rocm.docs.amd.com/projects/install-on-linux/en/docs-...
https://rocm.docs.amd.com/projects/install-on-linux/en/docs-...
The people who write their documentation aren't very good. "Support" seems to mean something like tested + enterprise grade support. It works on a fair number of officially unsupported cards. Eventually it'll work well on everything even though the support matrix is unlikely to ever get bigger.
Although, as mentioned, works under a cycloptic vision where the card is only doing compute tasks. I'd be interested to know if even supported cards can multitask GPU and pure compute tasks without crashing because it looks like it might be a design issue. Hard to tell with driver corruption. Maybe the testing catches that on supported cards; who knows.
Although, as mentioned, works under a cycloptic vision where the card is only doing compute tasks. I'd be interested to know if even supported cards can multitask GPU and pure compute tasks without crashing because it looks like it might be a design issue. Hard to tell with driver corruption. Maybe the testing catches that on supported cards; who knows.
The Mesa folks are working on the tooling situation with RustiCL, which has potential to support SYCL too in the future, and quite possibly HIP. Not just for ROCm-supported devices too, but across the board (subject to pure hardware constraints).
Introducing that amount of capital would be extremely disruptive for the rest of the market - was it really 7 trillion, or billion?
Yes...They are pulling what is called a Henry Paulson
"“It’s not based on any particular data point,” a Treasury spokeswoman told Forbes.com Tuesday. “We just wanted to choose a really large number...”" - https://archive.thinkprogress.org/treasury-explains-how-it-c...
"“It’s not based on any particular data point,” a Treasury spokeswoman told Forbes.com Tuesday. “We just wanted to choose a really large number...”" - https://archive.thinkprogress.org/treasury-explains-how-it-c...
Yes, they really mean trillion. I expect it would be the most resources anyone has ever invested into any single goal, even accounting for inflation.
That makes me wonder the entire global cost of things like world war 2. Anyone have relevant links? I can't find any.
Estimates range from about 1 T$ [0] to 4 T$ [1] or 5 T$ [2].
[0] https://www.britannica.com/event/World-War-II/Human-and-mate...
[1] https://moneywise.com/life/lifestyle/financial-facts-about-w...
[2] https://www.whatitcosts.com/world-war-ii-cost-united-states-...
[0] https://www.britannica.com/event/World-War-II/Human-and-mate...
[1] https://moneywise.com/life/lifestyle/financial-facts-about-w...
[2] https://www.whatitcosts.com/world-war-ii-cost-united-states-...
That involved massive movement of people, weapons, and vehicles, let alone factories and resources.
Chip making is expensive, but what do you do with 7T? Buy out every engineer on the planet with half of it, and have them work on problems? Does Altman think he’s Oppenheimer?!
Chip making is expensive, but what do you do with 7T? Buy out every engineer on the planet with half of it, and have them work on problems? Does Altman think he’s Oppenheimer?!
every custom chip sold is another NVidia H100 not bought
That’s a bit like “every Honda Accord sold is another Maserati not bought”. Providing a cheaper option can net more profit on volume.
Or the other way around:
Every bespoke chip will be much more expensive - and profitable - than the generic units that were not bought.
Every bespoke chip will be much more expensive - and profitable - than the generic units that were not bought.
Not necessarily.
Custom chips are likely made in technology nodes step behind They are cheaper to manufacture. Nvidia's H200 and Apple M2 are so profitable that they get the latest technology nodes.
Custom chips are likely made in technology nodes step behind They are cheaper to manufacture. Nvidia's H200 and Apple M2 are so profitable that they get the latest technology nodes.
I would love to see a consumer graphics card with 128GB VRAM
Would be nice to be able to work with some of the larger open source LLM models.
Would be nice to be able to work with some of the larger open source LLM models.
Unfortunately, as soon as you make a card with specifications that make it great at enterprise-grade tasks, it will be bought in mass quantities by people building out data centers. This pushes the price up, as we’ve already seen.
So labeling it “consumer” doesn’t really mean much. They’ve tried to enforce the distinction with EULAs before, but that doesn’t work well.
So labeling it “consumer” doesn’t really mean much. They’ve tried to enforce the distinction with EULAs before, but that doesn’t work well.
> it will be bought in mass quantities by people building out data centers.
Isn't this good? The production at scale effects will kick in, lowering the price and supply will meet demand after some hiccups.
Isn't this good? The production at scale effects will kick in, lowering the price and supply will meet demand after some hiccups.
Nvidia doesn't want the price to be lower; that's why they won't make this card.
If demand drove prices down like that then we’d already have cheap cards available.
Demand puts upward pressure on prices.
Supply is already maxed out and growing as fast as possible.
Demand puts upward pressure on prices.
Supply is already maxed out and growing as fast as possible.
> Supply is already maxed out and growing as fast as possible.
No, it's not anywhere near maxed out anymore. The chip shortage has been over for a while; now there is too much supply.
But that's not the issue. The issue is that Nvidia really wants to force these enterprise customers to pay extremely high prices. This is why they are so afraid to give any more VRAM to the consumer chips, because if they were at all suitable for VRAM-heavy workloads, every HPC company would buy a couple $2,000 consumer cards in place of each $15,000 datacenter card. That'd lose Nvidia something like 70% which they would find entirely unacceptable.
No, it's not anywhere near maxed out anymore. The chip shortage has been over for a while; now there is too much supply.
But that's not the issue. The issue is that Nvidia really wants to force these enterprise customers to pay extremely high prices. This is why they are so afraid to give any more VRAM to the consumer chips, because if they were at all suitable for VRAM-heavy workloads, every HPC company would buy a couple $2,000 consumer cards in place of each $15,000 datacenter card. That'd lose Nvidia something like 70% which they would find entirely unacceptable.
> chip shortage has been over for a while; now there is too much supply.
Is that at 10-40nm or at cutting edge 3-5nm?
If there is a surplus supply, why aren't they making more H800, there demand is more than supply.
Is that at 10-40nm or at cutting edge 3-5nm?
If there is a surplus supply, why aren't they making more H800, there demand is more than supply.
By "too much supply" I mean that once the chip shortage ended, they tried to take advantage of that and produced far too many 30-series cards. When the mining crash happened, they went surprised pikachu face because then they couldn't easily sell off the rest of their inventory.
Not sure if they still have surplus.
Not sure if they still have surplus.
> Isn't this good? The production at scale effects will kick in, lowering the price and supply will meet demand after some hiccups.
You didn't account for the "hiccups", which can vary from 5-20 years until competition catches up, longer than the life of many companies. In spherical cows worlds of economics that would be just a hiccup.
You didn't account for the "hiccups", which can vary from 5-20 years until competition catches up, longer than the life of many companies. In spherical cows worlds of economics that would be just a hiccup.
Not exactly what you've asked but Mac Studio exists, with 192GB at that.
Calling a Mac Studio equipped with 192GB of RAM "consumer grade" is a big stretch.
"Consumer ${thing}" to me is somewhere =< $3000.
"Consumer ${thing}" to me is somewhere =< $3000.
True, but you get a full general compute machine for under $6k, after a few years you can sell it for 3-4k and upgrade to the latest one. Personally I think it's the best thing to happen to local LLMs, by accident. I have an M1 Max 64GB and I'm blown away every day by these local models. Apple didn't plan for it, but it happened by accident due to unified memory with godzillion throughput having integrated silicon.
This isn't possible currently unless you use HBM, which is significantly more expensive both to actually buy the HBM, and to package it (and all the capacity for packaging HBM interposers is already used for data center hardware).
The most VRAM you could have on a consumer GPU today is 48GB, and that'd be on a 4090 or 7900xtx with clamshell VRAM (which increases cost and makes cooling significantly harder due to putting GDDR6 chips on the back of the GPU where there aren't any fans).
To calculate how much VRAM is possible, you just need to divide the GPU's bus width by 32 (or 64 for Samsung's new weird double capacity but double bit width) and multiply that by the largest GDDR capacity currently available (16Gbit).
As for why GPUs don't increase their bus width, there have been GPUs with 512bit busses in the past, but it makes it quite a bit more expensive (more vram chips, more traces to run, might require more/heavier PCB layers) and increases power draw.
The most VRAM you could have on a consumer GPU today is 48GB, and that'd be on a 4090 or 7900xtx with clamshell VRAM (which increases cost and makes cooling significantly harder due to putting GDDR6 chips on the back of the GPU where there aren't any fans).
To calculate how much VRAM is possible, you just need to divide the GPU's bus width by 32 (or 64 for Samsung's new weird double capacity but double bit width) and multiply that by the largest GDDR capacity currently available (16Gbit).
As for why GPUs don't increase their bus width, there have been GPUs with 512bit busses in the past, but it makes it quite a bit more expensive (more vram chips, more traces to run, might require more/heavier PCB layers) and increases power draw.
I think that goal is fine and good, but I would rather see huge investments toward in-memory compute like ReRAM and such. If we bridge the efficiency advancements of TinyML with the leap of LLM abilities, perhaps we can start on the road of not being limited by the impact of training on climate.
Agree with the comment, but riffing on the last few words:
Climate is pretty much my #1 concern about the world, but LLM use of energy is really really far down on the list of important actions for climate.
First and foremost are removing roadblocks for deploying existing technologies for clean energy, and speeding up the necessary supporting infrastructure such as transmission and market policies for choosing cheapest possible solutions (over the objections of dinosaur execs that choose last century's solutions). Then the big hard to decarbonize parts of industry like cement and steel, as well as deploying electrolyzers to get ammonia fertilizer production switched over to carbon neutral production rather than from fossil-generated hydrogen.
Reducing energy consumption is important for advancing AI in general, but ultimately all its energy consumption will be from clean energy sources anyway, and the switch that needs to happen is that switch in energy sources. Reducing energy use by 2x or 10x is not good enough, we must change the sources fundamentally.
Climate is pretty much my #1 concern about the world, but LLM use of energy is really really far down on the list of important actions for climate.
First and foremost are removing roadblocks for deploying existing technologies for clean energy, and speeding up the necessary supporting infrastructure such as transmission and market policies for choosing cheapest possible solutions (over the objections of dinosaur execs that choose last century's solutions). Then the big hard to decarbonize parts of industry like cement and steel, as well as deploying electrolyzers to get ammonia fertilizer production switched over to carbon neutral production rather than from fossil-generated hydrogen.
Reducing energy consumption is important for advancing AI in general, but ultimately all its energy consumption will be from clean energy sources anyway, and the switch that needs to happen is that switch in energy sources. Reducing energy use by 2x or 10x is not good enough, we must change the sources fundamentally.
Why the fuck was this downvoted.
Very occasionally I get the feeling HN is entering the /. phase
Very occasionally I get the feeling HN is entering the /. phase
I didn't downvote it but in-memory compute is crackpot and alternative memory tech is really crackpot. It's not going to happen and it's ridiculous to propose it on the same level as GPUs with more RAM.
Can you explain further why you believe it isn't worth exploring? Is it that you can't imagine how it would scale to the level of today's high performance compute hardware?
Just ten years back, squeezing ML models onto microcontrollers sounded completely insane, given their tight memory and power constraints. We've seen NN compilers developed, game-changing techniques like quantization, pruning, and graph-level optimizations pruning. This allowed deployment of ML models in microcontrollers with a newly developed framework like TFLite Micro.
Just ten years back, squeezing ML models onto microcontrollers sounded completely insane, given their tight memory and power constraints. We've seen NN compilers developed, game-changing techniques like quantization, pruning, and graph-level optimizations pruning. This allowed deployment of ML models in microcontrollers with a newly developed framework like TFLite Micro.
These technologies have already been explored and they failed every time. It's throwing good money after bad.
Also, speculative basic research isn't comparable to adding more RAM to a graphics card. You can't substitute one for the other.
Also, speculative basic research isn't comparable to adding more RAM to a graphics card. You can't substitute one for the other.
GPU's are already closer to "in-memory compute" compared to CPU's. It's just taking the existing pattern of NUMA (non-uniform memory access) to a greater extent, as a principled approach to the so-called 'Von Neumann bottleneck'.
In-memory compute is very easy, you just don't have to fall for the pipe dream of using the same process for both the memory and the compute.
All you have to do is follow a package on package strategy like we already do with smartphones. A Raspberry PI 5 gets 25GB/s memory bandwidth and it only has a single DRAM chip if I recall correctly.
So if you had a DIMM with 16 of these chips, you would already be on the same bandwidth as HBM. 96 DIMMs and you get 40TB/s memory bandwidth.
So if you had a DIMM with 16 of these chips, you would already be on the same bandwidth as HBM. 96 DIMMs and you get 40TB/s memory bandwidth.
[deleted]
As a side effect we are seeing lot of investment/innovation in 2-13B models. Considering price of 4090, 128GB GPU will be >6000 which practically no one can afford.
We are getting that (in early 2025?) With AMD Strix Halo.
40 CUs, 256 bit LPDDR5X, 16 CPU cores. Or so the rumors say.
40 CUs, 256 bit LPDDR5X, 16 CPU cores. Or so the rumors say.
That's overkill for any type of graphics application though.
What you want is a DL or parallel compute card, not a graphics card.
They are far more expensive though because compute doesn't sell to the average consumer like graphics does.
What you want is a DL or parallel compute card, not a graphics card.
They are far more expensive though because compute doesn't sell to the average consumer like graphics does.
A modern graphics card already is a parallel compute card. A modern graphics pipeline is mostly compute with only very specific stuff using fixed function functionality.
Yes, but that was not the point. The point is that they are sold as graphics cards, not compute cards. Therefore, expect these cards to be good at (and have the memory for) typical graphics operations.
I don't entirely agree, especially with the popularity of LLMs nowadays. While graphics are still a big portion of the pitch, compute oriented features are increasingly expected as part of the offering, especially at the higher end of the lineup. If you've bought a 3090/4090, you likely had in mind more computationally oriented tasks compared to if you've bought a 1660 or AMD card.
Are the ray tracing cores in Nvidia cards used for compute?
All that the ray tracing cores do is accelerate calculation of BVH intersections. The rest of the tracing (ie the part that gives meaning to the BVH intersection result) is done via compute, with the RT cores being able to be 'invoked' essentially arbitrarily. This is in contrast to the way graphics cards used to be, where you had certain stages for certain things, designed specifically around producing graphics, making the 'graphics' part of the name hold more weight than nowadays.
I'm not sure if you're asking if they're used for a compute application or if you're asking if they can be used from the same execution path as 'compute' code.
I'm not sure if you're asking if they're used for a compute application or if you're asking if they can be used from the same execution path as 'compute' code.
Seen from first principles, BVH is quite close to some kinds of hierarchical clustering. The ray intersection part can be understood in turn as a kind of normalization, or projection to the unit sphere. So depending on how much hardware is actually reserved to this task on-chip, there might be ways of making it somewhat useful in the ML context.
Just asking if those cores tend to sit unused when "standard" compute jobs are run on the cards. :)
Meh, that's like saying 640kb of RAM is more than anyone would ever need. Demand follows hardware development, which in turn accelerates demand. I'm sure game developers would easily find a way to use 128GB of VRAM if it was commonly available in their target market.
This isn't an argument against giving users this much VRAM, but I'm pretty sure they'd just drop the whole game into VRAM and call it a day instead of actually optimizing.
cough Apple Silicon cough cough
Unified memory is honestly a really, really good idea.
Supposedly, if you have professional Apple hardware experts hand-optimize a machine-learning model, then Apple's most powerful M3 chips can compete against, say, a typical/unoptimized PyTorch implementation of the same model on a 4090.
Except the M-series chips can currently have up to 8 times the memory of the 4090, and they don't have to worry about shuffling things in and out of VRAM. The GPU can just work with any of the data that the CPU already has.
Unified memory is honestly a really, really good idea.
Supposedly, if you have professional Apple hardware experts hand-optimize a machine-learning model, then Apple's most powerful M3 chips can compete against, say, a typical/unoptimized PyTorch implementation of the same model on a 4090.
Except the M-series chips can currently have up to 8 times the memory of the 4090, and they don't have to worry about shuffling things in and out of VRAM. The GPU can just work with any of the data that the CPU already has.
I'd be hard pressed to buy a comparison to a 4090. But yeah, things would be better if they didn't "dump everything into VRAM"
They're comparing incredibly hand-optimized MLX inference on the Mac to off-the-shelf stuff on the 4090. So I'd believe it just because the 4090's being given an artificial disadvantage, and the Mac is being given meticulous hand-optimization by the foremost experts on the very platform it runs on.
In other words, the Mac is being given a huge optimization advantage compared to the 4090. And it's roughly competitive.
Even that is no mean feat.
Source: https://appleinsider.com/articles/23/12/13/apple-silicon-m3-...
In other words, the Mac is being given a huge optimization advantage compared to the 4090. And it's roughly competitive.
Even that is no mean feat.
Source: https://appleinsider.com/articles/23/12/13/apple-silicon-m3-...
Isn't that exactly how things improve? When you can reduce the cognitive load on bookkeeping, you can accomplish more productive work.
I can produce a featureful webapp in Python-Django solo because I do not have to worry about optimally filling registers and CPU cache. As you cut deeper to the hardware limitations, you have to be significantly more cognizant of writing to the hardware constraints than solving the business problem.
Taken to the extreme, we have Electron applications consuming multi-GB of RAM, but it does expand the universe of possibilities.
I can produce a featureful webapp in Python-Django solo because I do not have to worry about optimally filling registers and CPU cache. As you cut deeper to the hardware limitations, you have to be significantly more cognizant of writing to the hardware constraints than solving the business problem.
Taken to the extreme, we have Electron applications consuming multi-GB of RAM, but it does expand the universe of possibilities.
> Isn't that exactly how things improve?
There's a joke "20 years of software progress has completely undone the 20 years of hardware progress."
Essentially the joke is about how people don't optimize software like they used to. There is some tongue in cheek though because there are more platforms and variations that we have to account for these days but there is also truth that there is some laziness. I mean I have a laundry app on my phone that takes 5s to load each time because it redraws the home screen while it is trying to connect to the network and find the machines but won't use the bluetooth or nfc chips that both my phone and the machines have.
I do think there is the case to move fast and break things, but at some point you got to slow down and fix things. There should be cycles. Don't forget that debt has interest and that interest compounds. If you have enough debt you'll be "moving fast" but being fast in molasses isn't fast. It's kinda why a lot of software have just declared bankruptcy and started over (e.g. "rewriting in rust"). We should definitely encourage people to hire a mixture of those that move fast and those that move slow. There's harmony there but I think we've forgotten that.
There's a joke "20 years of software progress has completely undone the 20 years of hardware progress."
Essentially the joke is about how people don't optimize software like they used to. There is some tongue in cheek though because there are more platforms and variations that we have to account for these days but there is also truth that there is some laziness. I mean I have a laundry app on my phone that takes 5s to load each time because it redraws the home screen while it is trying to connect to the network and find the machines but won't use the bluetooth or nfc chips that both my phone and the machines have.
I do think there is the case to move fast and break things, but at some point you got to slow down and fix things. There should be cycles. Don't forget that debt has interest and that interest compounds. If you have enough debt you'll be "moving fast" but being fast in molasses isn't fast. It's kinda why a lot of software have just declared bankruptcy and started over (e.g. "rewriting in rust"). We should definitely encourage people to hire a mixture of those that move fast and those that move slow. There's harmony there but I think we've forgotten that.
> Isn't that exactly how things improve? When you can reduce the cognitive load on bookkeeping, you can accomplish more productive work.
What are you talking about? Sure you can appear to be more productive if you are reckless and wasteful of resources, and adding more resources means there is more that you can afford to waste.
But for the past decade or so we have been seeing what happens if you teach every software developer that it's okay to waste resources: resources get wasted.
Back when you couldn't afford to waste resources, you'd see extremely skilled optimizations and extremely clever hacks to get advanced software to run on extremely slow and primitive CPUs. But ever since computing power increased by a couple orders of magnitude, what happens? The exact same software implemented by today's developers would more than compensate for the increased computing power by being a couple orders of magnitude less efficient.
Sure it expands the universe of possibilities. I do want more resources, I really do. But "reducing the cognitive load on bookkeeping" is making software get slower faster than hardware gets more powerful. Which is absolute insanity to me.
What are you talking about? Sure you can appear to be more productive if you are reckless and wasteful of resources, and adding more resources means there is more that you can afford to waste.
But for the past decade or so we have been seeing what happens if you teach every software developer that it's okay to waste resources: resources get wasted.
Back when you couldn't afford to waste resources, you'd see extremely skilled optimizations and extremely clever hacks to get advanced software to run on extremely slow and primitive CPUs. But ever since computing power increased by a couple orders of magnitude, what happens? The exact same software implemented by today's developers would more than compensate for the increased computing power by being a couple orders of magnitude less efficient.
Sure it expands the universe of possibilities. I do want more resources, I really do. But "reducing the cognitive load on bookkeeping" is making software get slower faster than hardware gets more powerful. Which is absolute insanity to me.
Think of all the software which does not exist if we are still writing in assembly because we are constrained by the hardware. Python, RPGMaker, Roblox, PICO-8, etc are no longer possible because they are too wasteful. Having bounty enables art to be made which is impossible if creators are bit twiddling.
Do I bemoan that my 2000s era super computer squanders resources to load 20 kb of text? Sure, but it is also amazing the things that are possible because tools are not restricted to the priesthood of assembly programmers.
Do I bemoan that my 2000s era super computer squanders resources to load 20 kb of text? Sure, but it is also amazing the things that are possible because tools are not restricted to the priesthood of assembly programmers.
> Think of all the software which does not exist if we are still writing in assembly because we are constrained by the hardware.
There is an entire spectrum between "640k ought to be enough for anybody" and "this app needs its own fully independent copy of an entire browser engine just to display a couple screens of text".
About Electron: Operating systems provide their own system-level webviews. Use those. Tauri uses those; Tauri apps are absolutely tiny and use very little RAM, because the system-level webview doesn't have to duplicate the overhead of an entire browser engine for every single app that uses it. It typically only has to duplicate the overhead of a single page.
But that is for apps that even need a webview. For apps that don't, you don't have to drop all the way down to assembly to pick something like WinForms. Even WinUI 3 isn't always terrible, at least compared to webviews.
Hell, I wonder if even Python + wxPython would be more efficient than using a webview. Honestly, none of the software you mentioned is "too wasteful", or at least those aren't what I was referring to with my original comment.
Have you heard about Microsoft Teams, and how it can take up to 10 seconds just to display the splash screen, even on a decently fast computer? I'd like to see you try to tell me that it would have literally killed them to optimize it a bit more. Just think back to, say, the IRC clients from back in the Windows 7 days where you wouldn't even have to blink before they finished opening up.
There is an entire spectrum between "640k ought to be enough for anybody" and "this app needs its own fully independent copy of an entire browser engine just to display a couple screens of text".
About Electron: Operating systems provide their own system-level webviews. Use those. Tauri uses those; Tauri apps are absolutely tiny and use very little RAM, because the system-level webview doesn't have to duplicate the overhead of an entire browser engine for every single app that uses it. It typically only has to duplicate the overhead of a single page.
But that is for apps that even need a webview. For apps that don't, you don't have to drop all the way down to assembly to pick something like WinForms. Even WinUI 3 isn't always terrible, at least compared to webviews.
Hell, I wonder if even Python + wxPython would be more efficient than using a webview. Honestly, none of the software you mentioned is "too wasteful", or at least those aren't what I was referring to with my original comment.
Have you heard about Microsoft Teams, and how it can take up to 10 seconds just to display the splash screen, even on a decently fast computer? I'd like to see you try to tell me that it would have literally killed them to optimize it a bit more. Just think back to, say, the IRC clients from back in the Windows 7 days where you wouldn't even have to blink before they finished opening up.
Another example is just how few simple optimizations I see in code now. Something like
The real issue comes when the user is doing things more than just your program...
`while i < someFunction():`
```
criteria = someFunction()
while i < criteria:
```
The real issue comes when the user is doing things more than just your program...
I have this theory that if you remove too much friction then things get worse. Because when there's friction it makes people cognizant of limitations. But when there's not enough friction it becomes easy to sweep aside or ignore. You can be too reliant on your metrics. Idk if this actually makes sense but maybe there's a nugget somewhere in the idea.
Turns out that demand for graphics memory got stuck at a point where compute is still hungry for more.
That may certainly change but it doesn't help compute much, today.
That may certainly change but it doesn't help compute much, today.
It's supply that's down - Nvidia is strictly enforcing their artificial segmentation where the jump from 24GB to 40GB of VRAM multiplies the price by around 5 (!).
They know that if you need a 40GB card, you probably can't use a 24GB card at all because it'd run out of VRAM. They charge such insane prices for those higher capacity cards, because they know that they can make you pay thousands upon thousands for every tiny scrap of extra VRAM, and you will have literally no choice but to pay that price because you need the VRAM.
Demand isn't low, demand is actually so high that they don't care if lowly consumers want VRAM, they already have enterprise customers that need it so badly that they'll pay prices higher than most consumers would ever imagine.
And that's why they're so stingy with VRAM on the consumer cards. They need to be careful not to accidentally make them useful in the datacenter, so they can ensure that their enterprise customers continue to be forced to pay those extortionary prices.
You see, it's all about the money, and it always will be. Capitalism, baby!
They know that if you need a 40GB card, you probably can't use a 24GB card at all because it'd run out of VRAM. They charge such insane prices for those higher capacity cards, because they know that they can make you pay thousands upon thousands for every tiny scrap of extra VRAM, and you will have literally no choice but to pay that price because you need the VRAM.
Demand isn't low, demand is actually so high that they don't care if lowly consumers want VRAM, they already have enterprise customers that need it so badly that they'll pay prices higher than most consumers would ever imagine.
And that's why they're so stingy with VRAM on the consumer cards. They need to be careful not to accidentally make them useful in the datacenter, so they can ensure that their enterprise customers continue to be forced to pay those extortionary prices.
You see, it's all about the money, and it always will be. Capitalism, baby!
You need a special license to run nVidia cards in the datacenter, so this rules out consumer gaming cards already.
There are already ways to get around this. For example, renting compute from people who aren't in datacenters. Which is already a thing: https://vast.ai
To be fair, 640kb is more than anyone really needs. It's just far, far less than we want.
Can you imagine the download sizes?
I would assume instead of Google and Microsoft designing their own ARM Processor on the server using ARM's IP with custom design. Nvidia could do that for them. After all Microsoft and Google dont have the economy of scale, nor do they have the ( or as much ) expertise. Nvidia could also provide other IPs such as Network, Ethernet and other GPGPU integration. I guess that is why they are in talks with Ericsson. Ericsson has a history of working with Intel and Intel failed to deliver.
Basically I think Nvidia will now move all the sunk cost of New Node to every new generation of AI chip. While the rest of the business unit, from GPU, Network, SoC, and now Custom Chips benefits from it. I am even wondering if Nvidia will come back to Smartphone Mobile SoC.
Basically I think Nvidia will now move all the sunk cost of New Node to every new generation of AI chip. While the rest of the business unit, from GPU, Network, SoC, and now Custom Chips benefits from it. I am even wondering if Nvidia will come back to Smartphone Mobile SoC.
I wonder if customers really want custom chips or just cheaper ones. Many of these custom AI chips are slower than flagship GPUs so presumably a cut-down GPU at a lower price would be just as good.
There is a lot of silicon consumer GPUs don't "need" for AI.
But on the other hand the software stack is very mature and they are heavily amortized by the huge volume, so its kinda hard to argue with. In fact its so good that Nvidia can charge outrageous prices for the L40, A10 and such and then turn around and sell the exact same dies to consumers (with less memory).
But on the other hand the software stack is very mature and they are heavily amortized by the huge volume, so its kinda hard to argue with. In fact its so good that Nvidia can charge outrageous prices for the L40, A10 and such and then turn around and sell the exact same dies to consumers (with less memory).
For customers like Ericsson it wouldn’t surprise me if they request special instructions and special hard function blocks. In telecom there are certain operations that’s specified by the standard (and some which aren’t but used as a de facto standard) which are performed so often that you want to do them in hardware instead of in software. Or the opposite, Ericsson just wants to integrate NVIDIAs IP into Ericsson’s own ASICs instead of using their own cores and other third party cores.
I imagine there will be cheaper service providers soon for training (2024/2025). Like what companies such as Hetzner, Digital Ocean and others are providing for cloud. They are not in the same league of AWS, Google Cloud, Azure but can add more specific cloud services.
AWS/Azure prices are really awful TBH. There are already much better places to get GPUs.
I know but, for example, Google Cloud has a current advantage with their own hardware (TPUs). What is approximately the cost of training something like ChatGPT or Gemini? They have an advantage because they can rely in Azure and Google respectively without paying anything or with subsidised prices. Could a new player compete with them for training for other companies?
Google prices the TPUs pretty exorbitantly, actually.
But they give a lot of TPU time away for research, which is nice.
It seems Intel Gaudi 2 is priced in a sweeter spot, but I've never head of anyone but Intel using them.
But they give a lot of TPU time away for research, which is nice.
It seems Intel Gaudi 2 is priced in a sweeter spot, but I've never head of anyone but Intel using them.
Then I understand their advantage is training their own models and pricing it high for others no matter the cost.
Nvidia isn't in the Fab biz, so maybe this will be easier for them to generate customer interest in a way that Intel has not been able to?
I read about a new approach for making AI chips sometime last year - analogue chips by this company - https://mythic.ai/
Haven't heard anything about it since though.
Haven't heard anything about it since though.
I wonder if we are somehow at peak GPU profitability? It seems that either efficiencies in AI or competition will emerge.
Diminishing returns on more pixels is also a factor.
4k vs 1440 is 2.4x the pixels (and related compute/heat/energy), but a barely-visible difference in visual clarity.
We can see successful consoles like the Switch that have given up on competing on resolution already.
4k vs 1440 is 2.4x the pixels (and related compute/heat/energy), but a barely-visible difference in visual clarity.
We can see successful consoles like the Switch that have given up on competing on resolution already.
Imagine if they got ARM, sort of good they did not as the competition would suffer
Is TSMC the exclusive manufacturer for this unit? I can’t find the info.
it wouldn't be uncommon for the fab not to be announced or not announced up front. it's not really a consumer-facing spec, and it's not NVIDIA's announcement to make.
samsung and intel foundry services are both serious possibilities imo, NVIDIA is incredibly portable across nodes and will take advantage of anything that's cheap and makes sense as a product.
for example pascal used samsung 14LPP for the 1050 and 1050 Ti, and the A100 was taped out on TSMC 7nm (not samsung 8nm), etc. They are actually quite diversified, they have a product foothold on almost every node that matters, if Samsung suddenly becomes a blazing deal they're ready to go. Etc.
they also already signed a semicustom licensing deal with Mediatek last year, this is part of an overall trend of NVIDIA pivoting towards licensing and platform. ARM wasn't a play to sell more Tegras, it was a play to have GeForce be the default IP for the base tier ARM licenses.
https://corp.mediatek.com/news-events/press-releases/mediate...
in a world where software innovation is replacing hardware innovation... platform is king.
He's struck gold, now he is trying to convert it into platform. And with Sony pivoting towards AI/ML and RT upscaling with PS5 Pro, they will be essentially on par with Ada in broader feature set. And the Playstation API is a platform worthy to rival his own, Sony is uniquely positioned to go after him in the AI market and leverage studios into creating some value for them (especially with CPU speeds not increasing - use the tensors or don't, I guess!). Apple is surging ahead with Metal too - they have an excellent platform as well. His time is not unlimited here.
https://www.resetera.com/threads/tom-henderson-ps5-pro-specs...
samsung and intel foundry services are both serious possibilities imo, NVIDIA is incredibly portable across nodes and will take advantage of anything that's cheap and makes sense as a product.
for example pascal used samsung 14LPP for the 1050 and 1050 Ti, and the A100 was taped out on TSMC 7nm (not samsung 8nm), etc. They are actually quite diversified, they have a product foothold on almost every node that matters, if Samsung suddenly becomes a blazing deal they're ready to go. Etc.
they also already signed a semicustom licensing deal with Mediatek last year, this is part of an overall trend of NVIDIA pivoting towards licensing and platform. ARM wasn't a play to sell more Tegras, it was a play to have GeForce be the default IP for the base tier ARM licenses.
https://corp.mediatek.com/news-events/press-releases/mediate...
in a world where software innovation is replacing hardware innovation... platform is king.
He's struck gold, now he is trying to convert it into platform. And with Sony pivoting towards AI/ML and RT upscaling with PS5 Pro, they will be essentially on par with Ada in broader feature set. And the Playstation API is a platform worthy to rival his own, Sony is uniquely positioned to go after him in the AI market and leverage studios into creating some value for them (especially with CPU speeds not increasing - use the tensors or don't, I guess!). Apple is surging ahead with Metal too - they have an excellent platform as well. His time is not unlimited here.
https://www.resetera.com/threads/tom-henderson-ps5-pro-specs...