Leak claims RTX 5090 has 600W TGP, RTX 5080 hits 400W(tomshardware.com)
tomshardware.com
Leak claims RTX 5090 has 600W TGP, RTX 5080 hits 400W
https://www.tomshardware.com/pc-components/gpus/leak-claims-rtx-5090-has-600w-tgp-rtx-5080-hits-400w-up-to-21760-cores-32gb-vram-512-bit-bus
82 comments
My 2080Ti is still holding up reasonably well, helped by the fact I only run 1440p, but it's also doing OK for inference[1]. Though my "must have shiny new things" craving is getting hard to ignore.
Considered getting a new GPU earlier this year but then realized 5xxx was "around the corner", but now it seems they're pushed back to next year. And with AI being what it is, I'm guessing prices won't drop significantly.
Would be nice if AMD could get their GPU act together so it was a more viable alternative, NVIDIA could do with some competition.
edit: just recalled I had a dual-chip GPU back in the day, the ATI 4870X2[2]. Though that was more like two GPUs glued to one PCB, so effectively "single card SLI".
Hopefully the 5090 would be a better experience, as my 4870X2 never quite lived up to what it could theoretically do.
[1]: https://www.pugetsystems.com/labs/articles/llm-inference-con...
[2]: https://www.techpowerup.com/gpu-specs/radeon-hd-4870-x2.c236
Considered getting a new GPU earlier this year but then realized 5xxx was "around the corner", but now it seems they're pushed back to next year. And with AI being what it is, I'm guessing prices won't drop significantly.
Would be nice if AMD could get their GPU act together so it was a more viable alternative, NVIDIA could do with some competition.
edit: just recalled I had a dual-chip GPU back in the day, the ATI 4870X2[2]. Though that was more like two GPUs glued to one PCB, so effectively "single card SLI".
Hopefully the 5090 would be a better experience, as my 4870X2 never quite lived up to what it could theoretically do.
[1]: https://www.pugetsystems.com/labs/articles/llm-inference-con...
[2]: https://www.techpowerup.com/gpu-specs/radeon-hd-4870-x2.c236
Sadly AMD has given up on the high end. Which is the worst timing because desktop GPU sales are increasingly used for AI.
They had a multi-chip consumer flagship for the next rdna4, but it was reportedly having lots of problems.
I do think games have a lot more vary needs than most AI - which primarily need to crunch matrixes - so I'm a little sympathetic to a jump to many-chip failing. You also burn power on chip to chip interconnect, try as we might to ever optimize this down.
> Which is the worst timing because desktop GPU sales are increasingly used for AI.
Well, the good news is RDNA4 is the last consumer chips. Good news because they are merging RDNA with CDNA (compute/ai) to make a new UDNA (unified), which all next next gen chips will be.
https://www.tomshardware.com/pc-components/cpus/amd-announce...
I do think games have a lot more vary needs than most AI - which primarily need to crunch matrixes - so I'm a little sympathetic to a jump to many-chip failing. You also burn power on chip to chip interconnect, try as we might to ever optimize this down.
> Which is the worst timing because desktop GPU sales are increasingly used for AI.
Well, the good news is RDNA4 is the last consumer chips. Good news because they are merging RDNA with CDNA (compute/ai) to make a new UDNA (unified), which all next next gen chips will be.
https://www.tomshardware.com/pc-components/cpus/amd-announce...
Prediction: the cards will sell crazy well.
Gamers don't need high end video cards, they want high end video cards. In general, the high marginal price for low marginal value of high end video cards prevents most gamers from acting on their desires.
But this generation of video cards provides a couple of other justifications for the purchase:
- it will allow them to run uncensored ML stacks locally - it will allow the buyers to train themselves on the hottest new career path.
A large number of people who use these excuses to justify the purchases to themselves or their loved ones will only use it for gaming, but those excuses will fuel a lot of sales.
This seems like the wrong generation for AMD to skip the halo tier of gaming cards.
Gamers don't need high end video cards, they want high end video cards. In general, the high marginal price for low marginal value of high end video cards prevents most gamers from acting on their desires.
But this generation of video cards provides a couple of other justifications for the purchase:
- it will allow them to run uncensored ML stacks locally - it will allow the buyers to train themselves on the hottest new career path.
A large number of people who use these excuses to justify the purchases to themselves or their loved ones will only use it for gaming, but those excuses will fuel a lot of sales.
This seems like the wrong generation for AMD to skip the halo tier of gaming cards.
> But this generation of video cards provides a couple of other justifications for the purchase - it will allow them to run uncensored ML stacks locally
It does not. They will/are reselling exactly the same graphics cards that have been sold for the last ten years; the same computing capacity (and a modest increase for their top model), with very limited RAM, at very high prices.
They simply changed the silicon manufacturing process to smaller nodes without achieving a meaningful reduction in power consumption with respect to the previous generation, they still being high-consuming energy wasters heat generators.
It does not. They will/are reselling exactly the same graphics cards that have been sold for the last ten years; the same computing capacity (and a modest increase for their top model), with very limited RAM, at very high prices.
They simply changed the silicon manufacturing process to smaller nodes without achieving a meaningful reduction in power consumption with respect to the previous generation, they still being high-consuming energy wasters heat generators.
Flimsy justifications can be flimsy.
The same justifications sold lots of 20X0 and 30X0 generation cards too.
The same justifications sold lots of 20X0 and 30X0 generation cards too.
People simply take the marketing bait when a simple spreadsheet of 20X0, 30X0, 40X0, 50X0 GPU specifications (cores/shader units, tensor, rt) plus memory bandwidth reveals that they are just reselling the same computing capacity as they have been for the last ten years, just with smaller manufacturing nodes.
A RTX 5060 with 4352 Su, I bet will be a RTX 4060 Ti AD107 with 4352 Su, the same as a RTX 2080 Ti TU102 with 4352 Su, all ones with Mem-Bus 128 bit.
Just smaller manufacturing nodes with name change.
A RTX 5060 with 4352 Su, I bet will be a RTX 4060 Ti AD107 with 4352 Su, the same as a RTX 2080 Ti TU102 with 4352 Su, all ones with Mem-Bus 128 bit.
Just smaller manufacturing nodes with name change.
3D rendering engine benchmarks suggest what you're saying is nonsense.
Ten years of graphics card hardware tested and ranked
https://www.tomshardware.com/reviews/gpu-hierarchy,4388.html
RTX 4070 Ti is a RTX 3090 Ti
RTX 4070 is a RTX 3080
RTX 4060 Ti is a RTX 2080 Ti and RTX 3070
RTX 4060 is a RTX 2080 and RTX 3060 Ti
RTX 4050 is a GTX 1080 Ti , RTX 2070 SUPER, RTX 3060
It is not difficult to see the pattern, just look at the GPU specifications, it is not a coincidence, and it is quite predictable. The manufacturer is just playing with disabling cores.
It is not new, Nvidia has always done this. Initially, longer than a decade ago, limiting the GPU through drivers, but someone found a way to hack it, so the manufacturer started limiting the GPU by hardware at the factory. The same thing happened when people started putting vRAM in themselves and the manufacturer restricted it again on following models.
https://www.tomshardware.com/reviews/gpu-hierarchy,4388.html
RTX 4070 Ti is a RTX 3090 Ti
RTX 4070 is a RTX 3080
RTX 4060 Ti is a RTX 2080 Ti and RTX 3070
RTX 4060 is a RTX 2080 and RTX 3060 Ti
RTX 4050 is a GTX 1080 Ti , RTX 2070 SUPER, RTX 3060
It is not difficult to see the pattern, just look at the GPU specifications, it is not a coincidence, and it is quite predictable. The manufacturer is just playing with disabling cores.
It is not new, Nvidia has always done this. Initially, longer than a decade ago, limiting the GPU through drivers, but someone found a way to hack it, so the manufacturer started limiting the GPU by hardware at the factory. The same thing happened when people started putting vRAM in themselves and the manufacturer restricted it again on following models.
Well sure, current generation mid-range performs like previous generation high end. That's a tech product truism.
> they are just reselling the same computing capacity as they have been for the last ten years
I'm not 100% sure what you're trying to say here, but as far as I can tell a 4090 offers quite a bit more "computing capacity" than a Titan RTX, and that's just over five years ago.
> RTX 4060 is a RTX 2080
Probably, but I just put one in a small desktop for $280, not the $700 a 2080 cost new. So, yeah they're selling the same stuff, it just costs less.
> they are just reselling the same computing capacity as they have been for the last ten years
I'm not 100% sure what you're trying to say here, but as far as I can tell a 4090 offers quite a bit more "computing capacity" than a Titan RTX, and that's just over five years ago.
> RTX 4060 is a RTX 2080
Probably, but I just put one in a small desktop for $280, not the $700 a 2080 cost new. So, yeah they're selling the same stuff, it just costs less.
>I'm not 100% sure what you're trying to say here
What I'm trying to say is that they're deceiving customers year after year about costs and computing capacity.
It was supporting my comment six messages above:
>>> But this generation of video cards provides a couple of other justifications for the purchase - it will allow them to run uncensored ML stacks locally
>> It does not. They will/are reselling exactly the same graphics cards that have been sold for the last ten years; the same computing capacity (and a modest increase for their top model), with very limited RAM, at very high prices.
What I'm trying to say is that they're deceiving customers year after year about costs and computing capacity.
It was supporting my comment six messages above:
>>> But this generation of video cards provides a couple of other justifications for the purchase - it will allow them to run uncensored ML stacks locally
>> It does not. They will/are reselling exactly the same graphics cards that have been sold for the last ten years; the same computing capacity (and a modest increase for their top model), with very limited RAM, at very high prices.
> they're deceiving customers year after year about costs and computing capacity
> a modest increase for their top model ... limited RAM, at very high prices
Yes, I see your point, and agree.
There are far too many products in the range, and the consumer cards seem to be intentionally hampered to protect their professional / studio card business. eg. huge unnecessary 3-slot cooling arrangement on 3090/4090.
> a modest increase for their top model ... limited RAM, at very high prices
Yes, I see your point, and agree.
There are far too many products in the range, and the consumer cards seem to be intentionally hampered to protect their professional / studio card business. eg. huge unnecessary 3-slot cooling arrangement on 3090/4090.
Even the 250W 2080Ti (+150W Intel) is oppressive to be in the same room with during warmer months. I know it probably won't be, but it should be a hard sell in countries that don't have air conditioning as standard. Not to mention the noise needed to cool such heat
I'm running a 4090 at 280W, and I'm seeing ~96% of the performance of 450W. There's no need to run it at full power.
Also doing the same here. I'm hoping that more places will run power to performance analysis like the derbauer 4090 review video.
Im thinking more and more about putting my computer outside when I game.
I have 50 and 150 ft fiber optic DisplayPort cables. Can do 8k60, or 4k240. Can be had for like $70, work fine.
The hard part is input? I'm no stranger to USB extension cables. But I don't love them. They're so bulky, and they usually need 5v in every 50ft, and since USB hierarchy tops out at 7 deep and each cable is actually 2 hubs, you can only really chain two 50 ft cables together (and a 25 ft shorter active extension too, then hub and device.. gee since the cables sre hubs would sure be nice if someone made an active USB extension cable that exposed all 4 ports at the end!). Here's a well reviewed example for $42, https://www.amazon.com/Extension-Extender-Repeater-Boosters-...
There are some usb4 over fiber optic solutions, but often more than $150 for 100 ft, which is kind of a drag. Spent money on stupider things, maybe will do it.
I have 50 and 150 ft fiber optic DisplayPort cables. Can do 8k60, or 4k240. Can be had for like $70, work fine.
The hard part is input? I'm no stranger to USB extension cables. But I don't love them. They're so bulky, and they usually need 5v in every 50ft, and since USB hierarchy tops out at 7 deep and each cable is actually 2 hubs, you can only really chain two 50 ft cables together (and a 25 ft shorter active extension too, then hub and device.. gee since the cables sre hubs would sure be nice if someone made an active USB extension cable that exposed all 4 ports at the end!). Here's a well reviewed example for $42, https://www.amazon.com/Extension-Extender-Repeater-Boosters-...
There are some usb4 over fiber optic solutions, but often more than $150 for 100 ft, which is kind of a drag. Spent money on stupider things, maybe will do it.
You can turn the power level down usually without hugely reducing performance, it's quite non-linear.
This 100%. When I still built PCs, I'd usually throttle the heck out of the GPU and CPU fan curves and keep them off or just barely on but inaudible. Usually that'd mean going down to medium settings or so, but games still look good enough (and run at a high FPS) it doesn't really matter. And that made the in-room experience a lot nicer.
I wonder how DLSS and frame gen affects power usage. Presumably they save power vs drawing real frames, but I haven't tested it...?
I wonder how DLSS and frame gen affects power usage. Presumably they save power vs drawing real frames, but I haven't tested it...?
I used to do this too, now I mostly just game on Apple hardware or on Switch, which feels like the logical progression of what I was going for with less fuss.
Yeah, I can play BG3 on low settings on my M2 Max comfortably (at native res) and the fan doesn't get very loud. But the rest of the time it doesn't come on at all.
The Rosetta games (which is most of them) suck though, and Crossover/Whisky is such a hit or miss experience.
On the other hand, with GFN I can turn everything up to ultra and it looks great, and there's no fan or noise at all and barely sips the battery.
The Rosetta games (which is most of them) suck though, and Crossover/Whisky is such a hit or miss experience.
On the other hand, with GFN I can turn everything up to ultra and it looks great, and there's no fan or noise at all and barely sips the battery.
The library of games suck on Mac, but I realized my backlog has gotten so huge that I will still never run out of games that run well on there. I've got BG3 in my backlog, but I figure I'll wait until I get a new machine in a few years before I try running it.
My work machine is a M2 Max and I have a studio display. Every day I fight the temptation to install Steam and load up No Man's Sky on it...
My work machine is a M2 Max and I have a studio display. Every day I fight the temptation to install Steam and load up No Man's Sky on it...
I bet the Studio Display would be beautiful to game on, especially for a colorful game like that!
I don't think the M2 can comfortably drive that resolution at 60 fps though :/ As much as I love Apple Silicon, the GPU is a lot weaker than a RTX card. Especially since it lacks any of the AI features.
GFN offers day passes and works with your Steam library if you want to give it a shot. You don't have to install anything, just sign up for a day and you can play No Man's Sky in a browser window at max settings. (But I think you might need the GFN app to play at full native resolution or with proper HDR, not sure).
I don't think the M2 can comfortably drive that resolution at 60 fps though :/ As much as I love Apple Silicon, the GPU is a lot weaker than a RTX card. Especially since it lacks any of the AI features.
GFN offers day passes and works with your Steam library if you want to give it a shot. You don't have to install anything, just sign up for a day and you can play No Man's Sky in a browser window at max settings. (But I think you might need the GFN app to play at full native resolution or with proper HDR, not sure).
The nice thing about Studio Display is that it's exactly 2X the resolution of 1440p, so that resolution works very nicely on it. Definitely want to give GFN a shot too though, thanks for the recommendation.
AMD's decision to back out of the high end cards seems even more logical given this information. High end may be trending too powerful. I used to buy the high end cards, but my next one will not be so. The cost and the power consumption are large factors in that decision. I do not need nor can I afford a super computer for gaming.
Does it really? I keep thinking that they could just slap on 32G of VRAM on a midrange card and they would get roaring enthusiast support. I'm guessing the only reason they don't do that is to not encroach on their more expensive ML cards?
For gaming I would be happy with 16GB card. Enough to survive three or so generations more.
I personally game a bit but I spend more time these days experimenting with deep learning so I would pay a premium for extra VRAM on my home machine.
It seems both Nvidia and Intel are now brute forcing things with more energy consumption and possibly much larger dies. Which is not that great in either power efficiency or costs.
Doing best at 100-150W GPU at top seems most responsible move to me. With reasonable cost for the GPU to boot.
Doing best at 100-150W GPU at top seems most responsible move to me. With reasonable cost for the GPU to boot.
Have you ever considered Geforce Now? It's incredible to use, and all the power consumption is in their data center instead of your home. Right now it's a 4080 but presumably they'll upgrade to 5080s once those are out.
I used to build my own PCs, but GFN is a much nicer (and significantly cheaper!) experience overall. I can play everything on ultra, at 4k (with DLSS) for $20/mo and don't need to worry about keeping up with new GPUs or local heat, noise, and system maintenance. For an aged, busy gamer, it's really really nice.
I used to build my own PCs, but GFN is a much nicer (and significantly cheaper!) experience overall. I can play everything on ultra, at 4k (with DLSS) for $20/mo and don't need to worry about keeping up with new GPUs or local heat, noise, and system maintenance. For an aged, busy gamer, it's really really nice.
What games are you playing? I'm guessing FPS don't work so well with latency
I do play shooters from time to time. It's not quite as smooth as a good local GPU, but perfectly playable if you're not competitive. A bit of Destiny 2, The Division 2, Remnant 2, Apex Legends, a bunch of other forgettable extraction shooters, etc. I win some matches but not a lot. At my level, it's my skill (or lack thereof) that's the bottleneck, not GFN lag. If you're trying to go pro, I probably wouldn't consider this good enough. But for casual play it's great.
With 120fps and Reflex on a fast connection, the lag is minimal and lower than what you might find on many consoles with TV lag. Not as crystal smooth as a true high end PC with a fast mouse and 240 Hz monitor, but really not as bad as you'd fear.
Aside from shooters, I play 95% of my games on GFN (along with a handful of unsupported ones using Crossover or Parallels, which both suck compared to GFN). Mostly ARPGs these days (PoE, D4, Last Epoch), some factory games and city builders (Cities Skylines, Frostpunk), and a handful of light sims (Snowrunner).
It's so nice. I could never afford a 4080 or any of their GPUs anymore, but honestly I wouldn't bother even if I could. Not having to manage my own thermals and noise is amazing, as is being able to play on a MacBook, my handheld, my phone, and my TV.
With 120fps and Reflex on a fast connection, the lag is minimal and lower than what you might find on many consoles with TV lag. Not as crystal smooth as a true high end PC with a fast mouse and 240 Hz monitor, but really not as bad as you'd fear.
Aside from shooters, I play 95% of my games on GFN (along with a handful of unsupported ones using Crossover or Parallels, which both suck compared to GFN). Mostly ARPGs these days (PoE, D4, Last Epoch), some factory games and city builders (Cities Skylines, Frostpunk), and a handful of light sims (Snowrunner).
It's so nice. I could never afford a 4080 or any of their GPUs anymore, but honestly I wouldn't bother even if I could. Not having to manage my own thermals and noise is amazing, as is being able to play on a MacBook, my handheld, my phone, and my TV.
The load times are pretty terrible and game compatibility is 80-90%. But otherwise I are it’s a good deal. Great for turn based and rpg type games, but very mid for first person. I would recommend it with those reservations. Also I don’t think the high end tier is worth it because of the bandwidth inconsistency in their data centers.
Load times, you mean when you first start a game? Once you're in, everything is fast.
As for bandwidth inconsistencies, what do you mean? In the US northwest and in Chicago at least, I've had the high tier for years and it's been fantastic. Sure it's not your ISP or router? FWIW I've found hardwired ethernet to work a lot better than WIFI.
As for bandwidth inconsistencies, what do you mean? In the US northwest and in Chicago at least, I've had the high tier for years and it's been fantastic. Sure it's not your ISP or router? FWIW I've found hardwired ethernet to work a lot better than WIFI.
Yeah first start is such a terrible experience. 2-3 splash screens all taking forever. Compared to modern consoles where you can just suspend and go. Once it gets going it’s a non issue but fists start up is a couple minutes which is aggravating
If the trend in NVIDIA deceiving customers continue, the 5090 will be the new xx70 and the 5080 the new xx60.
This tango NVIDIA is dancing will continue as long as ( local ) AI doesn't reach plateau where even most enthusiasts are happy with the AI model they have, because when it comes to games, other than a few streamers and rich kids, nobody will run to buy this, even AAA games are dying and the appetite for running them with the latest and greatest graphics has mostly disappeared ( given the cost ).
From a gaming perspective, technicality is not anything like the ending days of Voodoo cards, but somehow it reminds me of the same feeling.
From a gaming perspective, technicality is not anything like the ending days of Voodoo cards, but somehow it reminds me of the same feeling.
True, but also AI could be the reason why games would need more powerful GPUs in the future (but not yet). For example LLMs for NPC dialogs, or the wilder things like running the whole game inside a neural network, or running an AI style filter over the output of the regular game engines.
https://futurism.com/doom-running-on-neural-network
https://www.youtube.com/watch?v=udPY5rQVoW0
https://x.com/VaigueMan/status/1836802465133437056
https://futurism.com/doom-running-on-neural-network
https://www.youtube.com/watch?v=udPY5rQVoW0
https://x.com/VaigueMan/status/1836802465133437056
For example LLMs for NPC dialogs
What's cool about this is that (unlike so much generative AI) is that a game based around this would not be cutting writers out of the loop.
I'm imagining a game where the writer(s) produce backstories for the characters, and all sorts of dialogue they might possibly say. And then the LLM is sort of remixing that stuff in real time to produce novel dialogue and behaviors.
Done well, I think it could be pretty seamless and compelling...
I mean, Nvidia ended up buying 3dfx and using some of their SLI tech for a while. But then it turns out DLSS and frame generation were much better ways at increasing FPS and quality.
PC gaming was always a niche market relative to the consoles, and it will probably remain so. But also, it's never looked better, had such a great selection (especially indie games), or been this affordable (between Gamepass, GeForce Now, and various publisher subs). For like $30/mo you can play many of the latest amazing games on max graphics without owning a GPU at all.
I hope, like servers, more and more of this moves to the cloud. It doesn't make sense to run much of this workload locally when they can be more efficiently managed and shared in data centers anyway, with better power and cooling management than most home PCs could have.
PC gaming was always a niche market relative to the consoles, and it will probably remain so. But also, it's never looked better, had such a great selection (especially indie games), or been this affordable (between Gamepass, GeForce Now, and various publisher subs). For like $30/mo you can play many of the latest amazing games on max graphics without owning a GPU at all.
I hope, like servers, more and more of this moves to the cloud. It doesn't make sense to run much of this workload locally when they can be more efficiently managed and shared in data centers anyway, with better power and cooling management than most home PCs could have.
>Nvidia ended up buying 3dfx and using some of their SLI tech for a while
The SLI Nvidia used had nothing to do with the tech from 3df besides the initials and the general idea of the concept, especially since the way the Nvidia GPUs worked at the time Nvidia dropped SLI was totally different than the way 3dfx's 3d accelerators worked.
The SLI Nvidia used had nothing to do with the tech from 3df besides the initials and the general idea of the concept, especially since the way the Nvidia GPUs worked at the time Nvidia dropped SLI was totally different than the way 3dfx's 3d accelerators worked.
Ah OK, thanks for the correction!
> PC gaming was always a niche market relative to the consoles, and it will probably remain so.
The difference is not that huge though. Supposedly 43:57 by revenue:
https://www.visualcapitalist.com/visualizing-pc-vs-console-g...
According to other stats:
https://www.statista.com/statistics/292460/video-game-consum....
PC is even bigger.
And this is by revenue, I would guess that on average PC games might be significantly cheaper? Meaning that more people actually play them.
Or it might just be skewed by some highly addictive/competitive in Asia etc. (where IIRC console gaming was never that big outside of Japan).
The difference is not that huge though. Supposedly 43:57 by revenue:
https://www.visualcapitalist.com/visualizing-pc-vs-console-g...
According to other stats:
https://www.statista.com/statistics/292460/video-game-consum....
PC is even bigger.
And this is by revenue, I would guess that on average PC games might be significantly cheaper? Meaning that more people actually play them.
Or it might just be skewed by some highly addictive/competitive in Asia etc. (where IIRC console gaming was never that big outside of Japan).
If this doesn't stop, then very soon we'll just end up with external video cards because this is ridiculous. We already kinda have that [0], but it's quite a hack. I wonder how well PCIe can be delivered over the ribbon cables though...
[0] https://www.razer.com/mena-en/gaming-laptops/razer-core-x
[0] https://www.razer.com/mena-en/gaming-laptops/razer-core-x
I saw someone who used two humongous, external watercooling radiators to cool both their GPU and CPU. Apparently these are now available to buy.
Kinda, because USB-C/Thunderbolt eGPUs like the Razer Core have noticeably less bandwidth than the same card installed on your motherboard.
Which is why OCuLink is gaining traction. Lenovo added hot swap capability and sells it under the name "TGX" in China but the compatibility is total both ways.
Also Beelink has MiniPCs with external PCIe x8 slots, it's a clever trick: https://liliputing.com/beelink-gti14-ultra-is-an-intel-meteo...
Also Beelink has MiniPCs with external PCIe x8 slots, it's a clever trick: https://liliputing.com/beelink-gti14-ultra-is-an-intel-meteo...
I would actually welcome that. It would make building small pcs way easier. And make upgrades and swaps a breeze
Can I boil water for my tea with it already, or should I wait next gen?
Are computers that consume over 1500 watts obligated to return a 418?
A proper electric kettle is what, 3 kW?
But if you plug two 5090s and some next gen Intel CPU in a very inefficient PSU, you have a chance...
But if you plug two 5090s and some next gen Intel CPU in a very inefficient PSU, you have a chance...
1600-1800W in North America, because your standard outlet is 120V 18A max.
Most home offices are likely converted bedrooms. Those plugs, at least where I'm from, are likely to have 15A outlets, only.
1 kW in Japan. Same power plugs as North America, but only 100 V.
I have been running 4090 with 13900KS (both water cooled with AIO) for 18 months now. Typical power consumption during use is around 70..80W (as measured from the wall). Max I have managed to press for under stress testing is 660W. System’s 850W power supply feels just fine, should have got a smaller one.
Guess I will heat my basement with 5090 this winter. 3090 was one of the best purchases I have ever made. Mined ETH to pay for itself within two months. Then came all the AI fun. Can’t think of anything that was as useful in my life.
Finally a valid reason for those 1000W platinum PSUs.
Their highest efficiency is at 80-90% utilization, but the efficiency drops off when underutilized.
Their highest efficiency is at 80-90% utilization, but the efficiency drops off when underutilized.
The reason for them is reliability. Running everything closer to limits really limits reliability.
Aside from that, 1000W is very theoretical. You have some maximum on each rail. If you have 500W on the 12V rail max, and need 600W, it doesn't matter you have another 500W on the other rails. You're SOL.
Another reason is start-up surge. It's less of an issue without mechanical disks, but charging all those capacitors on power-on can really lead to badness.
And droop and ripple are also lower if you're not near the limit. Computers have things like audio cards, WiFi, and other analog parts which perform better with cheap power.
I always overspec PSUs.
Aside from that, 1000W is very theoretical. You have some maximum on each rail. If you have 500W on the 12V rail max, and need 600W, it doesn't matter you have another 500W on the other rails. You're SOL.
Another reason is start-up surge. It's less of an issue without mechanical disks, but charging all those capacitors on power-on can really lead to badness.
And droop and ripple are also lower if you're not near the limit. Computers have things like audio cards, WiFi, and other analog parts which perform better with cheap power.
I always overspec PSUs.
It would suck to build a system only to find out the PSU isn't up to snuff but my 4090 and overclocked CPU run fine on an 850w PSU.
I may upgrade preemptively to beat the PSU rush when their next cards release.
I may upgrade preemptively to beat the PSU rush when their next cards release.
Instead of hunting globally for places to power more and more AI,
Perhaps instead electricity usage should be frozen at some level
and it would be up to the AI chip makers to produce solutions
that gives more calculations for the same amount of power.
Which has been a huge factor in data center servers from what i know for a long time. Somehow that has been forgotten
Which has been a huge factor in data center servers from what i know for a long time. Somehow that has been forgotten
Will the price be 3000 or 5000 for 5090? With nice top of the line CPU you can a small space heater here. Must be nice in summer.
We want cheap(ish) desktop GPUs with 80G or more VRAM, so not only large corporations could meaningfully participate in AI research.
The price divide between “desktop” and “datacenter” GPUs is artificial and no doubt there is a collusion of some kind.
The price divide between “desktop” and “datacenter” GPUs is artificial and no doubt there is a collusion of some kind.
You can get Macs with 192GB of UMA, so maybe 180GBusable VRAM. But of course the GPU horsepower is much less tban an A100.
Sort of works for inference, but not for anything else. For now, I bought a 4x4090 workstation… but I am crying how power-inefficient it is.
For my education, would you be able to expand on what it doesn't work for and why? Thank you!
Inference (token generation) is memory-bound, KV cache prefill (prompt processing) is compute-bound. The ARM Macintoshes have lots of memory bandwidth but not a lot of compute power, so they're great for outputting text but terrible for tasks like analyzing documents. I've never done fine-tuning but my understanding is that that is a highly-parallelizable compute hog as well.
You might like this article, which looks at the arithmetic intensity of LLM processing: https://www.baseten.co/blog/llm-transformer-inference-guide/
You might like this article, which looks at the arithmetic intensity of LLM processing: https://www.baseten.co/blog/llm-transformer-inference-guide/
You know that their top tier datacenter card has "only" 80GB? Their workstation cards with 48GB are also reasonably priced... with 3-4k pricing.
>Their workstation cards with 48GB are also reasonably priced... with 3-4k pricing.
That's twice the price of a 4090. You can't tell me the extra RAM costs that much.
That's twice the price of a 4090. You can't tell me the extra RAM costs that much.
It's not only VRAM. more cores, more bandwith, better power delivery, workstation features...
Which card do you mean that has more cores/computing performance than 4090 and costs 4-5k?
NVIDIA RTX 6000 Ada Generation
Although they retail for more than $4k.
The previous generation RTX A6000 is probably what they're referring to above.
Although they retail for more than $4k.
The previous generation RTX A6000 is probably what they're referring to above.
It is $7.5k, and slower than 4090 compute-wise. Two 4090s (which are cheaper than A6000) are strictly superior to it.
Indeed two 4090s provide more CUDA/RT/Tensor cores, but the A6000 gives you 48GB of RAM which can end up making the card faster at certain tasks that would require swapping out data to system RAM on a 24GB card.
Two 4090s have 48G too. For most (if not all) LLM-related tasks, there is no difference how many cards are used for the same amount of VRAM.
Nvidia B100 has 192G VRAM.
They are not shipping in any reasonable quantity. They are also 2x96GB.
[1] https://wikipedia.org/wiki/Performance_per_watt