Dell exec reveals Nvidia has a 1,000-watt GPU in the works(theregister.com)
theregister.com
Dell exec reveals Nvidia has a 1,000-watt GPU in the works
https://www.theregister.com/2024/03/05/nvidias_b100_gpu_1000w/
58 comments
[deleted]
With the recent breakthrough in ternary weight models it'll be interesting to see how needed this kind of bruteforce power will be.
Why not have both software and hardware improvements? This is like saying we shouldn't have any new CPUs after the the PDP-1 because quicksort got invented
> This is like saying we shouldn't have any new CPUs after the the PDP-1 because quicksort got invented
I didn't assert anything here, I merely pondered. However, lets pretend that I did say that we shouldn't need heavy GPUs anymore. I don't think that's an entirely bad take because it would be fantastic if we could run larger models with more parameters on lower spec hardware. Especially phones that aren't flagship models.
I didn't assert anything here, I merely pondered. However, lets pretend that I did say that we shouldn't need heavy GPUs anymore. I don't think that's an entirely bad take because it would be fantastic if we could run larger models with more parameters on lower spec hardware. Especially phones that aren't flagship models.
Sure, the analogy was a bit exaggerated perhaps :)
I also of course didn't necessarily want to imply whether it's actually a good idea to have a 1000W hogging monster card, vs e.g. putting more memory on consumer cards
I also of course didn't necessarily want to imply whether it's actually a good idea to have a 1000W hogging monster card, vs e.g. putting more memory on consumer cards
Yeah, there's this weird notion that OSS work on running LLMs on consumer hardware, and researching more efficient architectures, is somehow in opposition or competition to the Evil AI Wielding Corporations. It's not. Like always, whatever tool an individual has, a corporation can afford to use more of it, and use it better.
You would still use floating point for training afaik.
The big question is "Will it blend (with the datacenter dynamics, power distribution and cooling)?".
Considering a conservative 4GPU system adds a 1.2KW additional overhead, packing these together densely in racks is a huge ordeal.
Considering a conservative 4GPU system adds a 1.2KW additional overhead, packing these together densely in racks is a huge ordeal.
The big questions is, my heat pump is 2.6 kW, can I remove the heat pump and run 2 of these GPUs instead?
Considering that heat pumps have a coefficient of performance of greater than 1 (meaning they are moving more than 2.6 kw of heat), no. You'd probably need 3 to 5 GPUs to get comparable levels of warmth.
These days you can already get water-cooled servers [1] and rack-mount water distribution manifolds [2]
[1] https://www.supermicro.com/en/solutions/liquid-cooling [2] https://www.ibm.com/docs/en/power8?topic=POWER8/p8had/p8had_...
[1] https://www.supermicro.com/en/solutions/liquid-cooling [2] https://www.ibm.com/docs/en/power8?topic=POWER8/p8had/p8had_...
I mean, we have these, but changing to a bigger CDU just because you decided to get a rack of GPU servers is not always fun/easy/doable.
Even if you just upgrade them, then there's the primary loop's capacity.
Even if you just upgrade them, then there's the primary loop's capacity.
I can't find it anymore, but I remember reading a few years ago about a watercooled server farm that used an outdoor water fountain. It wasn't a single rack, but a few 100s of them
Those dedicated gpus draw way too much power. Why isnt there more focus on low power cards?
Same on cpus.
Electricity prices are set to drastically increase in my area.
Same on cpus.
Electricity prices are set to drastically increase in my area.
Looks like a solution in search of a problem.
AI workloads are currently going through their "developers are expensive, hardware is cheap(ish)" era. Nobody is trying very hard to fit models to GPUs, and optimize utilization and whatnot.
Instead they throw in more GPUs. So B100 will fit right into the bill.
Instead they throw in more GPUs. So B100 will fit right into the bill.
There's a lot of work being put in to make LLMs run on consumer hardware, albeit it's being done by the open source community and not the companies themselves.
The consumer hardware is made by the same company and there's little competition. Rather than undercut the price of commercial hardware, what commercial applications for GPUs has done is raise the price floor of consumer GPUs. The companies are not undercutting themselves. It's profit city.
I think AI accelerators in ARM based systems will be start to eat that green company from below. Even Intel's bottom of the barrel N95/N100 has accelerators now.
I mean, for the training part. Inference is easy (relative to training). Even my 3 year old camera can do real-time eye/face-tracking for humans and mammals via a real-time neural network running on a dedicated low power DSP.
>"developers are expensive, hardware is cheap(ish)"
Is it even true?
H100 is worth 1 year of salary of mid engineer in eastern eu - 40k usd before tax
Is it even true?
H100 is worth 1 year of salary of mid engineer in eastern eu - 40k usd before tax
Generally it's thought in the hourly cost, plus how much productivity it adds to your workflow.
IOW, you pay yearly price of an engineer, plus some hourly cost, but it generally adds more productivity rather than optimizing what you have at hand.
> H100 is worth 1 year of salary of mid engineer in eastern eu
If you can have them supplied to you, in the first place.
IOW, you pay yearly price of an engineer, plus some hourly cost, but it generally adds more productivity rather than optimizing what you have at hand.
> H100 is worth 1 year of salary of mid engineer in eastern eu
If you can have them supplied to you, in the first place.
The problem is EU's winter. Pair this bad boy with a heat pump and watch it roar.
Nice, we'll be able to consume the Earth's resources even faster. Congrats everyone!
I must disagree. Do you oppose renewable energy? If not then by definition, it is possible to use more energy and not deplete any natural resources. The consumer blaming technique is one fossil fuel companies like to spread and I believe you are inadvertently spreading.
I was just reading an HN post about AWS getting rights to a nuclear powerplant. FAANG companies are getting more and more carbon neutral datacenters. The increasing energy needs of humans can be met with renewable energy. But saying it's people's fault for not conserving energy is giving power companies an excuse to continue using fossil fuels.
I was just reading an HN post about AWS getting rights to a nuclear powerplant. FAANG companies are getting more and more carbon neutral datacenters. The increasing energy needs of humans can be met with renewable energy. But saying it's people's fault for not conserving energy is giving power companies an excuse to continue using fossil fuels.
Renewable energy does deplete a lot of natural resources. Even assuming it is carbon neutral (it isn't). Even assuming we're still capable of constructing those without fossil energy (we currently can't). Even assuming mining of various minerals is done without fossil energy (it isn't). Even assuming it doesn't require any mining (it does). It still largely requires "natural surfaces" where we have to disrupt local environments. It might be net benefits (offshore wind farms seem to improve diversity), but we have no renewable energy growth that doesn't require first for the natural resources to take a toll.
Don't get me wrong, I do think this is where we're heading towards, and I feel it's okay. But let's not be blind that yes even assuming we go towards fully renewable energy (still a big if, or rather very long when), any growth of energy consumption comes with a cost to natural resources.
Don't get me wrong, I do think this is where we're heading towards, and I feel it's okay. But let's not be blind that yes even assuming we go towards fully renewable energy (still a big if, or rather very long when), any growth of energy consumption comes with a cost to natural resources.
Unless you speak of the supernatural, all resources are natural. Humans and our desire for energy is also natural. Ecosystem disruption is also natural since we are natural also. The only question is sustainability. More solar farms in the desert disrupts an ecosystem but not to the extent of causing species to die off or a food chain collapse that affects humans. More nuclear plants use more fissile material but at such low volume and extraction cost it's negligible.
If you take a walk through the woods you'll step on grass and insects and cause damage. It's unnatural to live without interacting and harming flora and fauna and there are plenty of energy sources to sustain humans for millenia to come.
If you take a walk through the woods you'll step on grass and insects and cause damage. It's unnatural to live without interacting and harming flora and fauna and there are plenty of energy sources to sustain humans for millenia to come.
Consuming resources is a necessary part of living. Question is, what do we get out of consumed resources. And at this point, it's logical and clear, the only way to go from here is out:
https://www.youtube.com/watch?v=ab_mH8R0KTM
https://www.youtube.com/watch?v=ab_mH8R0KTM
We will nuke ourselves before resources are fully wasted.
Maybe. The increased competition and other side effects of increased resource usage also increase the chance of us nuking ourselves, so it's liable to be a close-run thing.
The resources are there for us to consume
Huh? This is an AI accelerator card. So you should be looking at performance per Watt, instead of absolute numbers. The linked article doesn't mention anything about that.
As long as we insist on doing the same thing, we will never use less in total. Point me to any sizable consumption field where gains in efficiency didn't translate to more usage.
Lighting using using LEDs vs incandescent bulbs: streets and buildings are not ten times brighter than a couple of decades ago.
Good example. Indeed light bulbs didn't get 6-10x brighter than before. But there are that many more of them now, sources: Kyba, Christopher CM, et al. “Artificially lit surface of Earth at night increasing in radiance and extent.” Science advances 3.11 (2017): e1701528. http://advances.sciencemag.org/content/3/11/e1701528.full?in...; Tsao, Jeffrey Y., et al. “Solid-state lighting: an energy-economics perspective.” Journal of Physics D: Applied Physics 43.35 (2010): 354001. https://www.researchgate.net/publication/228627788_Solid-sta...
Heating / cooling. People keep their houses at roughly the same temperature regardless of consumption. You don't get your house from 21 C to 26 just because your heater became more efficient.
Based on this https://www.epa.gov/climate-indicators/climate-change-indica... it doesn't look like it's going down to me.
The transition from incandescent to LED lighting, perhaps?
Yeah so they should increase the efficiency instead of the power.
Well, there's no yeah, because there's no ratio...
For all you could know energy went twofold and performance threefold..
For all you could know energy went twofold and performance threefold..
They said nothing about performance, and if you think it increased without them saying anything about it, then you're being highly optimistic.
Right now I can only assume they made it easier to burn more energy.
Right now I can only assume they made it easier to burn more energy.
> They said nothing about performance
1kW says a lot about perfornance. You can heat a house with it. /s
1kW says a lot about perfornance. You can heat a house with it. /s
The best way to enjoy HN is to just ignore these comments. There are a lot of bored, angsty, frustrated, lonely people, many struggling with mental illness that like to make comments like this for no other reason than getting attention.
Can't take a statement like this seriously when received in digital form, send it me as a letter.
We've never consumed resources faster, yet there's never been more resources made accessible by our efforts for us all to consume. Every tech advance unlocks more!
Space exploration and mining will be a step change in accessible resources. Why not enjoy the non-scarcity!
Space exploration and mining will be a step change in accessible resources. Why not enjoy the non-scarcity!
We'll blow ourselves up before climate and scarcity are solved. AI might help with this.
Yes, AI might help blow ourselves up.
Depends what the externalities of that resource usage are, and who bares them.
Currently while NVidea shareholders reap the reward, it's people in the global south that suffer the immediate consequences (although the rest of us will eventually too)
Currently while NVidea shareholders reap the reward, it's people in the global south that suffer the immediate consequences (although the rest of us will eventually too)