Gaudi 2 – An AI Accelerator Card with 96GB of HBM2E Memory(habana.ai)
habana.ai
Gaudi 2 – An AI Accelerator Card with 96GB of HBM2E Memory
https://habana.ai/products/gaudi2/
5 comments
Hmm… has anybody being at Habana[^1] recently? It’s like naming a company “Pyongyang”. In couldn’t use any of their products just because of the halo of “bad things”[^3] that covers the country that crumbling[^2] city is in. And Intel will need an extra wallop of good faith from the public if they want their AI products to gain any mind share.
[^1]: In case you are not sure, “Habana” is the Spanish spelling of Havana.
[^2]: https://pages.vassar.edu/realarchaeology/2019/11/10/ruinatio...
[^3]: I’m visiting that country right now.
[^1]: In case you are not sure, “Habana” is the Spanish spelling of Havana.
[^2]: https://pages.vassar.edu/realarchaeology/2019/11/10/ruinatio...
[^3]: I’m visiting that country right now.
I was curious about energy efficiency and took two samples from the linked MLPerf GPT3 results. H100 seems about three times more efficient than Gaudi2.
256 Gaudi2 600W TDP: 256 * (442.578 / 60) * 0.6 = ~1133 kWh
512 H100 700W TDP: 512 * (64.264 / 60) * 0.7 = ~384 kWhI'm guessing H100 has 2x host energy overhead for connecting those GPUs? That might offset some of the perf/W benefits of nvidia's offering.
I have to say, I was expecting the PyTorge migration guide <https://docs.habana.ai/en/latest/PyTorch/PyTorch_Model_Porti...> to be filled with a lot more silliness while trying to break the CUDA stranglehold so I wish them all the best
So until I hear otherwise, I'm going to assume that this doesn't provide a better price to perf ratio than existing commercial setups.