Nvidia DGX Spark(nvidia.com)

15 pointsby tanelpoderanno scorso10 comments

https://www.nvidia.com/en-us/products/workstations/dgx-spark/

13 comments

vaxmananno scorso

Heard coming from the underground parking area at One Apple Park Way earlier today https://youtu.be/BMm4FFrSxMk

Heard coming from the Taiwan-infused PC clone manufacturer industry https://youtu.be/wIE5wwvicew

Meanwhile, at whatever planet OpenAI lives on https://youtu.be/615lNmMiOSQ

moondevanno scorso

"ConnectX-7 Smart NIC"

https://resources.nvidia.com/en-us-accelerated-networking-re...

So those two QSFP ports can be configured for 400GbE IB and/or RoCE?

Also is it fair to say that this can run a desktop considering it has an HDMI port?

dragonwriteranno scorso

Yes, the whole point is that it is a compact desktop for AI applications, so, yes, it can run a desktop.

moondevanno scorso

Nvidia DGX and HGX servers packed NVME, 8 H100 GPUs and 8 Infiniband cards don't run a desktop. They don't even have an HDMI port.

[deleted]anno scorso

andrewSCanno scorso

Am I missing something here or is inference going to be painful given the "low" memory bandwidth compared to, say, HBM2E?

slikenanno scorso

273GB/sec with good FP4 performance should be fine for developers playing with inference. This isn't the kind of thing that you'd use for inference workloads supporting millions of users.

I'd like to see a inference benchmark vs the strix halo, which has better memory bandwidth and costs 2/3rds as much.

tanelpoderanno scorso

I guess since these devices aren't meant for production throughput, but rather about having enough RAM for local experimentation with large enough models, it's an ok tradeoff at this price point...

dragonwriteranno scorso

Have you seen anything with 128GB of HBM2E at anywhere near the DGX Spark's $3,000 price point?

karmakazeanno scorso

The AMD Radeon VII has 16GB HBM2 and sold for $700 in 2019. I don't know how that would translate to today's HBM2E's pricing like if its price change follows that of GDDR's.

[deleted]anno scorso

andrewSCanno scorso

I honestly can't say I have however, that doesn't mean it couldn't physically exist/happen? Perhaps a "little" more cost but I'd be willing to bet people would gladly pay the premium for such a device. I'm also very curious to know what the BOM for an A100 actually is as well as HBM2E per GB.

[deleted]anno scorso