NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inferencelmsys.org115 ポイント·投稿者 yvbbrjdr·9 か月前·93 コメント
yvbbrjdr·9 か月前·議論We actually profiled one of the models, and saw that the last GeMM, which is completely memory bound, is taking a lot of time, which reduces the token speed by a lot.
yvbbrjdr·9 か月前·議論I see! Do you know what's causing the slowdown for ollama? They should be using the same backend..
yvbbrjdr·9 か月前·議論Hi, author here. I crowd-sourced the devices for benchmarking from my friends. It just happened that none of my friend has this device.
yvbbrjdr·9 か月前·議論Hi, author here. I crowd-sourced the devices for benchmarking from my friends. It just happened that one of my friend has this device.