yvbbrjdr

9 месяцев назадdiscuss

We actually profiled one of the models, and saw that the last GeMM, which is completely memory bound, is taking a lot of time, which reduces the token speed by a lot.

yvbbrjdr

9 месяцев назадdiscuss

I see! Do you know what's causing the slowdown for ollama? They should be using the same backend..

yvbbrjdr

9 месяцев назадdiscuss

Hi, author here. I crowd-sourced the devices for benchmarking from my friends. It just happened that none of my friend has this device.

yvbbrjdr

9 месяцев назадdiscuss

Hi, author here. I crowd-sourced the devices for benchmarking from my friends. It just happened that one of my friend has this device.

yvbbrjdr

в прошлом годуdiscuss

We use DeepSeek v3 in prod. Works even better than GPT-4o.

Submissions

NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference

LMRouter: Run Claude Code on Any LLM with One Command

Athena – An open source production-ready general AI agent

comments