HackerTrans
TopNewTrendsCommentsPastAskShowJobs

balnazzar

no profile record

comments

balnazzar
·letztes Jahr·discuss
Not true. It performs 20-30% better than a RTX A6000 (I have both). Except it has more than 10 times the VRAM. For a comparison with newer Nvidia cards, benchmarks say it does substantially better than a 5070ti, a bit better than a 4080, and a bit worse than a 5080. But once again, it got 30 times the vram amount of the mentioned cards, which for AI workloads are just expensive toys due to lack of vram indeed.
balnazzar
·letztes Jahr·discuss
It can run models that cannot fit on TEN rtx 5090s (yes, it can run DeepSeek V3/R1, quantized at 4 bit, at a honest 18-19 tok/s, and that's a model you cannot fit into 10 5090s..).
balnazzar
·vor 3 Jahren·discuss
Might be relevant: https://twitter.com/dzhulgakov/status/1733217065811742863.

Anyway, if the vanilla version requires 2x80gb cards, I wonder how would it run on a M2 Ultra 192gb Mac Studio.

Anyone having the machine could try?