Ask HN: I have $5k to spend on a local AI machine, what should I get?
I would like to run local models as large and as fast as possible for around $5,000 USD. Is an Apple machine the best choice with their shared memory or is there a particular GPU that would be more cost effective? Thanks!
22 comments
I guess I would buy nvidia digits https://www.nvidia.com/en-us/project-digits/
This is exactly what I'd recommend too, it's cheaper than buying GPUs individually and Digits has way more VRAM.
Is it available for purchase?
Thank you! This looks awesome!
Not precisely what you were looking for, but this was going around yesterday: https://rasim.pro/blog/how-to-install-deepseek-r1-locally-fu...
Unpopular but highly promising way to go if training is on your mind- 4x 7900 xtx cards and some nuts and bolts to feed them could be a price per GPU memory high point. There are folks using ROCm with that to put up some interesting numbers in terms of wall clock and power required per training run.
This Reddit comment mentioned this site with used servers:
https://pcserverandparts.com/ https://www.reddit.com/r/LocalLLaMA/comments/1i8rujw/comment...
https://pcserverandparts.com/ https://www.reddit.com/r/LocalLLaMA/comments/1i8rujw/comment...
Be more specific - AI is a very broad field.
nVidia GPUs have the best inference speed (particularly around SDXL, Hunyuan, Flux, etc), but unless you're buying several used 3090s SLI style, you're going to have to split larger LLM GGUFs across main memory and GPU. I'm excluding the RTX 5090 since two of them (plus tax) would basically blow your budget out the water.
With Apple I think you can get up to 192GB of shared memory allowing for very large LLMs.
Another thing is your experience. Unless you want to shell out even more money, you'll likely have to build the PC. It's not hard but it's definitely more work than just grabbing a Mac Studio from the nearest Apple Store.
nVidia GPUs have the best inference speed (particularly around SDXL, Hunyuan, Flux, etc), but unless you're buying several used 3090s SLI style, you're going to have to split larger LLM GGUFs across main memory and GPU. I'm excluding the RTX 5090 since two of them (plus tax) would basically blow your budget out the water.
With Apple I think you can get up to 192GB of shared memory allowing for very large LLMs.
Another thing is your experience. Unless you want to shell out even more money, you'll likely have to build the PC. It's not hard but it's definitely more work than just grabbing a Mac Studio from the nearest Apple Store.
I bought a fully tricked out Mac Studio and had planned to use it but I found I was far too busy in my final career and actually it sits idle.
So, while I do not regret my decision based on what I knew at the time, my life trajectory has left it unused.
So, I want to get back to doing LLM work on it for fun, Julia and R programming.
I just got a Mac Mini with maximum specs (can't believe how small the box it came in was!) and that's not a bad choice. As you say it has the advantage of handling large models. I think the 5090 will outperform it in terms of FLOPS but it only comes with 32MB compared to the 64MB you can get on an M4 mini. The 5090 itself will be $2000 (if you can get it at that price) compared to the $2500 max mini M4. You'll probably spend at least $1k for the rest of the PC worthy of the 5090 card.
More info needed but for 5k you can get a near maxed mbp that should be able to handle most models, perform pretty well, and also serve as your laptop so it’s not a pure AI box. Then if you have heavier needs go to the cloud.
Here's a nice looking DeepSeek build https://x.com/carrigmat/status/1884244369907278106?mx=2
https://www.amazon.com/Yassk-Fortune-Telling-Floating-Answer...
AI in the palm of your hand! Best deal evarrr!
AI in the palm of your hand! Best deal evarrr!
Why get an Apple? Even th3 keyboard lacks required keys for development. They're purely tech bro poser machines.
What keys does it lack for development? I haven’t noticed any missing in 40 years of coding on Macs.
[deleted]