My $600 Mac Mini Runs a 35B AI Model(thoughts.jock.pl)

4 pointsby danebalia3 tháng trước3 comments

https://thoughts.jock.pl/p/local-llm-35b-mac-mini-gemma-swap-production-2026

3 comments

bigyabai3 tháng trước

> The 35B Trick (Your SSD Is the New GPU Memory)

Wave "bye bye" to your write cycles.

RobMurray3 tháng trước

why? it's mostly reads. the weights are static.

bigyabai3 tháng trước

llama-cpp's process is, but macOS itself will swap hard when 10-14gb of memory is paged for LLM inference. Dense models especially would thrash zram.