We aren't quite there yet, but the last year has been an incredibly exciting time for the open source LLM community. If your computer is decently powerful, you might be really surprised by what's already possible. LM Studio on Apple Silicon supports GGUF quants on GPU out the box.
For me it's pretty simple- LM Studio supports Apple Silicon GPU acceleration out of the box, and I like the interface better than Gradio Web UI. It saves me the headache and the tinkering of the alternatives. That said, free software that's hiring developers probably won't stay free for long, so I'm keeping my eye on other options.