Author of the post here. I have to caveat this - I do use cloud-hosted SOTA models extensively, but whilst I am British, I am not Simon Willison and do not live and breathe local models!
I'm curious what results you can get from your own kit, if you can tune the setup better, using the same inputs or even a different model.
Do test and experiment for yourself. Do not take me as an expert on all things local LLM.
I did use one of our products to sandbox Ollama, and Claude - other sandboxes are available, many cloud-hosted, I'm sure they are also excellent choices.
I'm curious what results you can get from your own kit, if you can tune the setup better, using the same inputs or even a different model.
Do test and experiment for yourself. Do not take me as an expert on all things local LLM.
I did use one of our products to sandbox Ollama, and Claude - other sandboxes are available, many cloud-hosted, I'm sure they are also excellent choices.