Right? I feel like local model support is something that has strong ideological appeal and might influence someones feeling towards the app, but when it comes down to it most people will probably just use a cloud model for larger tasks unless they have beastly hardware. It's like how I have an Android in part because I might one day flash the ROM.
Ollama just exposes models via an OpenAI compatible endpoint though (I'm pretty sure), so adding that standard is probably a good idea. The prompts are a bit tuned for Gemini. I'd have to test how much that matters.
Hey thanks! Good call. I had limited success getting Qwen 3.5 9B working for some of the longer prompts that require lots of json output. I feel like completely on-device is so close to being usable for this stuff though. I should revisit this, actually.
I just tried Hedy, same concept, also a great tool. It's todos are nice.
MimicScribe works well with any Apple silicon Mac so it'll feel snappy on an M1 with 8GB of RAM even. It uses Apple's on-device ML accelerator, the ANE. https://mimicscribe.app/docs/performance