Ask HN: Who is using local LLMs in a production environment here?
3 comments
The federal government, especially the dod, has adopted local llms. Now, they also have the big iron closed models "locally" so that stretches your definition I'm sure. But they use other models too
Intersting, not my government though as I am in Germany. But are those huge deepseek models worth it? It seems that only proprietary models can match up.
On the other hand, we need to talk specifics. Measure up, how and regarding which benchmark.
On the other hand, we need to talk specifics. Measure up, how and regarding which benchmark.
things are changing too quickly for it to be worth it yet. eventually LLMs won't really increase in capability or resources anymore, and at that point, if the hardware itself isn't becoming more optimized for LLM workloads, you'd see people do this.
I have to say there's one exception for me and that's Whisper. I actually do use Whisper a lot. But I just don't use local LLMs. They're just really, really bad compared to cloud GPUs.
And I don't know why, because for me it seems that having a speech-to-text model is much more challenging to create than just a model that creates text.
But it seems that they really cannot remove the differences and have it run on consumer computers. And so I also go back to cloud LLMs, all privacy aside.