Building Privacy-First AI Agents on Ollama: Complete Guidenativemind.app2 points·by briansun·há 10 meses·0 comments
Ask HN: Why aren't local LLMs used as widely as we expected?5 points·by briansun·há 10 meses·10 comments
briansun·há 10 meses·discussThanks for the view from a very privacy‑sensitive environment — agreed that hosted SOTA still leads on broad capability.Could you share a quick split: which tasks truly require hosted SOTA than open‑weight? I think gpt-oss is quite good for a lot of things.SMBs can’t get enterprise contracts with OpenAI/Anthropic, so local/open‑weight may be their only viable path — or wait for a hybrid plan.
briansun·há 10 meses·discussWouldn't it be cool to have a local AI agent? It could access search engines and browse any website through a headless browser.
briansun·há 10 meses·discussThanks — I agree with your three big pain points: quality vs hosted SOTA, token speed, and economics/utilization.Have you run into cases where on‑device still makes sense?1. Data that is contractually/regulatorily prohibited from being sent to any third‑party processor (no exceptions).2. Very large datasets where throughput can be low (overnights acceptable) but the cost is high for cloud models.3. Inputs behind a password-wall that hosted assistants/chatgpt/claude can’t reach and can't do agentic things with them.
briansun·há 10 meses·discussWell put. Management overhead + unclear capacity planning kills many pilots.
briansun·há 10 meses·discussTotally fair. On a normal laptop you also need headroom to do your actual job, and KV cache + context length can eat that quickly.
Could you share a quick split: which tasks truly require hosted SOTA than open‑weight? I think gpt-oss is quite good for a lot of things.
SMBs can’t get enterprise contracts with OpenAI/Anthropic, so local/open‑weight may be their only viable path — or wait for a hybrid plan.