Building Privacy-First AI Agents on Ollama: Complete Guidenativemind.app2 points·by briansun·hace 10 meses·0 comments
Ask HN: Why aren't local LLMs used as widely as we expected?5 points·by briansun·hace 10 meses·10 comments
briansun·hace 10 meses·discussThanks for the view from a very privacy‑sensitive environment — agreed that hosted SOTA still leads on broad capability.Could you share a quick split: which tasks truly require hosted SOTA than open‑weight? I think gpt-oss is quite good for a lot of things.SMBs can’t get enterprise contracts with OpenAI/Anthropic, so local/open‑weight may be their only viable path — or wait for a hybrid plan.
briansun·hace 10 meses·discussWouldn't it be cool to have a local AI agent? It could access search engines and browse any website through a headless browser.
briansun·hace 10 meses·discussThanks — I agree with your three big pain points: quality vs hosted SOTA, token speed, and economics/utilization.Have you run into cases where on‑device still makes sense?1. Data that is contractually/regulatorily prohibited from being sent to any third‑party processor (no exceptions).2. Very large datasets where throughput can be low (overnights acceptable) but the cost is high for cloud models.3. Inputs behind a password-wall that hosted assistants/chatgpt/claude can’t reach and can't do agentic things with them.
briansun·hace 10 meses·discussWell put. Management overhead + unclear capacity planning kills many pilots.
briansun·hace 10 meses·discussTotally fair. On a normal laptop you also need headroom to do your actual job, and KV cache + context length can eat that quickly.
Could you share a quick split: which tasks truly require hosted SOTA than open‑weight? I think gpt-oss is quite good for a lot of things.
SMBs can’t get enterprise contracts with OpenAI/Anthropic, so local/open‑weight may be their only viable path — or wait for a hybrid plan.