Building Privacy-First AI Agents on Ollama: Complete Guidenativemind.app2 points·by briansun·10 เดือนที่ผ่านมา·0 comments
Ask HN: Why aren't local LLMs used as widely as we expected?5 points·by briansun·10 เดือนที่ผ่านมา·10 comments
briansun·10 เดือนที่ผ่านมา·discussThanks for the view from a very privacy‑sensitive environment — agreed that hosted SOTA still leads on broad capability.Could you share a quick split: which tasks truly require hosted SOTA than open‑weight? I think gpt-oss is quite good for a lot of things.SMBs can’t get enterprise contracts with OpenAI/Anthropic, so local/open‑weight may be their only viable path — or wait for a hybrid plan.
briansun·10 เดือนที่ผ่านมา·discussWouldn't it be cool to have a local AI agent? It could access search engines and browse any website through a headless browser.
briansun·10 เดือนที่ผ่านมา·discussThanks — I agree with your three big pain points: quality vs hosted SOTA, token speed, and economics/utilization.Have you run into cases where on‑device still makes sense?1. Data that is contractually/regulatorily prohibited from being sent to any third‑party processor (no exceptions).2. Very large datasets where throughput can be low (overnights acceptable) but the cost is high for cloud models.3. Inputs behind a password-wall that hosted assistants/chatgpt/claude can’t reach and can't do agentic things with them.
briansun·10 เดือนที่ผ่านมา·discussWell put. Management overhead + unclear capacity planning kills many pilots.
briansun·10 เดือนที่ผ่านมา·discussTotally fair. On a normal laptop you also need headroom to do your actual job, and KV cache + context length can eat that quickly.
Could you share a quick split: which tasks truly require hosted SOTA than open‑weight? I think gpt-oss is quite good for a lot of things.
SMBs can’t get enterprise contracts with OpenAI/Anthropic, so local/open‑weight may be their only viable path — or wait for a hybrid plan.