briansun

10 maanden geledendiscuss

Thanks for the view from a very privacy‑sensitive environment — agreed that hosted SOTA still leads on broad capability.

Could you share a quick split: which tasks truly require hosted SOTA than open‑weight? I think gpt-oss is quite good for a lot of things.

SMBs can’t get enterprise contracts with OpenAI/Anthropic, so local/open‑weight may be their only viable path — or wait for a hybrid plan.

briansun

10 maanden geledendiscuss

Wouldn't it be cool to have a local AI agent? It could access search engines and browse any website through a headless browser.

briansun

10 maanden geledendiscuss

Thanks — I agree with your three big pain points: quality vs hosted SOTA, token speed, and economics/utilization.

Have you run into cases where on‑device still makes sense?

1. Data that is contractually/regulatorily prohibited from being sent to any third‑party processor (no exceptions).

2. Very large datasets where throughput can be low (overnights acceptable) but the cost is high for cloud models.

3. Inputs behind a password-wall that hosted assistants/chatgpt/claude can’t reach and can't do agentic things with them.

briansun

10 maanden geledendiscuss

Well put. Management overhead + unclear capacity planning kills many pilots.

briansun

10 maanden geledendiscuss

Totally fair. On a normal laptop you also need headroom to do your actual job, and KV cache + context length can eat that quickly.

Submissions

Building Privacy-First AI Agents on Ollama: Complete Guide

Ask HN: Why aren't local LLMs used as widely as we expected?

comments