Second here. From recent Alibaba Qwen conference: the all-in-one box (DC in a box - I think I was called Apsara, 0.6x0.6x1.5m) plug and play, 1.5TB GPU RAM, capability to run in a fully air gapped environment, any open models... All of that is roughly $300k one time.
And this box can do non LLM tasks as well.
Performance (throughput) around 20k t/s. Delivery time - around 2 months.
For any medium sized company its perhaps cheaper to just buy it once than spending 1.5k for cloud per user
I have roughly 20-40M token usage per day for GLM only (more if count other models). Using API pricing from OR it means ollama more profitable for me after day (few days if count cache properly).
For several models like Kimi and glm they have b300 and performance really good. At launch I got closer to 90-100 tps. Nowadays it’s around 60 tps stable across most models I used (utility models < 120B almost instant)
The worst I saw - multiple parallel agents (opencode & pi-coding agents), with Kimi and glm, almost non stop development during the work day - 15-20% session consumption (I think it’s 2h bucket) max. Never hit the limit.
In contrast, 20$ Claude in the similar mode I consumed after just few hours of work.
My 50c - ollama cloud 20$. GLM5 and kimi are really competitive models, Ollama usage limits insane high, no limits where to use (has normal APIs), privacy and no logging
In my experience, promotion depends on: (a) is cheaper (including on boarding) to fire and hire or pay more, (b) will it cause domino effect so others will also ask for promotions, (c) will employee find an offer outside (matter of personal traits and market conditions).
Personal performance and achievements are usually secondary.
I tried BIND, Dnsmasq, unbound, adguard.
At the end I picked CoreDNS outside kubernetes. It can do everything (including wildcards and ACL), requires few resources and easy to configure.
Few years ago I felt the same and created trusted-cgi.
However, through the years I learned:
- yes, forks and in general processes are fast
- yes, it saves memory and CPU on low load sites
- yes, it’s simple protocol and can be used even in shell
However,
- splitting functions (mimic serverless) as different binaries/scripts creates mess of cross scripts communication
- deployment is not that simple
- security wise, you need to run manager as root and use unique users for each script or use cgroups (or at least chroot). At that moment the main question is why not use containers asis
Also, compute wise, even huge Go app with hundreds endpoints can fit just few megabytes of RAM - there is no much sense to save so few memory.
At worst - just create single binary and run on demand for different endpoints