Author here. Want to share the results of my experiments with local inference. I failed to get decent output from my previous setup (https://news.ycombinator.com/item?id=48587865). But this stack is performing much better.
The local stack only became reliable after fixing multiple separate failures in the stack, fixing the coding harness, improving system prompt and other tweaks in different components.
Concrete repo evidence in the post is from Clawrium(https://github.com/ric03uec/clawrium) PRs #855, #859, and #861. Will continue to iterate on this till I can offload about 30% of the project work completely to the local models.
Author here. Quick context the post doesn't quite spell out:
The tool_choice="auto" failure on Qwen3-Next isn't a parser issue — the model
reasons inside <think>, decides, and never emits the tool call. No error, just
empty tool_calls. The fix was swapping the backbone from Thinking to Instruct,
not tuning any parser flag.
The "load the bigger model first, size the smaller against actual residency"
playbook generalizes to anything with shared CUDA framework overhead. The ~5 GiB
framework floor shows up even at small gpu_memory_utilization values — plan
against actuals, not targets.
Building Clawrium (https://github.com/ric03uec/clawrium), a CLI to orchestrate a fleet of autonomous agents(like openclaw, hermes etc) on multiple physical hosts.
As I started running agents on my home network I needed an easy way to maintain them without sshing every every time. I built clawrium as a way to centralize config management, secrets, templates and llm providers. Think of it like a control plane similar to kubectl.
I have 2 OpenClaws running on an old Ubuntu box, and one on a PI. Every config change meant SSHing into each one, tweaking files, restarting services, checking logs. Its all fun during experimentation but now I'm using these for some real work and don't want to spend time troubleshooting config drifts.
So I've been building Clawrium (clm) - a CLI that treats your machines as a fleet and your agents as the things you deploy onto them. Point it at a host, deploy an agent, and you get one command center instead of three terminal tabs.
What works today:
- `clm` CLI to configure instances, LLM providers, IP addresses etc
- End-to-end install + lifecycle management for OpenClaw (Zeroclaw in progress)
- Basic TUI to list all the running agent instances in one place
- Normalized config format; Clawrium translates to Openclaw native format but I intend to add support for all *Claws
What's coming: ZeroClaw, Hermes and other claw support, token-usage guardrails across the fleet, config import/export, backups and other goodies. I'm the solo dev on this so I'm building the features based on my immediate needs.
It's built on Python + Typer + ansible-runner.
Still early and rough around the edges. Would love feedback from anyone running agents across multiple machines, especially on what the CLI ergonomics should feel like.
I'm running a few instances of OpenClaw(and Zeroclaw). I use it for a few things
1. Experimenting with local models. OpenClaw uses local inference on a DGX Spark machine. Helps me understand how different, newer models work and behave
2. Bot 1 helps me maintain a personal project. It sits in discord channel and I throw ideas at it, ask it to log issues, give me priorities, update docs etc. Basically maintenance work to manage the repo
3. Bot 2 is general purpose assistant to take notes(while driving, on a walk, etc), give me daily AI news summary from trusted sources and my relevance criteria. Give me suggestions on what should I dig deeper
4. Bot 3 is setup for work environment. Sits in slack as my team's assistant. Process meeting notes automaticlaly from google and send us summary for the week/month, important notes from a project, answer questions for rest of the org and more. This is most "real" use case so far for me
<plug>
I build https://github.com/ric03uec/clawrium as a lightweight orchestration system to manage agents on a local network because i was having a hard time keeping tabs on these agents. all the ssh-ing and config file gymnastics was too much to handle.
</plug>
I'm a heavy cc user for writing code, reviewing documentation, brainstorming, updating jira tickets etc etc.
For the past few months, I started experimenting with managing a team using cc. As a team, we got together and decided to experiment with a new way to run the team and now that we're looking at some good results, I wanted to share our learnings here
The local stack only became reliable after fixing multiple separate failures in the stack, fixing the coding harness, improving system prompt and other tweaks in different components.
Concrete repo evidence in the post is from Clawrium(https://github.com/ric03uec/clawrium) PRs #855, #859, and #861. Will continue to iterate on this till I can offload about 30% of the project work completely to the local models.