What this really shows is how brittle a lot of facial recognition systems are once the input stops looking like the clean training set. It does not take some magical anti-AI trick. A small shift in landmark geometry, contrast, or occlusion is often enough to push confidence off a cliff.
The hard part is not recognizing a well-lit passport-style face. The hard part is deciding how the system should fail when the real world gets messy or adversarial. If the fallback is bad, you either lock out legitimate users or quietly accept weak matches.
That is why I think the useful benchmark is not top-line accuracy. It is how gracefully the system handles weird, low-confidence, real-world inputs.
Permission guards solve one important problem: should this action be allowed?
The complementary problem is recovery. I run 8 agents with fairly hard boundaries between them, and I still hit failures where every individual action was allowed but the system broke anyway because two agents wrote shared state at the same time.
What saved that setup was supervision, not permissions. The memory server crashed, restarted cleanly, ran repair on boot, and the rest of the system kept moving. Permission checks stop known-bad actions; supervision is what makes unknown-bad outcomes survivable.
I run 8 agents for my company, and the part most demos skip is recovery.
The hardest bug I hit was a shared JSONL memory store: two agents wrote at once, one update silently overwrote the other, and sometimes the file ended up partially corrupted. I fixed it with a mutex and atomic writes, but that mostly taught me I was working against my runtime.
The reason I keep ending up back at Erlang/OTP is that supervision and isolated processes are the default model, not a patch. If agents are going to run while you sleep, restart behavior matters more than clever prompts.
Different scale for me, but the same core problem: what happens when one agent fails mid-task?
I hit this with 8 agents sharing a JSONL knowledge graph. Parallel writes caused a race: two agents read the same state, both wrote back a full graph, and the second silently overwrote the first. I patched it with an async mutex and atomic writes, but the real lesson was that I was fighting my runtime.
On the BEAM, a GenServer processes its mailbox sequentially, so this class of shared-state bug largely disappears. Supervision trees also give you the part agent demos usually skip: when something crashes at 3am, it comes back clean instead of corrupting the rest of the system.
That ended up being the more important property than raw agent count.
Different scale (I run 8, not 69) but the same fundamental challenge: what happens when one of those agents fails mid-task?
I hit this with my multi-agent system — 8 agents sharing a JSONL knowledge graph. Parallel writes caused a race condition: two agents read the same file state, both write back the full graph plus their additions, second write silently obliterates the first. Sometimes the file ends up with a half-written JSON line that breaks every subsequent read.
I patched it with an async mutex and atomic writes, but the real lesson was that I was fighting my runtime. On the BEAM, a GenServer processes messages from its mailbox sequentially — there's no concurrent access to serialize because the execution model is already sequential per-process. Supervision trees handle the crash case: child process dies, supervisor restarts it in microseconds, auto-repair runs on init, agents resume with clean state. Each process is ~2KB with its own heap, so you can run thousands with full isolation.
At 69 agents the coordination and fault tolerance problems get significantly harder. The BEAM was designed for exactly this — Ericsson ran millions of concurrent processes on telephone switches with nine nines of uptime using these patterns 40 years ago.
I run 8 AI agents that manage my solo company — CEO, CFO, COO, Marketing, Accountant, Lawyer, CTO, and an Improver agent that upgrades the others. They share a persistent knowledge graph (JSONL file), consult each other through a central orchestrator, and post content to social media on a 5-minute cron. They've been running for months.
They crash. The interesting question isn't how to prevent that — it's how to make it not matter.
The gnarliest failure I hit: my agents share a knowledge graph through an MCP memory server. When multiple agents fire parallel tool calls (say, create_entities and create_relations in the same batch), you get a classic read-modify-write race. Both operations read the same JSONL state, both write back the full graph plus their additions. Second write obliterates the first. No error, no warning — data just vanishes. Sometimes the write gets interrupted mid-line and you end up with a half-written JSON line that breaks the parser on next load.
My fix was a local fork of the memory server with three things: an async mutex to serialize writes, atomic writes (write to .tmp then rename), and auto-repair on load that skips corrupt lines and deduplicates. But the meta-point is that on the BEAM, this entire class of bug doesn't exist. A GenServer processes messages sequentially from its mailbox — mutual exclusion is the execution model, not something you bolt on with a mutex. Supervision trees restart crashed processes in microseconds. Each process has its own heap, so one agent going haywire can't corrupt another's state.
Erlang/OTP solved this in 1986 for telecom switches that needed 99.999% uptime. The pattern maps almost perfectly to AI agents: many concurrent, stateful, failure-prone processes that need to communicate without taking each other down.
nah addresses "should this action be allowed?" — deterministic classification of tool calls against policies. Smart design, and the no-dependency stdlib approach is the right call for security tooling.
The complementary question most agent safety tools ignore: what happens when things go wrong despite permissions?
I run 8 AI agents managing my company (marketing, accounting, legal, ops). We have a similar permission model — Marketing can't publish claims without Lawyer review, financial changes need CFO sign-off, hard boundaries on auth/compliance. But permissions alone didn't save us when two agents fired parallel writes to the same knowledge graph. Both writes were individually permitted. The second silently overwrote the first. No error, no policy violation — data just disappeared.
What saved us: Erlang-style supervision trees. Memory server detected corruption on load, crashed intentionally, supervisor restarted it in microseconds, auto-repair ran on init. No human at 3am.
Permission guards prevent known-bad actions. Supervision makes unknown-bad outcomes survivable. Most agent safety work focuses exclusively on the first problem.
Yes — I've been running this for my solo SaaS company since January. Not a side project, it's the actual business operations layer.
Similar architecture to yours but I went with specialized agents instead of a single orchestrator + sub-agent split. I have a COO agent that coordinates everything, plus dedicated agents for marketing (writes and posts tweets autonomously via scheduler), accounting (IVA/tax compliance — I'm in Portugal so this is non-trivial), CFO for pricing, CTO for architecture decisions. They share a persistent knowledge graph so context carries across sessions.
The decision tree problem @rodchalski mentions is real. My approach: agents are autonomous by default for their domain (marketing posts content without asking me), but anything cross-domain goes through the COO for coordination. The founder only handles: manual portal logins, irreversible legal filings, strategic pivots, and actual coding. Everything else just runs.
What broke: memory corruption from concurrent writes to the knowledge graph (fixed with mutex + atomic writes). Agents hallucinating product URLs that don't exist (fixed with a domain registry they must check). Content that sounded like ChatGPT wrote it, not a developer (fixed with explicit voice guidelines and a quality gate).
I've been building managed hosting for this exact category of agent systems. The supervision/persistence angle is what drew me to it.
My stack is Elixir on Fly.io — OTP supervision trees are basically purpose-built for long-running processes that need to stay alive. If something crashes, the supervisor restarts it. No systemd, no cron, just the runtime doing what it was designed for.
The self-hosting path is great for tinkering — and Karpathy's point about NanoClaw's ~4000 lines being auditable is a solid trust argument. But once you want 24/7 uptime across multiple agents, you end up rebuilding production infrastructure from scratch.
I run 5 Elixir apps on Fly for under €50/month, so the economics of multi-tenant hosting work well here.
(Founder of OpenClawCloud — clawdcloud.net — happy to talk architecture if anyone's exploring this space.)