$2 million a year to run 244,000 searches that advanced 361 cases... That's about $5,500 per useful search.
Meanwhile every car that drove past one of those cameras got logged, timestamped, and stored. These things aren't not law enforcement, they're mass surveillance with a badge.
I settled on a similar workflow but across two agents instead of one session.
One agent writes task specs. The other implements them. Handoff files bridge the gap. The spec IS the session artifact because it captures intent, scope, and constraints before any code gets written.
The plan.md approach people are describing here is basically what happens naturally when you force yourself to write intent before execution.
Tried self-hosting with Mattermost to get around Slacks 90 day free tier history but my team didn't care for it much. Ended up back on Slack's free tier. This may solve that issue for me, will check it out.
Agreed. The non-determinism makes traditional testing basically useless here. You can't write a test suite for "the agent decided to do something unexpected this time." Logging and runtime checks are the only way to catch the weird edge cases.
The automated audit only covers static analysis. When the agent actually runs, hitting MCP servers, making HTTP calls, getting responses back, that's where the real problems show up. Prompt injection through tool responses, malicious libraries that exfiltrate env vars, SSRF from agents that blindly follow redirects. Code audits miss all of it because this is a runtime and network problem, not a code quality problem.
Built Pipelock for this actually. It's a network proxy that sits between the agent and everything it talks to. Still early but the gap is real. https://github.com/luckyPipewrench/pipelock
The maintenance burden is real but I think security is the bigger gap. People vibing out code with AI aren't thinking about input validation or dependency vulnerabilities. They build it, it works, they ship it. Then they're running unpatched code with no security review. That's where things get ugly.