Ask HN: How are you controlling AI agents that take real actions?
23 comments
Hard-coded checks before every action, plus a governance layer that separates "what the agent wants to do" from "what it's allowed to do."
The deeper issue: if your agent decides whether to issue a refund, you're solving the wrong problem with prompt guards. A refund is a deterministic business rule — order exists, within return window, amount matches. That decision shouldn't be made by an LLM at all.
In my setup, agents propose actions and write structured reports. A deterministic quality advisory then runs — no LLM involved — producing a verdict (approve, hold, redispatch) based on pre-registered rules and open items. The agent can hallucinate all it wants inside its context window, but the only way its work reaches production is through a receipt that links output to a specific git commit, with a quality gate in between.
For anything with real consequences (database writes, API calls, refunds), the pattern is: LLM proposes → deterministic validator checks → human approves. The LLM never has direct write access to anything that matters.
"Just hoping for the best" works until it doesn't. We tracked every agent decision in an append-only ledger — after a few hundred entries, you start seeing exactly where and how agents fail. That pattern data is more useful than any prompt guard.
In my setup, agents propose actions and write structured reports. A deterministic quality advisory then runs — no LLM involved — producing a verdict (approve, hold, redispatch) based on pre-registered rules and open items. The agent can hallucinate all it wants inside its context window, but the only way its work reaches production is through a receipt that links output to a specific git commit, with a quality gate in between.
For anything with real consequences (database writes, API calls, refunds), the pattern is: LLM proposes → deterministic validator checks → human approves. The LLM never has direct write access to anything that matters.
"Just hoping for the best" works until it doesn't. We tracked every agent decision in an append-only ledger — after a few hundred entries, you start seeing exactly where and how agents fail. That pattern data is more useful than any prompt guard.
> A refund is a deterministic business rule — order exists, within return window, amount matches. That decision shouldn't be made by an LLM at all.
I feel like this is the real key. LLMs are good at some things and bad at others. Deterministic logic (e.g. don't ever do "x") is not one of them.
I feel like this is the real key. LLMs are good at some things and bad at others. Deterministic logic (e.g. don't ever do "x") is not one of them.
The separation between 'what the agent wants to do' and 'what it's allowed to do' is the right mental model.
The append-only ledger point is underrated too — pattern data from real failures is worth more than any upfront rule design.
How long did it take to build and maintain that governance layer? And as your agent evolves, do the rules keep up or is that becoming its own maintenance burden?
The append-only ledger point is underrated too — pattern data from real failures is worth more than any upfront rule design.
How long did it take to build and maintain that governance layer? And as your agent evolves, do the rules keep up or is that becoming its own maintenance burden?
That’s exactly the mental split we’ve been leaning on.
The ledger part turned out to be more useful than we expected. Every freeze/reject event becomes a concrete example of where the agent tried to do something inadmissible, which is much more informative than hypothetical rule design.
On the governance layer: for us keeping the core extremely small and deterministic is proving interesting. The gate itself doesn’t try to understand intent or policy: it only enforces mechanical invariants like sequencing, replay resistance and bounded actions.
So when the agent evolves, we’re mostly not changing the kernel. What changes are the constraints around it (things like ceilings, roles, or context updates). That keeps the maintenance burden manageable because the core logic doesn’t grow with the agent’s complexity.
Early days though the real test will be how it behaves once the agents start doing more varied workflows.
The ledger part turned out to be more useful than we expected. Every freeze/reject event becomes a concrete example of where the agent tried to do something inadmissible, which is much more informative than hypothetical rule design.
On the governance layer: for us keeping the core extremely small and deterministic is proving interesting. The gate itself doesn’t try to understand intent or policy: it only enforces mechanical invariants like sequencing, replay resistance and bounded actions.
So when the agent evolves, we’re mostly not changing the kernel. What changes are the constraints around it (things like ceilings, roles, or context updates). That keeps the maintenance burden manageable because the core logic doesn’t grow with the agent’s complexity.
Early days though the real test will be how it behaves once the agents start doing more varied workflows.
About 6 months of iterating, but in bursts — I built it while using it on a production project, so the governance layer grew alongside real failure modes rather than being designed upfront.
The maintenance question is the right one. The rules themselves are low-maintenance because they're deliberately simple and deterministic — file size limits, test coverage thresholds, blocker counts. They don't need updating when the model changes because they don't depend on LLM behavior.
What does evolve is the dispatch templates — how I scope tasks and what context I give agents upfront. That's where the ledger pays for itself. After 1100+ receipts, I can see patterns like "tasks scoped above 300 lines fail 3x more often" or "planning gates without explicit deliverables always need redispatch." Those patterns feed back into how I write dispatches, not into the rules themselves.
So the rules stay stable, but the way I use the system keeps improving. The governance layer is the boring part — the interesting part is the feedback loop from receipts to dispatch quality.
The maintenance question is the right one. The rules themselves are low-maintenance because they're deliberately simple and deterministic — file size limits, test coverage thresholds, blocker counts. They don't need updating when the model changes because they don't depend on LLM behavior.
What does evolve is the dispatch templates — how I scope tasks and what context I give agents upfront. That's where the ledger pays for itself. After 1100+ receipts, I can see patterns like "tasks scoped above 300 lines fail 3x more often" or "planning gates without explicit deliverables always need redispatch." Those patterns feed back into how I write dispatches, not into the rules themselves.
So the rules stay stable, but the way I use the system keeps improving. The governance layer is the boring part — the interesting part is the feedback loop from receipts to dispatch quality.
6 months and 1100+ receipts to get to useful patterns — that's the hidden cost nobody talks about. The governance layer is 'boring' but it's also 6 months you're not spending on the actual agent. That feedback loop from receipts to dispatch quality is exactly what we're building as infrastructure so teams don't start from zero.
Fair point on the time cost — but I'd frame it differently. The 6 months wasn't spent building a governance layer instead of building the agent. The governance layer grew out of the actual project work. Every receipt, every quality rule, every dispatch pattern was a direct response to something that broke in production. Day one I had zero governance and a working agent. By month six I had 1100+ receipts and a system that catches failures before they ship.
The infrastructure approach makes sense for teams who want to skip the learning curve. The trade-off is that pre-built governance rules are generic by definition — they can't know that your specific codebase breaks when tasks exceed 300 lines, or that planning gates without explicit deliverables always need redispatch. That pattern data only comes from running your own agents on your own work.
Curious what you're building — is it the ledger/tracking layer, the quality gates, or the full orchestration?
The infrastructure approach makes sense for teams who want to skip the learning curve. The trade-off is that pre-built governance rules are generic by definition — they can't know that your specific codebase breaks when tasks exceed 300 lines, or that planning gates without explicit deliverables always need redispatch. That pattern data only comes from running your own agents on your own work.
Curious what you're building — is it the ledger/tracking layer, the quality gates, or the full orchestration?
we're building the platform that manage all policies of the agent
check out our launch post https://news.ycombinator.com/item?id=47146354
check out our launch post https://news.ycombinator.com/item?id=47146354
Nice — just checked it out. The interceptor approach makes sense for
teams that need policy enforcement across multiple agents.
Interesting difference in philosophy though: Limits enforces rules defined upfront, while what I built learns rules from production receipts. After 1100+ task completions, the dispatch patterns look completely different from what I would have designed on day one.
Probably complementary — you'd want both. Pre-defined guardrails for the dangerous stuff (your approach), and pattern evolution for the quality/efficiency stuff (mine).
Interesting difference in philosophy though: Limits enforces rules defined upfront, while what I built learns rules from production receipts. After 1100+ task completions, the dispatch patterns look completely different from what I would have designed on day one.
Probably complementary — you'd want both. Pre-defined guardrails for the dangerous stuff (your approach), and pattern evolution for the quality/efficiency stuff (mine).
I've been approaching this from a slightly different angle: treating the problem less as "agent alignment" and more as an execution boundary problem.
Instead of trying to force the model to behave via prompts or policies, we assume the model will eventually propose something unsafe. The trick is making sure it can't commit irreversible actions directly.
So the pattern we've been experimenting with is:
agent proposes an action
proposal goes through a deterministic gate
gate checks things like replay, state advancement, spend ceilings, etc.
only then does the real-world action execute
In practice this looks more like a transaction firewall than a prompt guardrail.
The LLM can reason however it wants, but anything that changes real state (payments, DB writes, API calls) has to pass through the gate.
It doesn't solve the reasoning problem, but it makes the commit boundary deterministic, which removes a lot of the scary failure modes like duplicate actions or retries gone wild.
Still early experiments, but the model behaving badly becomes much less dangerous if it literally can't execute without passing the boundary.
Instead of trying to force the model to behave via prompts or policies, we assume the model will eventually propose something unsafe. The trick is making sure it can't commit irreversible actions directly.
So the pattern we've been experimenting with is:
agent proposes an action
proposal goes through a deterministic gate
gate checks things like replay, state advancement, spend ceilings, etc.
only then does the real-world action execute
In practice this looks more like a transaction firewall than a prompt guardrail.
The LLM can reason however it wants, but anything that changes real state (payments, DB writes, API calls) has to pass through the gate.
It doesn't solve the reasoning problem, but it makes the commit boundary deterministic, which removes a lot of the scary failure modes like duplicate actions or retries gone wild.
Still early experiments, but the model behaving badly becomes much less dangerous if it literally can't execute without passing the boundary.
Just treat the LLM as an NLP interface for data input. Still run the inputs against a deterministic heuristic for whether the action is permitted (or depending on the context, even for determining what action is appropriate).
LLMs ignore instructions. They do not have judgement, just the ability to predict the most likely next token (with some chance of selecting one other than the absolutely most likely). There’s no way around that. If you need actual judgement calls, you need actual humans.
LLMs ignore instructions. They do not have judgement, just the ability to predict the most likely next token (with some chance of selecting one other than the absolutely most likely). There’s no way around that. If you need actual judgement calls, you need actual humans.
Exactly right - the deterministiclayer is the only thing you can actually trust.
We landed on the same pattern: LLM handles the understanding, hard rules handle the permission. The tricky part is maintaining those rules as the agent evolves. How are you managing rule updates code changes every time or something more dynamic?
We landed on the same pattern: LLM handles the understanding, hard rules handle the permission. The tricky part is maintaining those rules as the agent evolves. How are you managing rule updates code changes every time or something more dynamic?
> Prompt instructions like "never do X" don't hold up. LLMs ignore them when context is long or users push hard.
Serious question. Assuming you knew this, why did you choose to use LLMz for this job?
Serious question. Assuming you knew this, why did you choose to use LLMz for this job?
Fair. We didn't choose LLMs to enforce rules — we chose them to understand intent. The enforcement happens outside the LLM entirely. That's the separation that actually holds up in production
> we chose them to understand intent
Yet they don't understand the intent of "Never do X" ?
Yet they don't understand the intent of "Never do X" ?
Prompt guardrails are theater - they work until they don't. We ended up building sandboxed execution for each agent action. Agent proposes what it wants to do, but execution happens in an isolated microVM with explicit capability boundaries. Database writes require a separate approval step architecturally separate from the LLM context.
Worth looking at islo.dev if you want the sandboxing piece without building it yourself.
Worth looking at islo.dev if you want the sandboxing piece without building it yourself.
Sandboxed execution is solid for isolation — separating proposal from execution is the right architecture. The piece we kept hitting was the policy layer on top: who defines what the agent is allowed to propose in the first place, and how do you update those rules without a redeploy every time?
MidasTools(1)
Prompt instructions like "never do X" don't hold up. LLMs ignore them when context is long or users push hard.
Curious how others are handling this: - Hard-coded checks before every action? - Some middleware layer? - Just hoping for the best?
We built a control layer for this — different methods for structured data, unstructured outputs, and guardrails (https://limits.dev). Genuinely want to learn how others approach it.