Thanks for the thoughtful read. You’ve described exactly the maturity stage we’re targeting: past demos, dealing with retries, partial failures, side effects, and the need for real control once systems are live.
On your questions:
1. Debugging and replay for stateful workflows
We capture step-level execution snapshots across the workflow. Each snapshot records inputs, outputs, duration, tokens, cost, evaluated policies, triggered policies, and the resolution (approved, blocked, overridden).
For enforcement-specific debugging, each snapshot includes which policy matched, what content triggered it, and how it was resolved. When a downstream step fails because an upstream step was blocked or modified, you can trace the execution timeline and see exactly where and how the data flow changed.
We also support human in the loop pause and resume. A step can be paused for approval and later resumed, with the decision and rationale recorded as part of the execution history.
This is not full deterministic replay yet, meaning re-running with identical LLM outputs, but it provides enough visibility to answer “what happened” and “why” in production, which covers most real debugging scenarios.
2. Latency overhead at scale
We operate in two modes depending on requirements:
- Compliance mode: policy violations and blocked requests are written synchronously before returning. This adds a few milliseconds for violation cases, but guarantees the audit record exists before the caller sees the result.
- Performance mode: audit writes are queued asynchronously. Policy evaluation still happens inline, since it may block execution, but persistence is decoupled using bounded queues and worker goroutines.
Most policies are rule-based and pattern matching rather than LLM calls. In practice, teams see single-digit millisecond overhead per request for typical policy sets. Heavier redaction or more complex policies can increase this, but the behavior remains predictable.
Observe-only mode adds essentially no latency beyond the audit write, since no blocking decisions are made.
On orchestration boundaries:
AxonFlow does not require replacing your existing orchestrator. Most teams keep LangChain, LangGraph, or CrewAI for stateful workflow execution and use AxonFlow as a step-level control plane, adding policy gates before each step runs.
For teams building from scratch or wanting tighter integration, AxonFlow can also handle orchestration end to end with governance built in. In practice, most start by adding governance to existing workflows and only consider deeper orchestration later.
We intentionally require explicit tool and connector registration rather than doing blind network layer interception.
The reason is deliberate: we want semantic understanding of what a call represents, not just raw HTTP payloads. When a connector is registered, AxonFlow knows the operation type (database, external API, internal service), can parse the intent, and apply phase aware policies such as blocking dangerous operations before execution, redacting sensitive fields from responses, or enforcing connector specific rate limits.
For arbitrary HTTP integrations, we provide a generic HTTP connector (platform/connectors/http/) that can be configured against any REST endpoint. You register the base URL, auth mechanism (bearer, basic, API key, OAuth2), and headers. From that point on, all requests flow through the same policy engine used for database and first class connectors.
Trade offs versus a pure proxy approach:
- Our approach requires explicit setup. In return, governance is richer: semantic policy enforcement, field level redaction, connector scoped rate limits, and clearer audit trails.
- A proxy based approach has near zero setup and full payload visibility, but enforcement has to reason over raw HTTP without understanding what the call actually does.
On the Stripe or Twilio “what was actually sent?” debugging problem you mentioned: with our model, the connector captures request, response, status, latency, and policy decisions in the audit log. You can see exactly what payload caused a failure, as long as the call was routed through AxonFlow.
The boundary issue you raised is real. Calls that bypass the connector layer (direct SDK usage, internal RPCs) are invisible to us. We are explicit with teams about this: if you want governance and auditability, the call needs to traverse the control plane. Anything outside that path is outside enforcement.
Curious whether you are seeing teams combine both patterns in practice: raw visibility layers for debugging, plus a semantic enforcement layer for policy. They seem to solve adjacent but distinct problems.
This is a very accurate framing. We see the same failure mode in practice.
You are right that enforcement is hard if you cannot first see what is actually happening at runtime.
How we think about the observability to enforcement gap:
In practice, most teams start with visibility and only add enforcement once they trust the signal.
AxonFlow is designed to collapse that loop by providing both in the same system. The same instrumentation that produces the audit trail can also gate execution.
What AxonFlow provides today:
- Full request-level capture for LLM calls and tool calls that flow through AxonFlow, including inputs, outputs, policy decisions, latencies, tokens, and cost.
- A complete audit log with step boundaries, policy evaluation results, and decision context, so teams can answer what happened and why it was allowed or blocked. Critical violations such as blocks and policy triggers are logged synchronously, while successful calls are queued asynchronously for performance.
- Operational controls like timeouts, approval gates, and the ability to block or require human review at a specific step.
- Two integration modes: Proxy Mode, where AxonFlow sits inline in the request path, and Gateway Mode, where it acts as a policy check before and an audit capture layer after existing LLM calls.
What we do not assume:
- We do not assume teams already have clean visibility into all agent calls.
- We do assume that for enforcement to be meaningful, the calls you want to govern must traverse AxonFlow. If some calls bypass it, you end up with fragmented audit trails and inconsistent policy enforcement.
How teams adopt it:
- Many teams start by routing a single workflow or a subset of tools through AxonFlow to get an end-to-end trace and audit. Once that is stable, they expand coverage and enable stricter enforcement policies.
- Interesting that you are separating visibility (toran.sh) and policy (keypost.ai). We took a combined approach, but I can see the rationale for decoupling.
If you have a concrete example of the invisible tool call problem you are seeing, such as custom HTTP tools, internal RPC, or database calls, I would be interested in how you are instrumenting it today. That boundary is where many systems break down.
Thanks for taking the time to write this. This is exactly the kind of critique I want.
1) Migration path gateway to proxy
You are right that many enterprise projects start with perimeter checks and then want deeper execution control. We designed gateway mode specifically as the lowest friction on ramp, not as the end state.
The migration path is not "rewrite everything into axonflow.execute_query". The core idea is to keep your orchestration code and incrementally move enforcement points under AxonFlow.
Practically, teams start with gateway mode around one workflow for pre check plus audit. Then they adopt step level enforcement by passing step metadata and calling AxonFlow per step. Finally, for workflows where they want deterministic replay, approvals, and full execution trace, they can run those paths in proxy mode while leaving other paths in gateway mode.
So it can be mixed. You do not have to flip the whole system at once.
2) Authoring, versioning, testing, rollout of complex policies
Agree that if policy logic grows beyond simple checks, the system has to support safe change management.
Today policies are treated as a first class config artifact with explicit versioning and audit trail, and we expect teams to test them like code. We are also investing in better tooling here, including policy simulation against recorded traces, staged rollout, and guardrails for regression.
On "dynamic implies DB/cache round trips", the hot path is designed to stay cached, and dynamic policy evaluation is bounded and observable. If a policy needs external calls, that is explicit and should be treated as a production dependency with its own SLO, not something hidden inside the control plane.
3) Benchmarks
Agree. The numbers in the docs are indicative for simple checks, not a substitute for real benchmarks.
We should publish benchmarks that include overlapping policies, cold cache, and more expensive dynamic lookups, and show tail latency percentiles. This is on the near term roadmap because for an inline control plane, p95 and p99 matter more than averages.
4) Multi agent planning versus control plane boundary
This is a fair callout. Our primary goal is governance, not planning.
Proxy mode needs some orchestration primitives to enforce execution safely, for example retries with side effect control, approval gates, and step level auditability.
We are not trying to compete with LangChain or CrewAI on planning sophistication. In most deployments, we expect users to keep their orchestrator and use AxonFlow as a governance layer. The multi agent planning capability exists mainly so teams can start with a governed runtime for simple cases, not as a replacement for full featured agent frameworks.
If you are open to it, I would love to dig into one concrete example from your experience, like what "migration cliff" looked like and which policy types became the bottleneck. Happy to correct any gaps in my understanding too.
Good question. The overhead is designed to be low enough for inline enforcement. For the fast, rule based checks we typically see single digit millisecond evaluation time, and in gateway mode the end to end pre check usually adds around 10 to 15 ms.
You’re right that relative to an LLM call this is usually negligible, but we still treat it seriously because policy checks also sit in front of tool calls and other non LLM operations where latency matters more. That’s why the static checks are compiled and cached and the gateway path is kept tight.
If you want more detail, I have a longer architecture walkthrough that goes into the execution path and performance model: https://youtu.be/hvJMs3oJOEc
Thanks. In practice, access control is enforced centrally by AxonFlow, not delegated to the orchestrator.
Each LLM or tool call is evaluated at execution time against the active policy context, which includes the user, workflow, step, and tenant. That allows different steps in the same workflow to run under different credentials, providers, or cost and permission constraints if needed.
In gateway mode, the orchestrator still issues the call, but AxonFlow pre-authorizes it and records the decision so the policy is enforced consistently. In proxy mode, AxonFlow holds and applies the credentials itself and routes the call to the appropriate provider.
The key point is that credentials and access rules are defined once and enforced centrally, while orchestration logic remains separate.
By deterministic policy enforcement we mean rule-based checks that evaluate to an explicit allow or block decision at execution time. Today that includes a mix of regex-based checks (for example PII patterns), structured detectors, and hard limits or business rules like cost caps, rate limits, and permission constraints. These policies are evaluated inline before model or tool calls, so the outcome is predictable and auditable rather than probabilistic.
On cost tracking: yes, AxonFlow captures per-call metadata including model, tokens, provider, and cost, and attributes it to user, workflow, and tenant. In gateway mode this is per-call audit logging, and in proxy mode it extends across multi-step workflows so you can see cost accumulation per user or execution. We also recently shipped Workflow Control Plane which tracks policy evaluation and cost accumulation across multi-step agent executions, so you get a single audit trail and cost rollup for an entire workflow, not just individual calls. That's been a common pain point we've seen with teams running agents in production.
We intentionally avoid framing guardrails as “X percent confidence” checks on prompts or model output. In practice, probabilistic confidence at the text level has been the weakest place to enforce safety, especially once workflows become multi step and stateful.
AxonFlow’s non human guardrails are primarily deterministic and context grounded rather than model judgment based. Concretely, they focus on:
- authorization checks on actions, tools, and write paths rather than output quality
- permission evaluation per step using actual tool arguments and proposed side effects
- invariant checks on state transitions, for example whether an action is allowed given what the system has observed so far
- policy decisions that can halt execution entirely rather than degrade or retry
We do use probabilistic components in narrow, explicit places such as PII detection or risk classification, but those always feed into a deterministic policy decision. The system never proceeds because a model “seems confident enough.”
Human approval gates are not there because non human guardrails are insufficient in principle. They exist because some actions are intentionally irreversible or high blast radius, and no amount of model confidence should bypass explicit authorization.
So the distinction we draw is less about validating LLM output and more about deciding whether the system is allowed to move forward at all, given the concrete context and constraints at that moment.
The intervention point ended up being more important than we initially expected. Once workflows become multi-step and stateful, the ability to pause, inspect, or halt execution based on context (not just inputs) becomes the difference between “we noticed later” and “we prevented it.”
We also found that logging alone wasn’t enough. Being able to see why a step was allowed to proceed, not just that it did, made post-incident analysis much less speculative.
Curious if you’ve seen similar issues once agents start touching real systems.
Hi HN. When teams move AI agents from demos to production, the failures are rarely about model quality.
They look a lot like classic distributed systems problems.
- Long-running state across multiple steps.
- Partial failures mid workflow.
- Retries that accidentally repeat side effects.
- Permissions that differ per step, not per agent.
- No clean way to stop, inspect, or intervene once execution starts.
Most agent frameworks are optimized for authoring workflows like prompts, tools, and plans. They are much less optimized for operating them once agents touch real systems, data, or users.
That is why teams often end up adding ad hoc layers after the fact. Logging wrappers. Policy checks. Manual approvals. Retry guards. Kill switches.
We built AxonFlow because these problems do not live at the API boundary. They happen inside execution paths.
AxonFlow is a self hosted control plane that sits under your LLM or agent stack and governs execution step by step across LLM calls, tool calls, retries, and approvals, without replacing your existing orchestration framework.
It supports execution aware policy enforcement, not just ingress checks.
Human approval gates for high risk actions.
Deterministic audit logs and replay and debug.
Cost controls and routing primitives.
Gateway mode alongside existing LangChain, CrewAI, or custom stacks.
The community core is source available under BSL 1.1, runs locally, and is fully self hosted with no signup and no hosted dependency.
Optional 2 minute demo showing gateway mode, policy enforcement, and a multi step workflow running locally:
https://youtu.be/WwQXHKuZhxc
I would especially value pushback from folks who have dealt with retries, side effects, permission boundaries, or post incident auditability in real agent workflows.
On your questions:
1. Debugging and replay for stateful workflows
We capture step-level execution snapshots across the workflow. Each snapshot records inputs, outputs, duration, tokens, cost, evaluated policies, triggered policies, and the resolution (approved, blocked, overridden).
For enforcement-specific debugging, each snapshot includes which policy matched, what content triggered it, and how it was resolved. When a downstream step fails because an upstream step was blocked or modified, you can trace the execution timeline and see exactly where and how the data flow changed.
We also support human in the loop pause and resume. A step can be paused for approval and later resumed, with the decision and rationale recorded as part of the execution history.
This is not full deterministic replay yet, meaning re-running with identical LLM outputs, but it provides enough visibility to answer “what happened” and “why” in production, which covers most real debugging scenarios.
2. Latency overhead at scale
We operate in two modes depending on requirements:
- Compliance mode: policy violations and blocked requests are written synchronously before returning. This adds a few milliseconds for violation cases, but guarantees the audit record exists before the caller sees the result.
- Performance mode: audit writes are queued asynchronously. Policy evaluation still happens inline, since it may block execution, but persistence is decoupled using bounded queues and worker goroutines.
Most policies are rule-based and pattern matching rather than LLM calls. In practice, teams see single-digit millisecond overhead per request for typical policy sets. Heavier redaction or more complex policies can increase this, but the behavior remains predictable.
Observe-only mode adds essentially no latency beyond the audit write, since no blocking decisions are made.
On orchestration boundaries:
AxonFlow does not require replacing your existing orchestrator. Most teams keep LangChain, LangGraph, or CrewAI for stateful workflow execution and use AxonFlow as a step-level control plane, adding policy gates before each step runs.
For teams building from scratch or wanting tighter integration, AxonFlow can also handle orchestration end to end with governance built in. In practice, most start by adding governance to existing workflows and only consider deeper orchestration later.
For related discussion on how we think about the observability to enforcement gap, there’s a deeper thread here that may be relevant: https://news.ycombinator.com/item?id=46603800
Happy to go deeper on any of this if useful.