As agents run longer and spend more money, many agent frameworks are adding resiliency features like checkpoint recovery and pause-resume approvals.
But to get your agent to production, checkpointing is not enough. There is quite a big gap left for you to handle: failure detection, automatic retries, high availability, scale-out, idempotency, concurrency, session coordination, versioning, ...
I wrote a blog post on what’s left to solve, and how to solve it.
TL;DR Instead of tying resiliency together with your agent framework, agents should be built on top of a highly-available orchestration layer that owns the end-to-end execution, guarantees it completes, and handles all of the points above.
Optionally, agent frameworks can be used on top of this to help abstracting away the agent loop.
Is this also how you see it and productionize your agents?
AI agents often run for minutes, hours, or even days. This makes deploying new versions tricky: what happens if an agent starts on one version of your code and resumes later after you’ve deployed another?
If you swap out code from under an agent, execution can break. Or worse, a changed description or tool implementation can cause your agent to silently misinterpret its own history and produce inconsistent results.
This blog post introduces how to solve this via immutable deployments and pinned executions:
- Each deployment lives at a unique endpoint and represents an immutable, versioned snapshot of your code, prompts, tools, and schemas
- Every execution is pinned to its deployment; retries, resumptions, and callbacks always return to the same version
- New requests route to latest
The blog post shows how Restate implements this in practice. Versioning becomes an infrastructure property you don’t need to think about, rather than something you solve in code.
But to get your agent to production, checkpointing is not enough. There is quite a big gap left for you to handle: failure detection, automatic retries, high availability, scale-out, idempotency, concurrency, session coordination, versioning, ...
I wrote a blog post on what’s left to solve, and how to solve it.
TL;DR Instead of tying resiliency together with your agent framework, agents should be built on top of a highly-available orchestration layer that owns the end-to-end execution, guarantees it completes, and handles all of the points above.
Optionally, agent frameworks can be used on top of this to help abstracting away the agent loop.
Is this also how you see it and productionize your agents?