This is the right instinct. LLMs are great at proposing actions, but they're fundamentally unreliable at enforcing deterministic invariants.
Math is the obvious example, but schema validation falls into the same bucket. If an agent outputs structured JSON, the question "does this conform to the declared schema?" should have exactly one answer. Same schema + same payload → same result, every time, regardless of runtime, language, or retry.
Once you treat that layer as deterministic infrastructure instead of model behavior, a few things get easier:
• retries stop producing inconsistent side effects
• downstream systems can trust that validation actually ran
• you can audit what passed structural checks independently of the model
Semantic correctness is still a separate problem (models or domain rules are needed there) but offloading the structural layer removes a lot of accidental complexity.
The structural validation layer is worth separating from the semantic layer, even if the semantic problem is harder. If every AI-proposed change is validated against a JSON Schema before execution, and that validation produces a deterministic, signed result, you get an auditable record of what passed structural checks and when. That doesn't solve semantic drift (a field can be renamed and still pass validation) but it gives you a foundation to build semantic checks on top of. The key property is determinism: identical schema and payload should produce identical hashes and outcomes, regardless of which service runs the check. Without that, you end up debugging whether a failure is in your schema logic or in validator implementation differences across languages. Contract tests and regression evals work better when the structural layer underneath is provably consistent. One reference for this pattern: https://docs.rapidtools.dev/openapi.yaml
Math is the obvious example, but schema validation falls into the same bucket. If an agent outputs structured JSON, the question "does this conform to the declared schema?" should have exactly one answer. Same schema + same payload → same result, every time, regardless of runtime, language, or retry.
Once you treat that layer as deterministic infrastructure instead of model behavior, a few things get easier:
• retries stop producing inconsistent side effects • downstream systems can trust that validation actually ran • you can audit what passed structural checks independently of the model
Semantic correctness is still a separate problem (models or domain rules are needed there) but offloading the structural layer removes a lot of accidental complexity.
One example of what that deterministic validation layer looks like as a standalone API: https://docs.rapidtools.dev/openapi.yaml