Yes — that layer is part of the runtime design. The AI never mutates structure directly. It only proposes a DSL change, which goes through a deterministic compile pipeline before it becomes canonical.
Schema evolution is treated as a runtime operation with versioning, migration logs, and a deterministic state hash (dslHash). Every compile produces a new schema version and writes a structured change plan to dsl_change_log, so structural mutations are fully auditable.
There’s also a cryptographic validation attestation step: the compiled DSL is hashed and attested so the runtime can verify that the schema being executed is exactly the one that passed the pipeline. That prevents unauthorized structural drift outside the compiler path.
Breaking changes, data compatibility, and migrations are evaluated before commit, so structural mutations are gated much like data mutations.
The stack to support this is admittedly quite complex, but most of that complexity lives in the runtime so the AI-facing interface can remain simple and safe. The big shift was treating AI as proposing structure, but never owning execution.
One thing I'm still unsure about is where the long-term governance layer should live once models can mutate system structure — inside the runtime itself, or higher up at the application/policy level. Curious how others are thinking about that boundary.
Good point. In the model I'm experimenting with, identity and permissions end up being part of the runtime primitives rather than external services.
In traditional SaaS architectures identity is usually handled by separate layers (auth providers, middleware, RBAC checks in APIs). The problem is that when the system structure itself becomes mutable — entities, workflows, dashboards — those checks are no longer enough.
If an AI proposes structural changes, the runtime needs to reason about who is allowed to modify which parts of the semantic model.
So identity becomes part of the execution semantics of the system rather than just authentication.
Conceptually it looks closer to something like:
identity → permissions → allowed state transitions
instead of just:
identity → API access
This is also why the runtime primitives include things like entities, relations, transitions and permissions.
The runtime can then enforce invariants such as:
- which roles can evolve parts of the model
- which workflows can trigger actions
- which datasets a component can bind to
So the trust boundary ends up inside the runtime rather than in the surrounding services.
Still exploring the design space here, but it seems necessary once AI can propose structural mutations to the system.
For context, I'm experimenting with applying this architecture to a CRM system where the entire application is defined as a semantic JSON model executed by a runtime.
The LLM proposes structural changes (entities, workflows, metrics), while the runtime enforces invariants and deterministic execution.
This is also why we’re building it as a compiler pipeline rather than “prompting harder”: the output is an auditable artifact (canonical DSL + hash + migration checksum) that makes contract tests/regression diffs trustworthy.
Next step is what you call out: a signed attestation per compile (schemaHash + pinned validator/version/options + canonicalHash), so identical input yields identical outcomes across services/languages.
You’re right it doesn’t solve semantic drift alone (renames can still validate), but once the deterministic floor exists we can layer semantic checks + rename-safe migrations on top without debugging the toolchain itself.
This makes a lot of sense — especially the “LLMs as nondeterministic services” framing. I agree that versioned prompts, schema validators, regression evals, and contract tests are essential if you're shipping LLM-powered systems.
What I’m wrestling with is a slightly different failure mode though.
Even if:
prompts are versioned
outputs conform to JSON schema
contract tests pass
…you can still end up with semantic drift inside the application model itself.
Example:
Entity field renamed
Metric still references old semantic meaning
Relationship cardinality changes
Permission scope shifts
All technically valid JSON.
All passing structural validation.
But the meaning of the system changes in ways that silently break invariants.
So the question becomes:
Should lifecycle control live only at the “LLM output quality” layer (prompts, evals, CI), or does the runtime itself need semantic version awareness and invariant enforcement?
In other words:
Even if the AI is perfectly guarded, do we still need an application-level planner that understands entities, metrics, relationships, and permissions as first-class concepts — and refuses inconsistent evolution?
I am definitively into this.
Schema evolution is treated as a runtime operation with versioning, migration logs, and a deterministic state hash (dslHash). Every compile produces a new schema version and writes a structured change plan to dsl_change_log, so structural mutations are fully auditable.
There’s also a cryptographic validation attestation step: the compiled DSL is hashed and attested so the runtime can verify that the schema being executed is exactly the one that passed the pipeline. That prevents unauthorized structural drift outside the compiler path.
Breaking changes, data compatibility, and migrations are evaluated before commit, so structural mutations are gated much like data mutations.
The stack to support this is admittedly quite complex, but most of that complexity lives in the runtime so the AI-facing interface can remain simple and safe. The big shift was treating AI as proposing structure, but never owning execution.
One thing I'm still unsure about is where the long-term governance layer should live once models can mutate system structure — inside the runtime itself, or higher up at the application/policy level. Curious how others are thinking about that boundary.