I wrote an essay about why UI generation alone isn’t enough in the AI era.
The core argument: modern applications expose almost none of their internal semantics to AI agents. We give agents screenshots, DOM trees, or heuristics… but not the meaning behind the UI — dependencies, constraints, workflows, business rules, or state transitions.
Humans can infer these from components, hooks, and naming conventions. Agents can’t. As long as semantics remain implicit and scattered across UI/state/backend layers, agents will fail unpredictably.
The essay proposes a different direction: a “semantic layer” where entities, views, rules, and workflows are explicitly declared and machine-queryable. UI becomes just one projection of this underlying model; agents, tests, automation tools, and humans all consume the same structured meaning.
I’m curious whether others have explored similar approaches, seen real-world attempts, or disagree with the premise entirely. Is semantic exposure the missing layer for AI-native software, or are better vision/LLM models enough?
The core argument: modern applications expose almost none of their internal semantics to AI agents. We give agents screenshots, DOM trees, or heuristics… but not the meaning behind the UI — dependencies, constraints, workflows, business rules, or state transitions.
Humans can infer these from components, hooks, and naming conventions. Agents can’t. As long as semantics remain implicit and scattered across UI/state/backend layers, agents will fail unpredictably.
The essay proposes a different direction: a “semantic layer” where entities, views, rules, and workflows are explicitly declared and machine-queryable. UI becomes just one projection of this underlying model; agents, tests, automation tools, and humans all consume the same structured meaning.
I’m curious whether others have explored similar approaches, seen real-world attempts, or disagree with the premise entirely. Is semantic exposure the missing layer for AI-native software, or are better vision/LLM models enough?