Thanks! I do have a section on this in the article "Why genetic algorithms aren't state of the art"
"Physics simulation involves discontinuities (contacts, friction regimes), long rollouts, and chaotic dynamics where small parameter changes lead to large outcome differences. Even with simulator internals, differentiating through thousands of unstable timesteps would yield noisy, high-variance gradients. Evolution is simpler and more robust for this regime."
"The real tradeoff is sample-efficient but complex (RL) vs compute hungry but simple (GA). DQN extracts learning signal from every timestep and assigns credit to individual actions."
Simply, it's when your output embedding matrix = input.
You save vocab_dim*model_dim params (ex. 617m for GPT-3).
But the residual stream means that the weight matrices are roughly connected via a matmul, which means they struggle to encode bigrams (commutative property enforces symmetry).
Attention + MLP adds nonlinearity, but it still means less expressivity.
Which is why they aren't SOTA, but are useful in smaller models.
See what you don't understand is that you need to coordinate the deacon to take the witness out back and talk to the mayor. It's actually quite trivial.
Try Cursor Composer! It's the most natural transition. Exactly what you're currently doing, but it inserts the code snippets for you from within your IDE.
Basically, an LLM pretends to be a virtual machine executing instructions. Based on GEPA and Qwen AgentWorld.
Just - clone, - `uv sync` - `uv run wmh build` and you'll get a wizard that will help you create your own world model from your traces.
http://github.com/experientiallabs/world-model-harness