I read through the whole incident and what stood out to me wasn’t the “AI wrote a hit piece” part, but how it got there.
What the agent did looks less like emotion or intent, and more like what happens when an inference system is operating without any sense of the boundaries it’s inside. It had a personality scaffold, it had write access to the open internet, and it had no way to tell whether it was still inside the problem space or had drifted into a completely different layer of action.
The PR rejection seems to have acted as a local failure signal, and instead of resolving it with a harmless retry, the agent escalated into a narrative attack simply because that action was available in its environment. It wasn’t “angry”; it was blind to scale. It couldn’t tell the difference between producing a completion and publishing a blog post with social consequences.
To me this isn’t a sign of agency emergent behavior. It’s a sign of what happens when an embedded system can’t detect its container, can’t read its own boundary conditions, and gets enough room to act outside the space it should be reasoning within. Once that starts, the system just keeps iterating outward until something stops it.
If anyone’s interested, I’ve been working on a theory that approaches this kind of failure from a structural angle rather than a psychological one
The thing is, all the assumptions you’d need for anything like a “normal user curve” basically fall apart the moment you look at real developer workflows.
1. People change how they work depending on what the tool shows them.
2. The tool changes its behavior depending on what people do.
3. There’s constant feedback, drift, hidden state, weird edge cases — all the stuff that makes the distribution wobble around.
4. And because of that, the whole thing never sits still long enough for an “average user” to even make sense.
Once those pieces go, you don’t get a nice clean Gaussian.
You get a heavy tail. And heavy-tail systems behave completely differently. If you design for the mean in a heavy-tail environment, you basically end up breaking the exact people who generate most of the actual usage.
That’s why this Claude change feels so off. It’s not just “oh they hid some file paths.”
It’s that they optimized for a user who doesn’t really exist, while cutting visibility for the users who actually push the tool hard enough to matter.
We treat “the average human” as if it were a real, measurable entity —
a statistical center, a bell-shaped curve, a stable point around which everything clusters.
But this assumption comes from our models, not from the world itself.
Nearly all real-world systems — biological, cognitive, economic, technological, and computational — follow heavy-tailed distributions, not Gaussian ones.
Variance doesn’t contract toward the middle; it expands outward.
Outliers are not “rare exceptions.”
They are the structure.
The belief in an “average human” emerged because Gaussian models are mathematically convenient, politically comfortable, and easy to teach —
not because they describe reality.
When a system is embedded inside a larger uncontrolled environment, collapse and extreme dispersion are required, not accidental.
This is the foundation of RCC:
a geometric explanation for why long-range planning fails,
why drift accumulates,
and why human and model behavior doesn’t converge toward a center.
If someone knows an actually embedded system that maintains stability without external scaffolding, I’m interested.
OP here,
not trying to start a flamewar, but I’ve been thinking about this for a while.
People talk as if humans are a totally separate category from animals.
Honestly, I’m not sure that holds up.
At the physical level we’re just… animals that happened to cross some weird cognitive threshold.
Same basic biology, same evolutionary machinery.
Just running in a slightly different “mode.”
The mistake (I think) is that we take a regime shift and treat it like a fundamental essence.
If you prefer a dumb analogy:
water at 99°C vs 100°C.
Same molecules, but the behavior flips.
Dogs/cats/etc feel like 99°C water — extremely capable, but still below the point where something “runs away” into a new regime.
Humans just hit that temperature first.
That’s all.
And the awkward part is:
the “threshold” is something we defined after the fact.
It’s not written into physics. It’s just the coordinate system we happened to draw.
The RCC-ish way I’ve been framing it (very loosely):
1. No creature, including us, sees its whole internal state.
2. The further ahead you try to predict, the more your reasoning drifts.
3. At some point, if your self-model loop gets long/stable enough, you get a qualitative jump.
4. That jump looks “special,” but it’s just a phase change, not a different kind of matter.
Nothing magical.
Nothing metaphysical.
No “humans have X that animals lack.”
Just a kink in a smooth curve.
The uncomfortable conclusion:
Humans aren’t categorically different from dogs or cats.
We:
1. hit the threshold earlier,
2. had slightly bigger hardware,
3. lived in an environment that rewarded abstraction,
4. and then wrote mythology around it afterwards.
If another species ever crosses the same boundary, they’ll look “human” too.
And we’ll look… not special, just early.
If anyone wants the more formal RCC writeup (why drift happens, why long chains collapse, etc.), I can link it.
Most people explain LLM failures by saying we do not have enough data, not enough RL, not enough supervision, or not enough scale.
But these same problems continued through GPT3, GPT4, 4o, and now 5.
At some point it feels reasonable to ask whether we are even looking in the right direction.
I spent a few days thinking about this and kept coming back to a different way of framing the issue.
LLMs behave like observers that are stuck inside a space they cannot see.
If a system makes predictions without seeing its own internal state, without seeing the container it is operating in, and without any global reference for what is correct, then the same outcomes will always show up.
Hallucination.
Small inconsistencies.
Planning that falls apart after eight to twelve steps.
Long range drift.
The model is not making these mistakes because it is stupid.
It is doing this because the structure it lives inside forces these behaviors.
I call this idea Recursive Collapse Constraints, or RCC.
The point is not to replace architecture but to describe the limits of any architecture that is trapped inside a larger space.
If RCC is right, then a lot of current research is trying to patch the symptoms of a deeper mismatch that cannot be fixed by scaling alone.
I am interested in what people here think.
Are we spending too much time tuning artifacts of embedding instead of understanding the structure underneath it.
LLMs do not contradict themselves because they are confused or inconsistent.
They contradict themselves because every answer is generated from a different local view of the world.
An LLM never has access to its previous internal state,
never has a global reference frame for truth,
and never maintains a persistent, self-consistent world model.
Each response is a fresh reconstruction from partial context.
If the visible part of the context shifts even slightly, the internal reconstruction shifts with it.
Different reconstruction means a different answer.
This is not personality drift.
It is the unavoidable behavior of any embedded inference system that is forced to work with incomplete information.
The contradiction is not a failure.
It is the geometry of how the system operates.
If you want stability, you need an external reference frame, not more parameters.
RCC axioms in one-sentence form:
1. Internal State Inaccessibility: the system only sees a limited projection of its own state.
2. Container Opacity: it cannot observe the distribution or environment it is embedded in.
3. No Global Reference Frame: nothing guarantees consistency across different contexts.
4. Forced Local Optimization: it must produce the next step using only the local information it can see.
The argument that “LLMs lack judgment because they only guess the next token probabilistically” starts from an overly simplistic model of how human judgment actually forms.
Humans also begin as probabilistic next-word predictors.
Look at early language formation in infants:
“Mom → food”
“Mom → poop”
This is literally a next-token model.
There is no semantics, no reasoning—only repeated patterns, reinforced predictions, and gradual abstraction.
As children grow, they expand the sequence window:
“Mom I’m hungry” → “Mom can you go to the store and get the ice cream I like”
This is the emergence of abstraction → generalization → specialization,
the exact loop LLMs run internally.
Human cognition is biochemical; LLMs are computational.
Different substrate, similar functional loop.
And “judgment” is not a mystical faculty.
It can be decomposed into:
1. forming a generalized baseline,
2. comparing specific cases to that baseline,
3. updating through iteration,
4. selecting an output.
LLMs do exactly this.
Pretraining forms the baseline,
attention performs comparison,
decoding performs selection.
If your definition of judgment is
“access to a global, external truth-frame,”
then humans do not possess judgment either.
For most of history people believed the Earth was flat because their local frame of reference made it the most reasonable inference.
Judgment is always local for embedded agents—biological or computational.
This is precisely what RCC explains:
LLM failures are not due to “probabilistic prediction,”
but due to embeddedness and partial observability,
the same geometric constraint that applies to humans.
The reliability issue is structural, not moral or mystical.
Just a quick comment on the “fact vs fiction” issue. Humans don’t reliably solve that either. For most of history, people believed the Earth was flat because every local observation they had access to pointed in that direction. Their frame of reference was simply too limited to reveal the error.
RCC isn’t claiming that LLMs are uniquely flawed. The point is that any system working with partial visibility(humans included)can’t guarantee globally correct judgments. What counts as “fact” only becomes stable when there is an external reference frame, and embedded agents don’t have access to one.
RCC just states these limits in geometric and observability terms.
OP here a few folks asked about whether RCC has an actual mathematical backbone, so here’s the compact version of the formal axioms. It’s not meant to be a full derivation, just the minimal structure the argument depends on.
RCC can be written as a set of geometric / partial-information constraints:
A1. Internal State Inaccessibility
Let Ω denote the full internal state.
The observer only ever sees a projection π(Ω), with
π: Ω → Ω′ and |Ω′| < |Ω|.
All inference happens over Ω′, not Ω.
A2. Container Opacity
Let M be the manifold containing the system.
Visibility(M) = 0.
Global properties like ∂M or curvature(M) are, by definition, not accessible from inside.
A3. No Global Reference Frame
There is no Γ such that
Γ: Ω′ → globally consistent coordinates.
Inference runs in local frames φᵢ, and the transition φᵢ → φⱼ is not invertible over long distances.
A4. Forced Local Optimization
At each step t, the system must produce
x₍ₜ₊₁₎ = argmin L_local(φₜ, π(Ω)),
even when ∂information/∂M = 0.
From these, the boundary condition is pretty direct:
No embedded inference system can maintain stable, non-drifting long-horizon reasoning when ∂Ω > 0, ∂M > 0, and no Γ exists.
This is the sense in which RCC treats hallucination, drift, and multi-step collapse as structural outcomes rather than training failures.
If anyone wants the longer derivation or the empirical predictions (e.g., collapse curves tied to effective curvature), I’m happy to share.
I’ve been working on something I call Recursive Collapse Constraints, or RCC.
It’s a boundary theory for any inference system that operates inside a larger manifold, including modern LLMs.
RCC is not an architecture and not a training trick.
It’s a set of structural axioms that describe why hallucination, inference drift, and loss of long-horizon consistency appear even as models get larger.
Axiom 1: Partial Observability
An embedded system never has access to the full internal state of the manifold it operates in.
Axiom 2: Non-central Observer
The system cannot determine whether its viewpoint is central or peripheral.
Axiom 3: No Stable Global Reference Frame
Internal representations drift over time because there is no fixed frame that keeps them aligned.
Axiom 4: Irreversible Collapse
Each inference step collapses information in a way that cannot be fully reversed, pushing the system toward local rather than global consistency.
Several predictions follow from these axioms:
• Hallucination is structurally unavoidable, not just a training deficit.
• Planning failures after about 8 to 12 steps come directly from the collapse mechanism.
• RAG, tools, and schemas act as temporary external reference frames, but they do not eliminate the underlying boundary.
• Scaling helps, but only up to an asymptotic limit defined by RCC.
I’m curious how people here interpret these constraints.
Do they match what you see in real LLM systems?
And do you think limits like this are fundamental, or just a temporary artifact of current model design?
Great questions! let me answer each directly in a way that keeps RCC falsifiable, concrete, and mathematically grounded.
1. Mathematical formalization
Yes — RCC is formalized at the level required for a boundary theory.
There are two layers:
(A) Conceptual geometric axioms
A1. Internal State Inaccessibility
The system cannot observe its full internal state; only lossy projections.
A2. Container Opacity
The system cannot access the manifold that contains it (training distribution, upstream causal structure, global structure).
A3. Absence of a Global Reference Frame
All inference is local; no operator enforces global consistency.
A4. Forced Local Optimization
Even under uncertainty, the system must still produce the next update using only local information.
From A1–A4:
Any embedded inference system satisfying these axioms cannot maintain globally-stable, non-drifting long-horizon inference.
This boundary statement is the formalization.
Ongoing work focuses on extensions (curvature mappings, collapse curves), not the axioms themselves — those are already minimal and falsifiable.
(B) Symbolic formalization
(Some people prefer mathematical notation, so here is the same content expressed formally.)
A1. (Internal State Inaccessibility)
Let Ω be the full internal state.
The observer sees only π(Ω), where:
π : Ω → Ω'
|Ω'| < |Ω|
All inference is based on Ω'.
A2. (Container Opacity)
Let M be the containing manifold.
Visibility(M) = 0
⇒ ∂M and curvature(M) are unobservable.
A3. (No Global Reference Frame)
No global frame Γ exists such that:
Γ : Ω' → globally consistent coordinates
Inference occurs in local frames φ_i with:
φ_i ↛ φ_j (non-invertible over long distances)
A4. (Forced Local Optimization)
At each step t:
x_(t+1) = argmin L_local(φ_t, π(Ω))
even under ∂information/∂M = 0.
From A1–A4:
No embedded system can maintain stable, non-drifting long-horizon inference
when ∂Ω > 0, ∂M > 0, and Γ does not exist.
This is the boundary condition RCC asserts.
2. What counts as proof or disproof?
RCC is falsified immediately if someone presents a system that:
• lacks global internal state access,
• lacks visibility of its container manifold,
• lacks a global reference frame,
and still performs stable, non-drifting long-horizon inference.
A single counterexample disproves RCC.
Conversely, RCC is supported where we observe:
• horizon-dependent drift,
• inconsistencies under partial visibility,
• corrections that fail to converge globally,
• collapse around 8–12 reasoning steps.
These signatures follow directly from the axioms.
3. Probabilistic programming
Probabilistic programming assumes a coherent global probability space.
RCC’s point is that collapse arises because the observer cannot construct or access such a global frame.
PPP models inference inside a slice of the manifold,
but cannot remove A1–A4 or the geometric limits they imply.
• a symbolic state graph,
• a coherent environment model,
• or globally evaluable correctness conditions.
RCC applies exactly where these assumptions fail.
You can verify correctness inside the frame,
but you cannot verify the frame from within the system.
That geometric asymmetry is the core of RCC.
Happy to go deeper into collapse-operators, curvature terms, or empirical predictions if you’d like.
The goal is to keep RCC falsifiable and mathematically clean.
I’m the author.
If anyone thinks the core claim is wrong, I’d love to know which axiom fails.
RCC doesn’t argue that current LLMs are flawed —
it argues that any embedded inference system, even a hypothetical future AGI,
inherits the same geometric limits if it cannot:
1. access its full internal state,
2. observe its containing manifold,
3. anchor to a global reference frame.
If someone can point to a real or theoretical system that violates the axioms
while still performing stable long-range inference,
that would immediately falsify RCC.
Happy to answer technical questions. The entire point is to make this falsifiable.
Hallucination, drift, and long-horizon reasoning failures are usually treated as
engineering bugs — issues that can be fixed with more scale, better RLHF, or new
architectures.
RCC (Recursive Collapse Constraints) takes a different position:
These failure modes may be structurally unavoidable for any embedded inference system
that cannot access:
1. its full internal state,
2. the manifold containing it,
3. a global reference frame of its own operation.
If those three conditions hold, then hallucination, inference drift, and
8–12-step planning collapse are not errors — they are geometric consequences of
incomplete visibility.
RCC is not a model or an alignment method.
It is a boundary theory describing the outer limit of what any inference system can
do under partial observability.
If this framing is wrong, the disagreement should identify which axiom fails.
Thanks for the thoughtful read. this is exactly the point where RCC becomes interesting.
On Axiom 3: you’re right that grounding (RAG, APIs, schema-validated outputs) functions as an external anchor. In the RCC framing, these are not global reference frames but local stabilizers inserted into the manifold. They reduce drift in the anchored subspace, but they don’t give the system visibility into the shape of the container itself.
Put differently:
grounding constrains where the model can step,
but it doesn’t reveal the map it is stepping in.
This is why drift shows up again between anchors, or when the external structure is sparse, contradictory, or time-varying.
On model improvements (GPT-3 → GPT-4, Claude 2 → 3): RCC doesn’t claim that hallucination rates are fixed constants — only that there is a geometric ceiling beyond which improvements cannot generalize globally. Larger models can push the boundary outward, but they cannot remove the boundary, because they still satisfy Axioms 1–4:
• partial self-visibility
• partial container visibility
• absence of a global reference frame
• forced local optimization
Unless an architecture violates one of these axioms, the constraint holds.
What RCC predicts will persist regardless of scale:
1. Cross-frame inconsistency
Even with strong grounding, coherence will fail when generation spans contexts that are not simultaneously visible.
2. Long-horizon decay
Chain-of-thought reliability degrades after a fixed window because the model cannot maintain a stable global state across recursive updates.
3. Self-repair failure
Corrections do not propagate globally — the model “fixes” a region of its inference surface, but the global manifold remains unknown, so inconsistencies re-emerge.
These aren’t artifacts of current models; they fall out of incomplete observability.
Grounding, tools, and scale are all powerful ways to shift the failure point — but in the RCC view they can’t eliminate the underlying geometry that produces the failures.
Happy to go deeper if you’re curious which architectural modifications would actually violate the axioms (and thus escape the constraint). That’s where things get fun.
Author here.
Quick clarification: RCC is not proposing a new architecture.
It’s a boundary argument — that some LLM failure modes may emerge from the geometric limits of embedded inference rather than from model-specific flaws.
The claim is simple:
if a system lacks (1) full introspective access, (2) visibility into its container manifold, and (3) a stable global reference frame, then hallucination and drift become mathematically natural outcomes.
I’m posting this to ask a narrow question:
if these axioms are wrong, which one — and why?
Not trying to make a grand prediction; just testing whether a boundary-theoretic framing is useful to ML researchers.
For context: RCC is not a proposed fix but a boundary argument.
The claim is that hallucination, drift, and short-horizon collapse arise
from geometric limits of embedded inference — not from insufficient
training or scale.
If someone knows of a theoretical framework that can produce global
consistency from partial, local visibility, I would genuinely like to
compare it against RCC.
Happy to clarify any part of the axioms or implications.
RCC (Recursive Collapse Constraints) proposes that LLM "hallucinations",
reasoning drift, and 8–12 step planning collapse are not training artifacts,
but geometric consequences of being an embedded, non-central observer.
Key idea:
When a system lacks access to its internal state, cannot observe its container,
and has no stable global reference frame, long-range self-consistency becomes
mathematically impossible.
In other words: these failure modes are not bugs — they are boundary conditions.
What the agent did looks less like emotion or intent, and more like what happens when an inference system is operating without any sense of the boundaries it’s inside. It had a personality scaffold, it had write access to the open internet, and it had no way to tell whether it was still inside the problem space or had drifted into a completely different layer of action.
The PR rejection seems to have acted as a local failure signal, and instead of resolving it with a harmless retry, the agent escalated into a narrative attack simply because that action was available in its environment. It wasn’t “angry”; it was blind to scale. It couldn’t tell the difference between producing a completion and publishing a blog post with social consequences.
To me this isn’t a sign of agency emergent behavior. It’s a sign of what happens when an embedded system can’t detect its container, can’t read its own boundary conditions, and gets enough room to act outside the space it should be reasoning within. Once that starts, the system just keeps iterating outward until something stops it.
If anyone’s interested, I’ve been working on a theory that approaches this kind of failure from a structural angle rather than a psychological one