Loss of Agency Is a Scaling Failure in Modern Software Systems(traulmen.blogspot.com)
traulmen.blogspot.com
Loss of Agency Is a Scaling Failure in Modern Software Systems
https://traulmen.blogspot.com/2026/01/user-agency-as-first-class-requirement.html
5 comments
As systems scale, control increasingly shifts from users to opaque layers: policy engines, algorithms, and now LLM-based agents. This isn’t an anti-AI argument, but an engineering one: collapsing policy, logic, and execution creates systems that are harder to reason about, override, or trust. This post examines loss of agency as a recurring failure mode in modern software architectures.
These two are spot on:
> Outputs are probabilistic but treated as deterministic.
> [Systems that] replace explicit mechanisms with probabilistic ones.
In other words, many things should be deterministic, not probabilistic. That's why the notion of probabilistic programming never really took off for most application domains.
> Outputs are probabilistic but treated as deterministic.
> [Systems that] replace explicit mechanisms with probabilistic ones.
In other words, many things should be deterministic, not probabilistic. That's why the notion of probabilistic programming never really took off for most application domains.
mostly agree, with one nuance.
Probabilistic systems do make sense at the edges — perception, ranking, recommendation, search, fuzzy matching. The problem starts when we let probabilistic outputs cross into domains that used to have hard contracts: policy enforcement, state transitions, or irreversible actions.
What feels new isn’t probabilistic programming itself, but treating probabilistic inference as if it were a deterministic control layer. Once probability collapses into authority, you lose debuggability and guarantees.
So the failure mode isn’t “probabilistic vs deterministic” per se, but where the probabilistic boundary is drawn — and whether it’s explicit.
Probabilistic systems do make sense at the edges — perception, ranking, recommendation, search, fuzzy matching. The problem starts when we let probabilistic outputs cross into domains that used to have hard contracts: policy enforcement, state transitions, or irreversible actions.
What feels new isn’t probabilistic programming itself, but treating probabilistic inference as if it were a deterministic control layer. Once probability collapses into authority, you lose debuggability and guarantees.
So the failure mode isn’t “probabilistic vs deterministic” per se, but where the probabilistic boundary is drawn — and whether it’s explicit.
> Probabilistic systems do make sense at the edges
Sure, and that's why I used the wording most application domains.
> So the failure mode isn’t "probabilistic vs deterministic" per se, but where the probabilistic boundary is drawn -- and whether it’s explicit.
If you take a look at
Sure, and that's why I used the wording most application domains.
> So the failure mode isn’t "probabilistic vs deterministic" per se, but where the probabilistic boundary is drawn -- and whether it’s explicit.
If you take a look at
https://arxiv.org/pdf/2512.22418
I’m not arguing against scale or automation. I’m arguing that many modern systems optimize for throughput and engagement while quietly removing inspectability, reversibility, and human interruptibility. Curious how others here think about “agency” as a system requirement, not a UX concern.