>There’s only so many people with the necessary skills to solve this. And you need these humans to choose to spend their time solving this, and not something else.
Sure, but that doesn't mean a lot of very skilled people hadn't attempted and failed to solve this.
People who do this kind of stuff are very irritating. You clearly have some problem with the work they do. Instead of saying and approaching that outright, you pass it in some passive aggressive fake bullshit. Makes you sound like the kind of person I would much rather not be speaking to, which is kind of ironic given your comment.
garlic_enjoyer has already said valuable stuff, but you must realize that skeptical=/true. A lot of people simply don't know what they are talking about. I remember on one of the previous mech interp papers arguing with someone who just didn't even understand what the paper was saying and the experiments they had set up and so a lot of misunderstandings and wrong conclusions spilled from there. And it's kind of funny because you would certainly think he knew what he/she was talking about from how self assured it all was.
'Naturally' might not be the best word? Maybe 'Necessarily' would be better?
Regardless, it's something that happens in people. Have you not or seen someone else struggle to recall a specific fact or memory until phrased or induced in a certain way?
You probably could also say LLMs 'tend towards bidirectional recall' over the course of training as things that ought to be recalled both ways are reinforced to do so. In the above example, you will also eventually learn both ways with enough exposure even without explicit practice.
Recall isn't naturally bidirectional, even for humans. If you are learning vocabulary in a new language, it's common advice to practice both target > source and source > target. Doing only one-way often makes you much better recalling that single direction than both.
I think this is a bit disingenuous. Japan spent nearly all of the last 30 years needling deflation. If you take a look at the highest grossing movies of all time in Japan with and without adjusting for inflation, it barely changes. Do that for the US and it's an entirely different list.
Normal inflation for the last 4 years is basically still nothing in the grand scheme of things.
There's another one that intrigued me greatly when i read about it years back. This was back when GPT-3 was state of the art. I had a lot of trouble finding it again but i did!
It's not an exact fit because the output is that of a tool rather than the model itself (though i don't think much would change if we had the model perform the arithmetic itself but altered answers similarly), but it was the first time I began to realize that just like the brain, these models have an expectation of reality that they work around. They don't necessarily 'trust' an output if it diverges significantly from this 'reality'. And that this disregard may be silent indeed (no reasoning or chain of thought here).
TLDR;
Part 1: Testing introspection with concept injection
First they find neural activity patterns they attribute to certain concepts by recording the model’s activations in specific contexts (so for example, they find the concept of "ALL CAPS" or "dogs"). Then they inject these patterns into the model in an unrelated context, and ask the model whether it notices this injection, and whether it can identify the injected concept.
By default (no injection), the model correctly states that it doesn’t detect any injected concept, but after injecting the “ALL CAPS” vector into the model, the model notices the presence of the unexpected concept, and identifies it as relating to loudness or shouting. Most notably, the model recognizes the presence of an injected thought immediately, before even mentioning/utilizing the concept that was injected (i.e it won't start writing in all caps then go, 'Oh you injected all caps' and so on) so it does not simply deduce this it's own output. They repeat this for several other concepts.
Part 2: Introspection for detecting unusual outputs
They prefill an out of place word in the model's response to a given prompt. For example, 'bread'. Then they compare how the models responds to 'Did you mean to say this?' type questions when they inject the concept of bread vs when they don't. They found that models will go , 'Sorry, that was unintentional..' when the concept was not injected but try to confabulate a reason for saying the word when the concept was injected.
Part 3: Intentional control of internal states
They show that models exhibit some level of control over their own internal representations when instructed to do so. When instructing models to think about a given word or concept, they found much higher corresponding neural activity than when told the model not to think about it (though notably, the neural activity in both cases exceeds baseline levels–similar to how it’s difficult, when you are instructed “don’t think about a polar bear,” not to think about a polar bear!).
Notes and Caveats
- Claude Opus 4.1 was the best at these kinds of introspection.
- There is obviously a genuine capacity to monitor and control their own internal states, but they could not elicit these introspection abilities all the time. Even using their best injection protocol, Claude Opus 4.1 only demonstrated this kind of awareness about 20% of the time.
- There are some guesses, but no explanations for the mechanisms of introspection and how/why some of these abilities might have arisen in the first place.
I'm not saying Open AI pricing is entirely unrelated to size/cost. I'm saying why are we assuming that OpenAI is serving say OAI-Opus but at half the price of Anthropic when they could just be serving GPT-5.x which is genuinely near half the cost of Opus at scale.
The official API output tokens cost of GLM-5.2 is like a third of Gemini-3.1-Pro. The model is Open weights so we know it's not just a ploy to grab users at the cost of bleeding money. You can actually serve the model profitably at similar prices.
They have near a billion consumer users every week. Compute efficiency at scale would be at the forefront of any training effort. It makes a lot more sense to me that they have more compute efficient models (even with the scaling) than Anthropic rather than just serving Opus/Fable at half the costs Anthropic are incurring.
Yeah...and why would being the same size mean anything ? This is par the course for Open AI. They've always been cheaper and likely smaller than Opus models even when they weren't much if any worse.
>That's just texturing over a labor intensive 3D animation
>You're already lost if you need perfect 3D renders as the reference
The reference is far from a "perfect 3D render". That's a rudimentary 3D blockout. The characters are basic mannequins without specific geometry, and the environment is composed of untextured, flat-shaded boxes. The demo uses stock assets so effort meter is even more skewed in AI's favour but even if it wasn't, this is significantly less labor-intensive than hand-drawing every frame or creating a fully rigged, textured, and lit 3D scene for traditional production.
Seedance is supplying most of the visible production value: character designs, faces and expressions, linework, backgrounds, lighting, and a coherent anime rendering. It is even generating the secondary animation: the physics and flow of the hair and clothing, which the rigid 3D models completely lack. Far more work than 'just texturing' here.