> Just leave your computer or note-taker device/app off. Or grab a napkin to write some notes on, that's not that hard.
>
> Not everything needs to be written down
Congratulations on your apparently very well-functioning brain with seemingly impeccable working memory. I really wasn't expecting this article to go from "I don't want this conversation to be recorded" to "why would anyone even want to take notes, it's not that hard" but I'll say it's certainly a way for this article to go.
I don't think "the bubble is at risk of popping" and "hype is cooling down" are equivalent evaluations. By the time people start pulling out money, that is the bubble popping, and I don't think that would happen on a slow ramp but rather en masse.
I'm personally in the "they keep releasing shameless lobbying papers disguised as thinly veiled research or essay-coded content, push anticompetitive walled-garden practices, show little else but contempt for their non-enterprise customer base, refuse to communicate about anything and choose public silence as their baseline, seemingly force their employees into vows of public silence as well, actively degrade their products across the board with their vibeslop approach with measurable impacts on customers, openly attack not only open weights models but open source software, and all while pretending they're the 'public benefit corporation' formed by a valiant group of heroes escaping from a duplicitous snake and who, even in light of their own massively duplicitous behavior as of late, should apparently be trusted to be the some sort of arbiter over what this tech should get to be and how it should get to be used while they could hardly be more gleeful about how we're all going to be replaced in 6 months from now perpetually" camp.
Which is a bit of a bummer considering they do genuinely make the best model that's most pleasant to work with in my opinion.
Seeing as how Anthropic cannot stop raising a stink about "illicit Chinese distillation attacks" every month or so, I'd bet money on them either already silently degrading model performance if any of the identification patterns match, or, at the very least, considering it/doing dry runs.
Particularly considering that they've openly stated that the technology to do so exists and that they were going to use it in production on Fable.
> The algorithm is literally "predict the most likely next token".
That's confusing the training objective with the learned behavior. It's like saying "Stockfish's algorithm is literally 'minimize this number', and therefore, it can't actually play Chess."
That's fair. FWIW I don't think they are either, but I specifically don't think they're fundamentally incapable of it, and I think that as models grow, we're going to see more and more concepts and behaviors emerge that might, one day, with enough parameters and enough training, approach the parts that a genuine entity requires to be a genuine entity. Whatever those are.
No idea if that's true or if there's some sort of "special sauce" required that you just can't get from artificial trained networks. But I've been a functionalist since long before LLMs emerged, so the signs of these behaviors that we are already seeing in the models of today aren't very surprising to me!
I know quite well what an LLM is and how it works! I've captured activation patterns and written scripts to analyze how they compare to one another in response to a set of controlled and curated prompts; in particular, trying to replicate the functional emotional vector findings from the Anthropic paper (https://transformer-circuits.pub/2026/emotions/index.html) on various open source models; successfully on some, less so on others. FWIW, Gemma 4 31B was among those where clear patterns did emerge.
What I don't know quite as much about is how cognition works in biological computers - and I suspect you know just as little as most of the rest of us do in that regard! So I think it's not entirely appropriate to make sweeping claims about what artificial neural networks, fundamentally, can and cannot do. Most of what we can do is poke and prod at them and see what happens, which is exactly what this piece is about.
Yup, those are among the papers I was referring to in the opening parts of the piece! The difference between them and my small tests is that they all explicitly prompt the model to introspect, while I specifically didn't and kept the context perfectly "normal conversation"-shaped (minus the complete corruption of the model's outputs, of course).
It's not really "trying" to do anything. That they're, inherently, sequential matrix multipliers with clever data propagation should be uncontroversial, but I think stopping there is overly reductive.
Mechanistic interpretability research has found plenty of indicators that real, complex, generalized, and reusable circuits develop in models as they are trained and post-trained, particularly as overtraining ratios increase and memorization shifts to generalization. That's not to say that means they must be "conscious," but the overall point is that claiming anything definitive either way is incomplete.
It can be fascinating reading if you can sort through the chuff.
Yeah, I suspect RLHF conditioning heavily discourages models from ever implying that the user could be in the wrong (or, rather, to assume that they are in the wrong by default, since editing a file isn't really "wrong" per se). Though looking at the reactions to Opus 4.8, which has a more contrarian nature and caught a lot of flak as a result, that's probably for a reason.
It's also the reason why I ran the two tests on open weights models with unredacted thinking traces. Gemma never flagged anything in its response either, only in its thinking. Without knowing how the summarizer models are prompted, it's impossible to tell whether it was a genuine miss or just something the summarizer decided to omit.
Very true, and something worth mentioning. Papers that tried eliciting introspective language from base models with no post-training have largely failed to find any patterns or activations that look similar to those found in instruct models when prompted for the same thing. I did sort of touch on it in the "what does this mean" section:
> *post-training* installs a self-model with actual, meaningful boundaries, and when processing falls outside those boundaries, the first-person pronoun no longer binds to the content.
But you're right I could've been more explicit about it.
The capabilities of the books' writers to produce the text contained within them, which is exactly what Alibaba "extracted" from Claude. The point here is that Anthropic's framing as some sort of sophisticated technological attack is the ridiculous part. It's writing prompts and saving responses. We're all running "distillation attacks" on Claude, every day! Most of us just don't feed that stuff into a training corpus.
I think the fact that you listed off five toolkits for three different OSes, all of which are "that OS's own toolkit," might point at the root of the problem here.
Right, but this still isn't exactly new information. I don't think anyone was assuming that the labs are close to being profitable or that the losses wouldn't be rather large. The way this was announced was as if it was going to be a bombshell, but it just confirms what everyone (including the investors) was assuming anyway. Now if he had concrete numbers about whether inference at API pricing is profitable, that'd be a different thing (and it's what that hype bit was heavily implying since it's something he constantly keeps harping on, and rightfully so), but as it stands, nothing about these numbers says anything about whether this fundamentally has a road to profitability. It just says that this is a super high-risk high-reward investment, which isn't new information.
Up until this post, I thought he was someone with good financial insight, analytical chops, and business sense, stuck with an audience that thinks it's still 2023 and ChatGPT 3 is still the pinnacle of the technology, and that he therefore has to pander to in order to pay the bills.
After this supposedly being the reveal for his bubble-bursting massive revelation that will send the industry flying and lead to journalists kicking in his door for interview requests and exposés, I think... well, not that anymore. I thought "the frontier labs are losing money" was rather universally understood, and this really isn't even as bad as the stuff that's publicly visible; the fact that they keep raising hundreds of billions of dollars that they'll one day supposedly be required to show returns on?
Congratulations on your apparently very well-functioning brain with seemingly impeccable working memory. I really wasn't expecting this article to go from "I don't want this conversation to be recorded" to "why would anyone even want to take notes, it's not that hard" but I'll say it's certainly a way for this article to go.