The article says in the second section that the author did not have access to Mythos. I think it’s dangerous to rely on claims made by others without even bothering to read them first, let alone check.
It found hundreds of vulnerabilities in Firefox, according to Mozilla: how does Mozilla benefit? It found a 27 year old vulnerability in OpenBSD. How do they benefit from that? Is that made up? Are the maintainers of those codebases lying for the benefit of Anthropic’s IPO? Is copy fail a fabrication by big AI? The 12 OpenSSL vulnerabilities found in January?
Im not sure whose claims you think I’m relying on. I trust Firefox that they’re not overstating the number of CVES they’ve found. Same for OpenSSL. The OpenBSD folks definitely don’t seem like the types. I’ve not known Linux to fabricate CVEs either. I think my sources are fine.
The most important thing I would point to is Mythos et al and the wave of vulnerabilities that have been discovered in the past couple months. It’s a completely unprecedented event, brought forth almost entirely by improvements in the models themselves.
That said. keep in mind, I’m talking about over the past two years. With Claude code and the capabilities gained since December of last year, there have been incredible gains in the capabilities that are now available. Demand for inference is higher now than it was a year ago, because capability has improved. A specific criticism that I would hold is that claiming that progress with LLMs is slowing, prior to that point, is embarrassingly wrong in my view.
One could argue that the model capability improvements are slowing, and all the improvements were in harnesses. I think that’s a stronger argument, but I have a few problems with it.
1. Utility is utility. Whether that comes from the model or the harness is irrelevant when making claims about utility. I don’t think that’s a useful distinction most of the time, but especially when talking about the technology as a whole.
2. Marginal intelligence gain is different than marginal utility gain. It’s estimated that intelligence grows logarithmically relative to investment. However, the utility of a marginally more intelligent model may grow exponentially, because once behavior crosses a reliability threshold, it unlocks new capabilities.
3. Even on those terms, it’s not clear to me that frontier capabilities are slowing down. With Mythos and its contemporaries, we have been seeing a vast change in the security industry as vulnerabilities are discovered at an unprecedented rate. OpenBSD vulnerabilities, more Firefox vulnerabilities found in a single month than the past two years, critical Linux vulnerabilities. It’s hard for me to look at the effects there, a radical new capabilities baked into the model itself, and see stagnation. A part of the reason it might feel like it’s slowing down is because we plebs don’t have access to the top models.
He’s been continuously predicting that the collapse was just around the corner, that progress was slowing, and that there was no market for inference, since 2024.
The fact he’s never reflected on the glaring failures in his analysis tells what we need to know about his intellectual integrity. There’s truth in some of his words about financial risk, but if you can’t acknowledge that there’s upside too, you can’t evaluate risk properly either.
A well-optimized program is often a consequence of a deep understanding of the problem domain, good scoping, and mindfulness.
It often feels to me like we’ve gone far down the framework road, and frameworks create leaky abstractions. I think frameworks are often understood as saving time, simplifying, and offloading complexity. But they come with a commitment to align your program to the framework’s abstractions. That is a complicated commitment to make, with deep implications, that is hard to unwind.
Many frameworks can be made to solve any problem, which makes things worse. It invites the “when all you’ve got is a hammer, everything looks like a nail” mentality. The quickest route to a solution is no longer the straight path, but to make the appropriate incantations to direct the framework toward that solution, which necessarily becomes more abstract, more complex, and less efficient.