Yeah you definitely have to be skeptical regarding sentiment for open/local model capabilities, since there's bias from what people want to be true.
I generally agree with this in spirit https://www.seangoedecke.com/are-new-models-good/ , but I think you can read Anthropic's results showing Sonnet 5 as almost strictly worse than Opus 4.8 as very credible/meaningful, and then draw comparisons from that
I think the incentives are less bad since a good chunk of usage comes from subscription plans.
There was a fairly major regression in Claude Code performance for some time when they changed the system prompt to try and make it less verbose (saving tokens). And if I'm not misremembering, there were a lot of complaints when they changed the default effort from high to medium.
Wow, seems worse even on price/performance than GLM 5.2, which is only 744b parameters.
From the system card: "On CyberGym vulnerability discovery, Claude Sonnet 5 is less capable than Sonnet 4.6, and far less capable than Opus 4.8 and Mythos 5
As with the other evaluations in this section, these results were achieved with all safeguards turned off. When run with our default mitigations, Sonnet 5 scored a 0 on CyberGym"
That's for their `JSON` data types. In DuckDB it's just a string meaning lots of queries will have to do JSON parsing on every row, but the inserts are very fast. Definitely a bit of a footgun and when you actually just need STRUCT or MAP.
There's a talk about ClickHouse's approach from its creator: https://www.youtube.com/watch?v=xHj9mysh0GI , but the gist is that it maintains (sub)columns to store different paths in the JSON
In other ways DuckDB has very good JSON support, like you can do `CREATE TABLE name AS `SELECT * FROM 'data.json';` and it'll infer the schema when possible.
> If the government takes the bulk of your income after a certain point, there isn't really that big of a push to create ground-breaking technology.
I'm skeptical that high taxes is a large reason to lose to California of all places. Maybe in some important sense CA has "earned" that via talent and funding density while NL hasn't (from the perspective of a company, to be clear)
Interesting comparison, thanks for sharing! It reminds me of this post about how machine learning and encryption have some fundamental similarities: https://reiner.org/neural-net-ciphers
> I can certainly imagine LLMs taking a similar path.
Maybe it's useful to think about what fundamental differences could contribute to LLMs taking a very different path. What comes to mind is the scaling hypothesis, implying that the best LLMs will require enormous capital investment.
That seems largely incompatible with open source barring a fundamental change. There's open weights, but I can't think of a clean historical analogy there and find it extremely difficult to even guess how the future will go
My guess is that they liked the status quo with Project Glasswing and didn't want Fable to be public, especially if anyone is jailbreaking it into Mythos and using it for cyber
But then it backfired spectacularly and now it seems they can't use Mythos currently
It's certainly worse news for Anthropic than other labs since it's not completely random, and there's people in the administration (e.g. David Sacks) who don't like Anthropic -- perhaps seeing them as an enemy
Seems like estimates are that 70-85% of their revenue comes from API usage/pricing, so some users switching from Opus to Fable for that would've had a big impact
Then there's people switching from GPT 5.5 or upgrading their subscriptions, and Fable being scheduled for removal from subscriptions on the 23rd
I assume they didn't use the Cerebras version for this since it's probably very supply-constrained right now