What we learned recently is that they are tuning things. There is less compute, less tokens, less of what ever else is there thrown at it as time goes on.
They use these releases to get users. When they got them they can play around with "degrading" model just enough not to loose users but save on costs. It sadly kinda makes sense...
I agree!
My "dream" way to do it is closer to how Aider Leaderboard works but even bit better.
To have GDPEval like set to tasks but you have information across all tasks and all models of how much time/tokens/money/quality you get from particular model on particular task.
I was thinking to do evals against skills in that sense.
But that is huge and expensive project.
Only "approximation" I could pull of reasonably to get this started was to use benchmark scores as "surrogate" for that.
But working on a way to get this going. If you have additional thoughts on how to approach this I it would be super valuable.
Building a tool to compare value across LLM provider options.
Part of it tracks how many tokens you actually get from various subscriptions, over time.
Past week, multiple people asked me about it — they'd been hitting Claude and Codex limits faster than expected.
Ran the tests yesterday. Reran today. Here's what came back:
▸ ChatGPT Plus / GPT-5.5: 95M → 37M tokens/week (−61%)
▸ Claude Max 20× / Sonnet 4.6: 388M → 214M (−45%)
▸ Claude Max 20× / Opus 4.7: 248M → 162M (−35%)
▸ Claude Pro / Sonnet 4.6: 19.6M → 11.4M (−42%)
▸ Claude Pro / Opus 4.7: 15.6M → 10.2M (−35%)
5 of 5 retested plans dropped 35-61% in five days. None went up.
And it does look undeniable that LLMs are genuinely useful. The "are they useful at all" question feels settled.
Where I agree is on the financial side — what the top labs are doing does look like a bubble. They're racing toward being first to AGI as if that gives them world domination. That seems delusional, and they'll need to stop or collapse.
But collapse won't kill AI or LLMs. It's like saying the dot-com bubble collapse should have killed the internet because it wasn't useful. The internet was useful, dot-com bubble or not. LLMs are useful, bubble or not.
What I expect is things will get less crazy over the next few years, with or without a crash. GPT-4+ and Opus 4+ models have reached genuinely useful levels for knowledge work. And if the trend continues — from GPT-4 expensive in the cloud to Gemma 4 running locally and being smarter — we'll get GPT-5/Opus 4 level models running locally by 2028, and it will keep getting better. At that price point and with local use, AI will be where it should be.
The top labs are burning money like crazy to get there first, as if that gives them a lasting moat. It won't.
What they're trying to do is become the new Google, Apple, or Microsoft. Those companies did achieve sustained moats through speed of execution. I don't see that happening in AI right now.
Though the Anthropic moment this year did make me worry a little...
Yeah. And weird pricing seems like it's winding down.
It's interesting to compare it to electricity. Basically Anthropic was selling a flat fee electricity subscription, and when someone started connecting expensive washing machines (OpenClaw) to their subscriptions, instead of changing the pricing model, they banned washing machines...
I wonder if we will get to "electricity" style pricing for AI. What makes electricity predictable is relatively constant average usage over time + price is manageable. I'm just not buying electrical house heating and manage my electricity spending within some bounds.
With AI the problem is that we are only now getting to useful AI, and for now it's still too expensive to be useful, so they subsidize until they can stabilize at "cheap enough and smart enough" level. But it feels like that's still 2 years away while they are stopping to subsidize now. Will be interesting.
Actually AI also benefits from thins being organized.
I find Skills to be Zettlekasten inspired or wiki inspired in that sense.
Zettlekast has other benefits for humans though.
If your goal is to grasp lot of knowladge oyu need to do it in atomic way, connect mentally to what you already know and do spaced repetition to internalize.
Zettlkeaste forces you do it it all as part of organizing. Basically by organizing you make it your own.
Yes AI can help today but it also means it does not stay in your head.
Not sure its important if it is in your head or you can call AI at any moment instead of your own memory.
For the last year MCPs were backends without a frontend — dumping text into LLM context and hoping it would show it well to the user. Now MCP Apps are here and they could mean way more for MCPs than it seems at first.
Since ChatGPT we've been putting chatbots into our apps. Now we can put our apps into the chatbots. And if one image is worth a thousand words, good UI/UX can be worth ten thousand — without polluting the LLM context window.
I wrote this as someone building an MCP server (Desktop Commander, ~392K installs by now) who felt the pain of having zero UI control firsthand. Happy to answer questions about what building with MCP Apps is actually like.
The debate around local vs API vs subscriptions feels mostly anecdotal.
I tried building a tool that compares them using “quality-adjusted tokens per dollar.”