They do disclose that they scored much lower than Fable on SWEBench Pro, which is a pretty high-quality benchmark. I think it's partially just about what they choose to emphasize...
To clarify, the more accurate description would be "Testing how well LLMs can follow the rules of Magic", right? There is no actual evaluation of how "well" they are playing?
This is basically just the normal dynamic with American tax law where tax jurisdictions are terrible at coordinating, so they end up approving things and agreeing to tax things at a very low level in order to win the competition. Even when states/counties try to work together on this stuff there's a huge defector problem, like "hey I can back out of this multistate tax compact agreement and get 500 new jobs which will let me win local reelection".
I suppose you can reduce a lot of both good at bad things about the country to "because federalism".
I wonder how Paradox handles this stuff in their games like Europa Universalis. Have they ever made a "the pope wants you to switch calendar systems" event which changes the actual in-game date?
I really fundamentally do not understand what problem Gas City solves that is not already solved by normal subagent orchestration patterns. If you want to call your main LLM session the "mayor" and have it delegate its work out to planners and coders and reviewers and QA and so on, this is already a thing you can do! If you want to do this in a reusable way you can create skills and subagent definitions and use /commands, etc. Why do we need hundreds of thousands of lines of opaque Go code to accomplish any of this?
There's a real hobbyist vs professional distinction with Claude Code. For professionals, including when I use it at work, we're generally super happy to have Claude spawn as many subagents as possible and burn more tokens to get a better result. Hobbyist users on a $20/month plan, though, generally want more conservative behavior.
It's hard for Anthropic to cater to both sets of users with one model.
No "max" or "pro" equivalent? I wanted to get a new Macbook Pro, but there's no obvious successor to the M4 Max available, M5 looks like a step down in performance if anything.
Does the "caching containers for Codex Cloud" mean I have some chance of being able to reuse build artifacts between tasks? My Rust project takes around 20 minutes to set up from scratch in a new Codex environment, which seems extremely expensive.