Nah, the last few generations have more RLVR in the data mix. Which is more CPU intensive and very much amenable to the bitter lesson as you can reduce the loss by doing more rollouts in your tool environment.
Lower-priority backlog items. TODOs atop your new PRs. Code reviews. Exploratory work where you can discuss a design sync and then dispatch one or more agents to prototype async. Any workstream where you can define a loop and let the agent hill-climb towards the goal.
A lot of this is personal taste but the general thing I get most value from is asking an agent to speculatively build every idea, instead of writing down ideas in some backlog for later (it never happens later).
Agreed that switching costs to Codex are quite low (not zero due to harness, prompt, and skill differences), but only because OpenAI is running the same loss-leader strategy.
I think you need to consider how much dev mindshare you get from the loss-leader. Claude has grown at an insane rate. They brought vibe-coding mainstream.
At some point they might decide they have enough demand and inertia from enterprise to reduce the subsidy. But to say “it’s doomed” really misses the fact that it has already been immensely successful.
No. If you have a simple line-of-business app, writing Django/Rails models is FAR easier than the equivalent SQL.
Even if you think that maintaining your domain model is easier in SQL (it’s not, for most full-stack engineers), the extra capabilities you get from an ActiveRecord framework such as full-stack admin pages, free migrations, etc. win overall.
I can believe that the gap is closing with the “api for your Postgres” frameworks but really, try reaching your frontend developers sql and see if they have a better time than learning Django/Rails.
Ha! So we were not looking at the same chart. That makes more sense.
> Anthropic did post an official explanation, stating the original chart used a "simpler methodology" that "underestimated Sonnet 5's performance." The new chart supposedly uses their "standard methodology."
But, it is a big own goal, because once you invest in building evals for your internal use-case, 1) it’s easier to switch your model to whatever is cheapest, and 2) it’s way easier to fine-tune an oss model.
Evals are annoying to build and most companies were fine to rest on vibes. Now many companies have to do the work for insurance.
I guess I can see why they might nerf detected clients server side, but without evidence I would not assume it. Could also be so that 1) they can identify sus client IPs, 2) do a statistical analysis on distilled models to prove that their system prompts were clearly using unique tokens from Anthropic’s API.
> My whole point is that I don't want it to build an entire feature from one prompt
You are free to do you. But you were asking about why others want the best model.
The answer is, clearly, agentic coding (ie multiple agents each cranking through tasks independently) lets you ship A LOT more business value if used correctly.
This seems to be the crux, I couldn’t find a place where the bill explicitly says, so AIUI the rule making could fall either way.
Are you “compensated for being oncall” if your contract says you may do some unspecified amount of oncall, and your pay doesn’t change if you do or don’t?
You could imagine a judge / regulator deciding either way right?
This seems mostly good for restaurants, some concerns I had from the title seem to be handled reasonably.
It’s not preventing “can anyone cover Saturday” messages in a group chat. Just the case where shift changes are made and workers are _required_ to work outside their contracted hours. Seems this would fit with what good food service employers do, would put pressure on the more abusive fast food chains. Maybe the flexible shift is more important than I credit though?
Unless I’m missing something it would ban the standard startup model for oncall, meaning Michigan would be made (even more) unattractive for tech startups. Unless we just re-comp everyone to include an SRE stipend as part of the contracted salary package? Unsure if that could work, maybe? SWE is typically well over minimum wage so maybe this just nets out the same?
I for sure agree that plenty of current use-cases are solvable by non-frontier models.
However, you said “new versions with features that nobody asked for”, and I would prefer that you concede the point before shifting to arguing a new point.
What customers are asking for is smarter models. Because the tasks that only smarter models can solve are higher value, higher margin, than the tasks that non-frontier models can solve.
As GP said. More RLHF is in fact the bitter lesson.