Most of my comments are, unfortunately, subjectively and in my personal opinion somewhat overlong due to my, admittedly not necessarily problematic tendency to hedge my own statements which, while in some cases helpful to more clearly delineate when I am referencing empirical data in contrast to when I am making a statement based on my personal and somewhat limited, due to my life experience at this age and the fact that I have grown up in the EU, experiences, I have grown to find that I have on occasion taken up the habit of overindulging on said clarifications and thus have come to the conclusion that this may be something I should address, as such extensive clarifications can make it draining to read my writing and can hide my actual intended point inside an avalanche of unnecessary and annoying to read writing, which is why in more recent comments of mine, which I have written and am intending to write in the future should time permit and topics I am interested in arise, you may find a more direct, albeit still where necessary properly clarified, style of writing comments...
Title was shortened and slightly editorialized from "OpenAI’s newest AI model is 54% more token efficient on agentic coding, Altman tells CNBC" for readability.
I cannot watch the interview right now, but the way the sentence is written in the article, it reads equally possible to me that this was in reference to other frontier models on the market or compared to GPT-5.5. The former I'd instantly believe, the latter I am very skeptical of.
I'd be very impressed if they actually managed any reduction as 5.5 is already obscenely token efficient to the point that the reasoning traces seem to compact far less reliably compared to 5.4 as they've become barely comprehensible gibberish.
As more reliable compaction is a stated goal for 5.6, if they solved that and got token usage more efficient on top, that'd make the price to performance equation very much in favor of OpenAI over any other lab. 54% more efficient vs 5.5 would make the pricing for a full benchmark suite run competitive with open-weight models, even if we assume higher per token costs.
I just checked and for plain old C, there do not seem to be any reasonably comprehensive, current-day eval suites. Fully admitting that, even if there were, I couldn't assess their validity simply because I have never written or reviewed any C code in my life (something I should rectify probably). Maybe the closest proxy is just parsing through the experiences people claim to have whenever LLM assisted kernel development comes up [0], but if you have a dataset, experience, time and muse, I'd just go for it and do some tests yourself. Have been doing the same, mainly focused on code quality and dealing with a mix of Rust, frontend web tech and SQL which has been a small but meaningful project and part of my go to eval for over a year now.
I doubt that, in these tasks, model restrictions to prevent training are affecting the results, not least because for both evals, the labs provided pre-release model access and have an incentive to be seen as favorably. In any case, I have not seen regressions to prevent distillations myself even when working on microscopic model training projects with LLM assistance, what I have however reliably and consistently seen is that some providers do train on popular evals and can underperform with minor changes to the task due to that.
Yes, harnesses, including Claude Code can prompt the models to write throwaway code to execute certain tasks, mostly Python, bash scripts or TS/JS, with there being some biases towards one over the other depending on the lab or specific model. Mainly for repetitive tool calls with no pre-existing/provided tools enabling it. Is in most instances a lot more efficient then a model e.g. doing a refactor that requires consistent variable renaming directly and around Opus 4.1/GPT-5, models have been trained to very consistently and accurately gauge when a task can benefit from such scratchpad scripts vs when that is inefficient/not useful.
Either DeepSWE [0] or FrontierCode [1], depending on personal goals and requirements. The later is more interesting for me personally, due to the design of the benchmark heavily grading "mergability", i.e. how the provided output is to review and whether a serious developer can easily parse it and'd be willing to merge the result. In my mind and with my private evals, for quite some time I've held firm that a model can have a higher ceiling but that has limited value if I do not feel truly confident in signing off on the code.
Very. Fable 5 is incredibly efficient token wise, second only to GPT-5.5 and is far more affordable run-to-run than the pure input/ouput costs would suggest. Task adherence, task inference, tool calling and task assessment are all significantly ahead of GPT-5.5, especially as the later strongly degrades the second compaction comes into the mix, I suspect because of OpenAIs obsessive optimisation of reasoning tokens into a hard to read (and thus also hard to compact) mess.
Fable 5 meanwhile has a reliable 1m context window and compaction that the few times I did eval it does also do well. Not quite as easy to trust as GPT-5.4, but that's mainly because with thats 272k context window I simply got more familiar with GPT-5.4s incredibly dependable compaction.
Purely concerning encoded information wise, Fable 5 is near or on the same level as Gemini 3.1 Pro in my limited test set focused on those tasks, which in very niche cases can make a difference even with coding, but the truest advantage for coding assistance (besides frontend/UX) is that the code Anthropic models provide is more parsable. Hard to explain, but I can read, follow and mentally map Fable 5 (and even Opus 4.5-4.8) output far more than GPT-5.4 or GPT-5.5 code.
Task orchestration and (more importantly) knowing when to recommend against using such vs Opus 4.8 is another strength of Fable 5 I've use liberally, there is an understanding of what a tasks requirements and the most optimal setup for success are, I have not yet seen before. Computer use is also solid, albeit not as token efficient as GPT-5.5 for my limited use cases.
Lastly, I will say that the classifier has become far less intrusive for me compared to the initial release. During the previous launch window, on Claude.ai I triggered the classifier for simple frontend tasks for regular (not security vocabulary containing) webpages. Now that is no longer the case. Inside Claude Code I occasionally triggered the classifier previously, but after the re-release, I only managed one, even when working with a privacy focused section of the code base containing a significant number of code comments with security and privacy focused wording. That one instance was rectified quickly by trying again, so I really am having a hard time following how others experience the issues some describe. I do have routing to Opus 4.8 without confirmation by me deactivated too, simply because I want to know if it ever happens, so it's not that I missed reroutings.
That all being said, we are still far from a stage where I'd not want to review the output, but yes, I do rate Fable 5 very highly. GPT-5.5 can have a similar ceiling but long horizon has become less usable over GPT-5.4 and in either case, parsing their output is (far more) of a chore. Maybe post training can address some of this, hopeful on the compaction front myself. Also interested in what happened to OpenAI models on AWS Trainium, I was expecting that to be a major boon for their commercial adoption, but haven't heard anything since then...
On the post training front, I am still hopeful that the Gemini team can finally get tool calling and task adherence to an acceptable level as we do need every competitor possible and purely considering the information density the model was trained with, they have great potential.
Please. If you told a customer support rep that you are the former US president [0], they would not hand over the account straight away because you asked nicely.
These models are great tools, but putting them and people on the same level does a disservice to our species and also is simply incorrect to what we know these models to be and their capabilities/limitations.
Looking at the first comparison, I will admit, I thought the issue was with the iPhones example. The button and slider below the image disappear, then fade back in after each press of the rotate button, a behaviour I have seen on iOS across many applications that irks me to no end. The Screenshot app being a particular bug bear of mine.
If you have a UX element that I will be able to interact with before and after an interaction, then keep it visible during the transformation, process, whatever. What UX gain is there in hiding these buttons during the rotation on the iPhone? It doesn't even look better, though appearance has been the altar that recent Apple software has sacrificed actual UX gains.
Will agree with the author though that these taps need to be processed independent of animation.
Isn't that a different issue from what the blog post described and easily solved by holding everyone who allows their UX elements to get pushed around, for whatever reason, to the fire?
That's great. Started using Base UI early on via 9ui [0] and found the primitives very pleasant to work with, especially if one wants to compose more complex components from other Base UI components. Maybe Shad can reduce some of the dependencies they rely on now.
Terminalbench numbers are publicly available. What is more interesting, why is that the only benchmark they highlight. Maybe 5.6 isn’t that far ahead of Fable 5 in DeepSWE and FrontierCode (which I consider the most useful and close to my evals + subjective experience)…
I remember them using these chart colours during the 5 launch, maybe even 4.1 back in the day. Don’t know why, maybe its their CI manual that’s been generated by gpt-3.5-turbo…
Purely subjective, but I tend to prefer reading Opus 4.8 output over GPT 5.5 code, even when the latter can have a higher overall ceiling. The former is just a bit more convenient to review.
Is this a new pre training run independent of 5.5s or post trained on it with Cerebras support and a rebrand of Pro mode at more usable speeds as Sol? The latter seems more likely to me, especially as 5.5 scales very well across its modes so separate branding could make sense, but I don’t see any clear information either way.
Physical store space, promotional contracts with vendors, special editions, bundles, etc.
I find it very silly, utterly unnecessary, but it is far from unprecedented [0] for this industry. I think it's very problematic for preservation and will only lead to more interest from groups trying to bypass their DRM because of this.
Are we seriously using a Manga and a sci-fi movie from the 60s as a source? On the note of basing ideas on pure fiction, how far along are the hyper loop and starship earth-to-earth? The later even made it into the IPO filing of this very serious company...
> Normally, for every diagnostic medical capability you need FDA approval. We’re starting by just giving you detailed body composition maps — and we’ll be submitting regular test results to the FDA for increased capabilities.
Ah yes, just "detailed body composition maps", nothing major. It's just radiology, not like people spend years of extensive education and have to sign off on every finding, often lying awake at night that they may have missed something. It's easy, don't let the Doctorpolice tell you otherwise. Seems very ̶T̶h̶e̶r̶a̶n̶o̶s̶ familiar. Also, not saying em dash automatically denote LLM writing, but come on, this whole thing reads very slopgenerated.
I have questions in general.
Why Midjourney? Do they have expertise? Even if so, why reuse a name that doesn't exactly denote reliable, consistent or trustworthy output? Why start as a spa with fancy LED lights clearly focused on experience over selling/leasing the whole-body implementation to third parties? Is the latter actually theres, how exactly does the licensing deal look and again, why them? Have they got any type of independent data to back up any of their findings? This just has the smell of something that, a few years from now everyone will be astounded that anyone ever believed this to be possible, for it is so patently ridiculous.
Never been a fan of image generation models for a variety of reasons, but this is downright dangerous, no way about it. Even if the technology as licensed works well, there are very good reasons why operating an MRI and seeing patients is not something you can do, just because you can afford to buy one. There is expertise needed here that, if this was coming from an established Medical Clinic and backed by research I'd be skeptical for such a spa setup to overcome, but again, this isn't even that. Best case scenario, this causes a VC to go bankrupt before the "spa" open and gets a front page on the goop magazine, worst case, patients are harmed, families destroyed and a comparatively minor penalty is administered/a pardon bought.
Not an assessment on the underlying concept/technology mind you, just the way Midjourney of all people are going about this.
I had a situation yesterday were I was forced to explain that requiring a public userID with a curl request is not a safety measure. This is worse, somehow.