I’ve started to pick up on some of the “unwilling to dig deeply into the humans perspective” & “provide ideation and then run with it” in 4.7. I actually think it’s consistent with confabulation, now that they’ve removed most of the models ability to observe its own reasoning in 4.7.
The effect is over-complicated engineering that takes way more time to review as to its right-size for the job.
A couple drawbacks so far via our scenario-based tests:
1. You can't ask the model to "think hard" about something anymore - model decides
2. Reasoning traces are no longer true to the thinking – vs opus 4.6, they really are summaries now
3. Reasoning is no longer consciously visible to the agent
They claim the personality is less warm, but I haven't experienced that yet with the prompts we have – seems just as warm, just disconnected from its own thought processes. Would be great for our application if they could improve on the above!
I think spatial tokens could help, but they're not really necessary. Lots of physics/physical tasks can be solved with pencil and paper.
On the other hand, it's amazing that a 512x512 image can be represented by 85 tokens (as in OAI's API), or 263 tokens per second for video (with Gemini). It's as if the memory vs compute tradeoff has morphed into a memory vs embedding question.
This dichotomy reminds me of the "Apple Rotators - can you rotate the Apple in your head" question. The spatial embeddings will likely solve dynamics questions a lot more intuitively (ie, without extended thinking).
We're also working on this space at FlyShirley - training pilots to fly then training Shirley to fly - where we benefit from established simulation tools. Looking forward to trying Fei Fei's models!
They're building on a false premise that human equivalent performance using cameras is acceptable.
That's the whole point of AI - when you can think really fast, the world is really slow. You simulate things. Even with lifetimes of data, the cars still will fail in visual scenarios where error bars on ground truth shoot through the roof.
Elon seems to believe his cars will fail in similar ways to humans because they use cameras. False premise. As Waymo scales, human just isn't good enough, except for humans.
The effect is over-complicated engineering that takes way more time to review as to its right-size for the job.
Feels like hiding things, however.