Sure but... this is the technology at the most expensive it will ever be. I'm impressed that o3 was able to achieve such high performance at all, and am not too pessimistic about costs decreasing over time.
> We train our models on a combination of an internal dataset consisting of 14 million video-text pairs
The paper is sorely lacking evaluation; one thing I'd like to see for instance (any time a generative model is trained on such a vast corpus of data) is a baseline comparison to nearest-neighbor retrieval from the training data set.
Interesting analogy. Google, like the priests, is acting out of mix of good intentions (protecting the public from perceived dangers) and self-interest (maintaining secular power, vs. a competitive advantage in the AI space). In the case of the priests, time has shown that their good intentions were misguided. I have a pretty hard time believing that history will be as unkind towards those who tried to protect minorities from biased tech, though of course that's impossible to judge in the moment.
This seems to be the case with a lot of "methodologies" like TDD, Agile, XP, etc. as well as "XXX considered harmful"-style proscriptions.
A simple idea ("hey, I was facing a tricky problem and this new way of approaching it worked for me. Maybe it will help you too?") mutates into a blanket law ("this is the only way to solve all the problems") and then pointy-haired folks notice the trend and enshrine it into corporate policy.
But Fred Brooks was right: there are no silver bullets. Do what works best for you/your team.
I completely agree. I'll use TDD when implementing a function "where the code is mostly self-contained with few external dependencies and the expected inputs and outputs are well defined and known ahead of time" and where the function is complex enough that I'm uncertain about its correctness. Though I find I usually do property testing, or comparison to a baseline on random inputs, similar to the quicksort example in the blog post (against a slow, naive implementation of the function; or an older version of the function, if I'm refactoring) rather than straight TDD.
When debugging, I'll also turn failure cases into unit tests and add them to the CI. The cost to write the test has already been paid in this case, so using them to catch regressions is all-upside.
System tests are harder to do (since they require reasoning about the entire program rather than single functions) but in my experience are the most productive, in terms of catching the most bugs in least time. Certainly every minute spent writing a framework for mocking inputs into unit tests should probably have been spent on system testing instead.