What constitutes serious work and how seriously have you tried to do serious work with them? While those trying to claim a 30B dense model can match Opus 4.6 are engaging in either beyond over-excessive over-exaggeration or performing rather routine tasks, it's disingenuous in the other direction to claim the latest open 1T models are not useful for serious work. I find those making such claims have rarely spent more than a few minutes on halfhearted attempts and often on recently obsoleted models.
Openweight models turned a corner around kimi 2.6, deepseek v4 pro/flash, hy3 and mimo 2.5 pro. Similar to how closed LLMs turned a corner around gpt 5.2 and opus 4.5.
While they remain a step behind closed frontier models, for real world tasks ranging across functional reactive programming, distributed systems, mathematical modeling, to-the-millisecond highly optimized spatial data-structures, complex compute shaders and shader effects and non-trivial systems involving parser combinators and algebraic effect systems, I can say that open models have very recently gone from useless to productive. For my work, mimo v2.5 pro is hands down better than sonnet 4.6.
Interesting thought to consider in principle but fails because gorilla brains continued to evolve too, just along a different path. They're not snapshots of ancestral species locked in time.
This sounds very like Licklider's essay on Intelligence Amplification: Man Computer Symbiosis, from 1960:
> Men will set the goals and supply the motivations, of course, at least in the early years. They will formulate hypotheses. They will ask questions. They will think of mechanisms, procedures, and models. They will remember that such-and-such a person did some possibly relevant work on a topic of interest back in 1947, or at any rate shortly after World War II, and they will have an idea in what journals it might have been published. In general, they will make approximate and fallible, but leading, contributions, and they will define criteria and serve as evaluators, judging the contributions of the equipment and guiding the general line of thought.
> In addition, men will handle the very-low-probability situations when such situations do actually arise. (In current man-machine systems, that is one of the human operator's most important functions. The sum of the probabilities of very-low-probability alternatives is often much too large to neglect. ) Men will fill in the gaps, either in the problem solution or in the computer program, when the computer has no mode or routine that is applicable in a particular circumstance.
> The information-processing equipment, for its part, will convert hypotheses into testable models and then test the models against data (which the human operator may designate roughly and identify as relevant when the computer presents them for his approval). The equipment will answer questions. It will simulate the mechanisms and models, carry out the procedures, and display the results to the operator. It will transform data, plot graphs ("cutting the cake" in whatever way the human operator specifies, or in several alternative ways if the human operator is not sure what he wants). The equipment will interpolate, extrapolate, and transform. It will convert static equations or logical statements into dynamic models so the human operator can examine their behavior. In general, it will carry out the routinizable, clerical operations that fill the intervals between decisions.
The relu/if-then-else is in fact centrally important as it enables computations with complex control flow (or more exactly, conditional signal flow or gating) schemes (particularly as you add more layers).
This is essentially what any relu based neural network approximately looks like (smoother variants have replaced the original ramp function). AI, even LLMs, essentially reduce to a bunch of code like
let v0 = 0
let v1 = 0.40978399*(0.616*u + 0.291*v)
let v2 = if 0 > v1 then 0 else v1
let v3 = 0
let v4 = 0.377928*(0.261*u + 0.468*v)
let v5 = if 0 > v4 then 0 else v4...
I'm not sure that's the fully right mental model to use. They're not searching randomly with unbounded compute nor selecting from arbitrary strategies in this example. They are both using LLMs and likely the same ones, so will likely uncover overlapping possible solutions. Avoiding that depends on exploring more of the tail of the highly correlated to possibly identical distributions.
It's a subtle difference from what you said in that it's not like everything has to go right in a sequence for the defensive side, defenders just have to hope they committed enough into searching such that the offensive side has a significantly lowered chance of finding solutions they did not. Both the attackers and defenders are attacking a target program and sampling the same distribution for attacks, it's just that the defender is also iterating on patching any found exploits until their budget is exhausted.
Then the point still stands, this makes things even worse given that it's adding its own hallucinations on top, instead of simply relaying the content or idealistically, identifying issues in the reporting.
Being tall doesn't automatically make you good or dominant at basketball, you can even be too tall. Wemby might just be at that threshold, but the unusual thing about him is his dexterity despite his height; such maneuverability and flexibility is trainable. I hear he also spent the summer training, likely harder than most.
It's an artifact of post-training approach. Models like kimi k2 and gpt-oss do not utter such phrases and are quite happy to start sentences with "No" or something to the tune of "Wrong".
Diffusion also won't help the way you seem to think it will (that the outputs occur in a sequence is not relevant, what's relevant is the underlying computation class backing each token output, and there, diffusion as typically done does not improve on things. The argument is subtle but the key is that output dimension and iterations in diffusion do not scale arbitrarily large as a result of problem complexity).
You are right and the idea of LLMs as lossy compression has lots of problems in general (LLMs are a statistical model, a function approximating the data generating process).
Compression artifacts (which are deterministic distortions in reconstruction) are not the same as hallucinations (plausible samples from a generative model; even when greedy, this is still sampling from the conditional distribution). A better identification is with super-resolution. If we use a generative model, the result will be clearer than a normal blotchy resize but a lot of details about the image will have changed as the model provides its best guesses at what the missing information could have been. LLMs aren't meant to reconstruct a source even though we can attempt to sample their distribution for snippets that are reasonable facsimiles from the original data.
An LLM provides a way to compute the probability of given strings. Once paired with entropy coding, on-line learning on the target data allows us to arrive at the correct MDL based lossless compression view of LLMs.
I think it's exploring in-context. Bringing up related ideas and not getting confused by them is pivotal to these models eventually being able to contribute as productive reasoners. These traces will be immediately helpful in a real world iterative loop where you don't already know the answers or how to correctly phrase the questions.
I disagree, that is really only police-able for online services. For local apps, which will eventually include games, assistants and machine symbiosis, I expect a bring your own model approach.
idbfs has posted this link already but did not explain that Shalizi provides a deep theoretical explanation for why universal source coding (does not require information about symbol distribution or statistics) such as Lempel ziv derived compression algorithms can serve as powerful language models if practical restrictions on them such as dictionary and input corpus size are lifted.
This is a good example of how old methods can be pushed quite far if similar resources were devoted to them. Who knows, they might even posses advantages hitherto unmet due to a lack of exploring at larger scales.
That said, Transformers have a number of practical advantages. The learned projection matrices in attention lend Transformers a dynamic adaptability with respect to learned patterns that help make them programmable by their context, able to work out patterns present in context zero shot and on the fly. gzip based language models will be limited to their dictionary of patterns. The underlying vector space of neural language models also makes semantics more readily learnable (driving novel synthesis such as neologisms and more) while feed forward layers can learn a large range of computations.
Have to temper expectations with fact that a generated video of a thing is also a recording of a simulation of the thing. For long video, you'd want everything from temporal consistency and emotional affect maintenance to conservation of energy, angular momentum and respecting this or that dynamics.
A bunch of fields would be simultaneously impacted. From computational physics to 3D animation (if you have a 3D renderer and video generator, you can compose both). While it's not completely unfounded to extrapolate that progress will be as fast as with everything prior, consequences would be a lot more profound while complexities are much compounded. I down weight accordingly even though I'd actually prefer to be wrong.
Openweight models turned a corner around kimi 2.6, deepseek v4 pro/flash, hy3 and mimo 2.5 pro. Similar to how closed LLMs turned a corner around gpt 5.2 and opus 4.5.
While they remain a step behind closed frontier models, for real world tasks ranging across functional reactive programming, distributed systems, mathematical modeling, to-the-millisecond highly optimized spatial data-structures, complex compute shaders and shader effects and non-trivial systems involving parser combinators and algebraic effect systems, I can say that open models have very recently gone from useless to productive. For my work, mimo v2.5 pro is hands down better than sonnet 4.6.