NASA had real safety issues in the past - Space Shuttle is a testament to that. But if this is the fix, they clearly overcorrected.
There's some sort of happy middle ground between flying a ship that has a very significant chance of a catastrophic failure killing all crew and can't even fly unmanned, and almost not letting Ingenuity fly because the chance of failure was deemed too great. NASA didn't find it.
A big part of the reason for useless Gateway was that NASA wanted post-ISS missions, but was too afraid to roll with a permanent Moon base instead. The other big part was that Orion sucked, and somehow, neither "get Orion to suck less" nor "roll with Orion, provision for HLS to do more" was in the cards. This only changed under Isaacman.
For every Curiosity and JWST, there's ARM and Artemis. It seems like it's getting better now, but very slowly.
Sure, your ultra paranoid checking of everything might catch an extremely rare bug caused by something like interactions between a benign code change and a build system. But is it worth slowing down the development process by that much?
Is it worth missing out on an entire generation of technology, like what happened with US and the shift from 00s drone warfare and 20s drone warfare?
NASA's failures as of late are less "dramatic explosions" and more "delays", "cost overruns" and "lack of ambition so severe it borders on criminal".
The last time NASA caught any serious flak was what, the Starliner shitshow? And that was just splash damage from Boeing getting dunked on by everyone at once.
In Ukraine, the first place that was bombed was the red tape factory.
The drone industry was allowed to basically "do whatever as long as it works", consequences be damned. So they use civilian motors, batteries and SoCs, sketchy firmware with zero code inspection, and more. Does it work perfectly? No. It works well enough.
I wonder if anyone is going to learn a lesson about overregulation.
I'm not sure if "AI for red tape mitigation" is a thing, but "AI for killer drones" sure is. I suspect that "killer drones are insufficiently smart" is easier to fix with AI than "too much red tape". Because the amount of red tape, if unopposed, will expand to consume any capacity of dealing with it, AI or not.
I find the idea of all the AI tools I use keeping all the (mangled, decompiled, and not even mine in the first place) code I point it at, and then using it in training to be hilarious.
And if it results in the next generation of AIs having more suspicious knowledge of proprietary software internals and better reverse engineering capabilities, then all the better.
Mostly, because a lot of game engines are ancient relics, tracing their lineage all the way back to Quake 1.
The development practices are not exactly up to date, and game development is in no hurry to change. It doesn't help that software development wages there are not at all competitive - game development selects for passion, not skill. People who want to build robust modern codebases and people who want to build AAA games are different people. So there aren't many game devs who want to push for better test coverage.
But it's also because game engines are dealing with many, many things that are hard to test for.
You know how messy it is to test a website for "does this layout look right" or "can you navigate from A to B"? Now multiply that by complex 3D geometry. A lot of what game engines do is dealing with complex 3D geometry, where the primary verification is "does it look right" and "does this interaction feel right". Which is why game development traditionally has wide human QA, and slim unit testing.
Only now do we have software that can sort of, semi-reliably, automate testing for "does it look right".
It's basically re-linking the executable. Way too easy to shoot foot - miss one reference and things break in a spectacular, or, worse, subtle fashion. Which means: you definitely need to know where all the references are and what they point to.
References are under no obligation to be presented in a sensible, decompiler-friendly way.
Which is why "jump out into the patch, jump back in" is such a staple of patching. The other way is the hard way, full of pitfalls and potential issues.
And today's records on ARC-AGI-2 are >80%. Held by LLMs that use text modality for input.
The issue with multimodal training is that it doesn't seem to bring a step-change improvement in spatial reasoning either. It helps some, but the gain is surprisingly small compared to the data and compute expended. What it helps with the most is, unsurprisingly, spatial reasoning when using image inputs.
Maybe there are gains we don't know how to extract there.
Overall, LLM performance at spatial tasks is improving, especially on things like puzzles, but that mix of "commonsense + spatial" in the same task still eludes them.
Do birds expose enough of their cognition through birdsong?
Do birds expose locomotion-relevant functions specifically through birdsong?
Do we have enough birdsong data available to start solving the inverse problem?
If "yes" on all, then we might be able to do it.
I expect "no" on most of that, for birds. But humans treat language as an interface to their higher cognitive functions, and stockpile language data. That looks an awful lot like a set of two "yes".
The last open question is: is there enough spatial reasoning reflected in the language data we have?
It's plausible that spatial reasoning is too evolutionary old and too low-level, too far removed from higher cognition, to leak into language heavily. And it's also plausible that existing LLM architecture is uniquely poorly suited to learning spatial reasoning - higher cognitive functions involved in things like writing code or even composing poetry are a better fit for the architecture. And it's plausible that we're underestimating just how complex spatial reasoning truly is - Moravec's paradox strikes again.
We know that LLMs perform poorly and improve slowly on spatial reasoning tasks, but not exactly why. And progress on things like ARC-AGI series shows that they're not completely inept.
What's your evidence of that? That AGI requires a truly novel architecture, and not just another iterative "LLM but with an extra trinket and wheels that spin ten times faster".
It's a "commonsense spatial reasoning/problem solving" kind of problem. LLMs fail at spatial reasoning forever.
What humans "easily" solve in seconds with raw spatial reasoning LLMs often find easier to solve by invoking A* or a constraint solver.
Might be that text data is particularly bad at teaching that to LLMs. Or that being good at spatial reasoning requires true recurrence, and autoregressive chain of thought is a poor substitute. Or it might be that human brain was optimized by evolution for solving spatial problems in open ended 3D environments for hundreds of millions of years, optimized for language for mere hundreds of thousands of years, and only optimized for writing computer code for a few decades at most.
The current frontier is halfway competent at benign closed 2D work, but still completely fumbles anything remotely close to open ended real world 3D work. It's getting better, but very slowly.
The classifier is about as refined as a brick to the face.
You can ask it elementary school grade biology trivia, or obscure facts about recently documented insect species, and both will downgrade to Opus 4.8 straight away.
And Opus itself was already bad with biotech questions. The fact that they somehow made it WORSE for Fable is mindboggling.
Every time things like this come up, I can't help but think of the ending of Inception.
It's less that you're convinced it's real and more that you no longer care if it is. "Feels real enough" is good enough.
I'm a technical user first, so I'm not sure if models have improved for RP the way they improved for applied STEM tasks and technical brainstorming. But if there is an improvement curve there, I wouldn't be surprised if this only grows in popularity.
That does work. Even if you drop the "specialized" part. Ensembles of the same architecture at the same scale trained on the same data do outperform a singular model of the same line - especially on corner cases. Successes of an ensemble correlate stronger than failures do.
The usual argument against is that if you have "a number of specialized models" that perform well in ensemble, you can take that ensemble, and distill it into a single larger model (dense or integrated sparse, like MoE), and get the same improvement in performance with an efficiency win.
This works because having those "specialized models" duplicates a lot of the highly conserved "low level" wiring that's required for a model to function at all. As such, you end up running a small scale version of the same "backbone" computational processes many times. "Merging" those models into a larger, denser model allows for a singular strong "backbone" to be used for everything.
Less weird unexpected failures, more innate ability to handle edge cases gracefully. Quite important when you're running high on automation and low on oversight.
It's almost like a twisted mirror of Conway's law.