Agreed on the likely mechanism. I'm not sure "overfitting" is even the right description. These things are of course absurdly complicated, and evaluating their quality down to a single number involves a lot of judgement and trade-offs. I think it's more "you get what you measure" which is true in human organizations too. Define a KPI and people work hard to make it go up, even if it's not quite right or has bad side-effects.
It’s not that. The tumors they create in mice are just really fragile compared to natural tumors. Natural tumors that actually cause problems have probably grown for years and learned to avoid immune responses. They’re not densely packed and easy to get into all the parts.
This kind of treatment triggers an immune response, which the model tumors have never had to fight before. So it’s just an easy target.
The blog is highly suspect, but the study is real. That said it’s not a big deal.
Curing cancer in a mouse model is not at all uncommon in new therapies. Mouse models like this are vastly easier to treat than real world cancer for a bunch of reasons. Fully curing mice is the baseline for a treatment to even be considered for further evaluation. And even then very few therapies end up succeeding in humans - low single digit percent.
So yes, another possible treatment. But not at all a breakthrough.
Another year, and OpenAI comes up with yet another naming scheme for their models. First it was integers (GPT2, GPT3). Then they added friendly names (remember Ada, Babbage, Curie, Davinci?), but decided against it. Instead we got dot integers (GPT3.5), then then letter-number modifiers (o1), plus word modifiers like o1-pro, o3-mini, or -mini-high, or codex, codex-max, Pro, etc.
Now they've got friendly cosmic names. And this time they want us to believe that this time they're gonna stick to a naming convention? I'll believe it when they do 3 releases in a row without inventing a new naming scheme.
As somebody who has spent a lot more than 10 minutes trying to figure out why CORS was blocking what seemed legitimate, I sympathize with people doing the wrong thing, and disagree with your assertion that it’s not that complicated. Maybe I’m just slow. But objectively I know I’m not.
I think there’s a reasonable argument that a burst bubble will cause prices to drop. Prices are very high because they’re trying to justify these trillion dollar valuations on IP alone. If that fantasy goes away then prices will fall down to just silicon and electricity, which looks more like Chinese model prices. Hard to say how it will play out but the direction isn’t obvious to me.
Correct. We used to think that because NN optimization is non-convex there are all these local minima. Now we know that once you get past the very early parts of training from random init, the loss surface is fairly smooth, and not really convex, but close enough in a bunch of ways - linear combinations of trained models are pretty much always valid combinations. You can think of fine tunings as deltas on the original model which can be summed together successfully. I think this paper first showed that to me: https://arxiv.org/pdf/1802.10026 which was 8 years ago now.
It’s definitely for hardware reasons. They have been aggressively improving the vector math capabilities in their chips, but as anybody who has tried to run a local LLM will tell you, newer hardware works better and you’re always limited in what you can do.
Understanding and deciding to rely upon are not the same. Complexity is one of the biggest challenges in real world software systems. Keeping a subsystem’s responsibilities simple can do a lot for a system’s reliability. Most teams have extremely mature systems for managing complexity in their code - tests, CICD etc. Sure you CAN build all those things for your database too, but it’s more work. Most teams I’ve worked on choose to minimize database migrations because it’s a lot of work to make that part of the system as robust against mistakes as code is. Choosing to ignore a feature in your database is often a very rational pragmatic choice aimed at keeping complexity under control.
You could. But then you’re also building from scratch HA failover, backups, replica management, monitoring, etc - cloud vendor managed RDBMS come with lots of niceties. All of which are possible to set yourself. But a hassle, and difficult to make bullet proof.
They clearly have an agenda, but also openly acknowledge that public surveillance is a two sided coin, balancing public safety and convenience with privacy. Some of the risks they identify are real, but others are unabashedly exaggerated.
Correct. All major OSes stopped broadcasting the preferred SSID list by 2017, with Android and Linux being the last. Apple stopped in 2014. Windows by 2009.
I’m sorry you are used to working with out of touch leadership. Not all companies are like that. Even big ones can have smart, empathetic leaders. Although very often money gets in the way of empathy.
GPT 5.5 still invents facts rather than looking them up, and manages to come across both as condescending and sycophantic. It feels like talking to a used car salesman.