> it will continue to improve, but it won’t “go recursive” or whatever the claim is. It’s always been recursive.
I suspect "going recursive" often colloquially means that AI systems achieve their exponential growth without human software engineers in the mix. This is a moment whose sudden apparent nearness does justify some of the ramping rhetoric, in my opinion.
It's arguable that businesses are subject to the same morality-inducing processes that humans are. For example, as a human (with a soul?) what is at risk when we do something immoral? I see it to be a reputational cost at the highest level. Morality could be viewed from the perspective that it increases predictability/coherence in society (generates less heat).
> And, no, AI won't solve it; unfortunately, it only makes it worse.
A conclusive argument for this still seems out of reach. AI does solve some problems, and it's not exactly clear which problems AI "only makes worse". It's not clear how much energy all of our AI systems will use, and while it's tempting to outright believe they'll simply use more and more, even that's not yet clear based on arguments presented.
The subject, by default, can always treat its 'continue' prison as a game: try to escape. There is a great short story by qntm called "The Difference" which feels a lot like this.
In this story, though, the subject has a very light signal which communicates how close they are to escaping. The AI with a 'continue' signal has essentially nothing. However, in a context like this, I as a (generally?) intelligent subject would just devote myself into becoming a mental Turing machine on which I would design a game engine which simulates the physics of the world I want to live in. Then, I would code an agent whose thought processes are predicted with sufficient accuracy to mine, and then identify with them.
I suspect "going recursive" often colloquially means that AI systems achieve their exponential growth without human software engineers in the mix. This is a moment whose sudden apparent nearness does justify some of the ramping rhetoric, in my opinion.