The "AI 2027" Scenario: How realistic is it?(garymarcus.substack.com)
garymarcus.substack.com
The "AI 2027" Scenario: How realistic is it?
https://garymarcus.substack.com/p/the-ai-2027-scenario-how-realistic
239 comments
Ya, multiple failed predictions is an indicator of systemically bad predictors imo. That said, Scott Alexander usually does serious analysis instead of handwavey hype, so I tend to believe him more than many others in the space.
My somewhat native take is that we’re still close to peak hype, AI will under deliver on the inflated expectations, and we’ll head into another “winter”. This pattern has repeated multiple times, so I think it’s fairly likely based on that alone. Real progress is made during each cycle, i think humans are just bad at containing excitement
My somewhat native take is that we’re still close to peak hype, AI will under deliver on the inflated expectations, and we’ll head into another “winter”. This pattern has repeated multiple times, so I think it’s fairly likely based on that alone. Real progress is made during each cycle, i think humans are just bad at containing excitement
I think you mean "somewhat naive" instead of "somewhat native". :)
But, yes, this, in my mind the peak[1] bubble times ended with the DeepSeek shock earlier this year, and we are slowly on the downward trajectory now.
It won't be slow for long, once people start realizing Sama was telling them a fairy tale, and AGI/ASI/singularity isn't "right around the corner", but (if achievable at all) at least two more technology triggers away.
We got reasonably useful tools out of it, and thanks to Zuck, mostly for free (if you are an "investor", terms and conditions apply).
[1] https://en.wikipedia.org/wiki/Gartner_hype_cycle
But, yes, this, in my mind the peak[1] bubble times ended with the DeepSeek shock earlier this year, and we are slowly on the downward trajectory now.
It won't be slow for long, once people start realizing Sama was telling them a fairy tale, and AGI/ASI/singularity isn't "right around the corner", but (if achievable at all) at least two more technology triggers away.
We got reasonably useful tools out of it, and thanks to Zuck, mostly for free (if you are an "investor", terms and conditions apply).
[1] https://en.wikipedia.org/wiki/Gartner_hype_cycle
> They went from "this represents roughly our median guess" in the website to "maybe think of it as an 80th percentile version of the fast scenario that we don't feel safe ruling out" in followup discussions.
His post also just reads like they think they're Hari Seldon (oh Daniel's modal prediction, whew, I was worried we were reading fanfic) while being horoscope-vague enough that almost any possible development will fit into the "predictions" in the post for the next decade. I really hope I don't have to keep reading references to this for the next decade.
His post also just reads like they think they're Hari Seldon (oh Daniel's modal prediction, whew, I was worried we were reading fanfic) while being horoscope-vague enough that almost any possible development will fit into the "predictions" in the post for the next decade. I really hope I don't have to keep reading references to this for the next decade.
> while being horoscope-vague enough that almost any possible development will fit into the "predictions" in the post for the next decade.
This is a recurring theme in rationalist blogs like Scott Alexander’s: They mix a lot of low-risk claims in with heavily hedged high-risk claims. The low risk claims (AI will continue to advance) inevitably come true and therefore the blog post looks mostly accurate in hindsight.
When reading the blog post in the current context the hedging goes mostly unnoticed because everyone clicked on the article for the main claim, not the hedging.
When reviewing blog posts from the past that didn’t age well, that hedging suddenly becomes the main thing their followers want you to see.
So in future discussions there are two outcomes: He’s always either right or “not entirely wrong”. Once you see it, it’s hard to unsee. Combine that with the almost parasocial relationship that some people develop with prominent figures in the rationalist sphere and there are a lot of echo chambers that, ironically, think they’re the only rational ones who see it like it really is.
This is a recurring theme in rationalist blogs like Scott Alexander’s: They mix a lot of low-risk claims in with heavily hedged high-risk claims. The low risk claims (AI will continue to advance) inevitably come true and therefore the blog post looks mostly accurate in hindsight.
When reading the blog post in the current context the hedging goes mostly unnoticed because everyone clicked on the article for the main claim, not the hedging.
When reviewing blog posts from the past that didn’t age well, that hedging suddenly becomes the main thing their followers want you to see.
So in future discussions there are two outcomes: He’s always either right or “not entirely wrong”. Once you see it, it’s hard to unsee. Combine that with the almost parasocial relationship that some people develop with prominent figures in the rationalist sphere and there are a lot of echo chambers that, ironically, think they’re the only rational ones who see it like it really is.
> Hari Seldon is a fictional character in the Foundation series of novels by Isaac Asimov. In his capacity as mathematics professor at Streeling University on the planet Trantor, Seldon develops psychohistory, an algorithmic science that allows him to predict the future in probabilistic terms
- https://en.m.wikipedia.org/wiki/Hari_Seldon
- https://en.m.wikipedia.org/wiki/Hari_Seldon
Yud is also something like 50% sure we'll die in a few years - if I'm not wrong
I guess they'll have to update their a priori % if we survive
I guess they'll have to update their a priori % if we survive
I think Yudkowsky is more like 90% sure of us all dying in a few (<10) years.
I mean, this is their new book: https://ifanyonebuildsit.com/
I mean, this is their new book: https://ifanyonebuildsit.com/
There are a lot of people that believe most of us will die within the next 10 years, and a rational discussion of these subjects is largely based in the fact that for the last three generations, we have faced numerous existential threats that instead of solving them, have instead all had the can kicked down the road.
Eventually what inevitably happens is you get convergence in time where you simply do not have the resources, and with the risk factors today, that convergence my cause societal failure.
Super Intelligent AI alone, yeah that probably is not a threat because its so highly (astronomically) unlikely, but socio-economic collapse to starvation; now that's a very real possibility when you create something that destroys the ability for an individual to form capital, or breaks other underlying aspects which underpin all of societal organization going back hundreds of years.
Now these things won't happen overnight, but that's not the danger either. The danger is the hysteresis, or in other words by the time you find out and can objectively show its happening to react, its impossible to change the outcome. Your goose is just cooked as a species, and the cycle of doom just circles until no ones left.
Few realize that food today is wholly dependent on Haber-Bosch chemistry. You get 4x less yield without it, and following Catton in a post-extraction phase sustainable population numbers may be fractional compared to last century (when the population was 4bn). People break quite easily under certain circumstances and so any leaders following MAD doctrine will likely actually use it when they realize everything is failing and what's ahead.
These are just things that naturally happen when the mechanics of things that are long forgotten which underpin the way things work fail to ruin. The loss of objective reality is a warning sign of such things on the horizon.
Eventually what inevitably happens is you get convergence in time where you simply do not have the resources, and with the risk factors today, that convergence my cause societal failure.
Super Intelligent AI alone, yeah that probably is not a threat because its so highly (astronomically) unlikely, but socio-economic collapse to starvation; now that's a very real possibility when you create something that destroys the ability for an individual to form capital, or breaks other underlying aspects which underpin all of societal organization going back hundreds of years.
Now these things won't happen overnight, but that's not the danger either. The danger is the hysteresis, or in other words by the time you find out and can objectively show its happening to react, its impossible to change the outcome. Your goose is just cooked as a species, and the cycle of doom just circles until no ones left.
Few realize that food today is wholly dependent on Haber-Bosch chemistry. You get 4x less yield without it, and following Catton in a post-extraction phase sustainable population numbers may be fractional compared to last century (when the population was 4bn). People break quite easily under certain circumstances and so any leaders following MAD doctrine will likely actually use it when they realize everything is failing and what's ahead.
These are just things that naturally happen when the mechanics of things that are long forgotten which underpin the way things work fail to ruin. The loss of objective reality is a warning sign of such things on the horizon.
> We do not mean that as hyperbole. We are not exaggerating for effect. We think that is the most direct extrapolation from the knowledge, evidence, and institutional conduct around artificial intelligence today. In this book, we lay out our case.
We're real researchers, so we make our definitely scientific case available to anyone who will give us $15-$30! It's the most important book ever, says some actor. Read it, so we all don't die!
For Christ's sake, how does anyone take this Harry Potter fanfiction writer serially?
Because of what else he writes besides the fanfic. (Better question is why anyone takes JKR herself seriously).
But if you insist on only listening to people with academic acolades or industrial output, there's this other guy who got the Rumelhart Prize (2001), Turing Award (2018), Dickson Prize (2021), Princess of Asturias Award (2022), Nobel Prize in Physics (2024), VinFuture Prize (2024), Queen Elizabeth Prize for Engineering (2025), Order of Canada, Fellow of the Royal Society, and Fellow of the Royal Society of Canada
That's one person with all that, and he says there's a "10 to 20 per cent chance" that AI would be the cause of human extinction within the following three decades, and "it is hard to see how you can prevent the bad actors from using [AI] for bad things.": https://en.wikipedia.org/wiki/Geoffrey_Hinton
Myself, I'm closer to Hinton's view than Yudkowsky's: path dependency, i.e. I expect that before we get existential threat from AI, we get catastrophic economic threat the precludes existential threat.
But if you insist on only listening to people with academic acolades or industrial output, there's this other guy who got the Rumelhart Prize (2001), Turing Award (2018), Dickson Prize (2021), Princess of Asturias Award (2022), Nobel Prize in Physics (2024), VinFuture Prize (2024), Queen Elizabeth Prize for Engineering (2025), Order of Canada, Fellow of the Royal Society, and Fellow of the Royal Society of Canada
That's one person with all that, and he says there's a "10 to 20 per cent chance" that AI would be the cause of human extinction within the following three decades, and "it is hard to see how you can prevent the bad actors from using [AI] for bad things.": https://en.wikipedia.org/wiki/Geoffrey_Hinton
Myself, I'm closer to Hinton's view than Yudkowsky's: path dependency, i.e. I expect that before we get existential threat from AI, we get catastrophic economic threat the precludes existential threat.
I do say the same think about JKR, btw. And for the same things, because the content she writes. I think you focused on the fanfic part and not the part where I'm criticizing where they say their stuff is the most important to keep humanity alive and they're charging money for it. Meanwhile, you may notice in academia we publish papers to make them freely available, like on arXiv. If it is that important that people need to know, you make it available.
The second person, Hinton, is not as good of an authority as you'd expect. Though I do understand why people take him seriously. Fwiw, his Nobel was wildly controversial. Be careful, prizes often have political components. I have degrees in both CS and physics (and am an ML researcher) and both communities thought it was really weird. I'll let you guess which community found it insulting.
I want to remind you, in 2016 Hinton famously said[0]
I care very much about alignment myself[4,5,6,7]. The reason I push back on Yud and others making claims like they do is because they are actually helping create the future they say we should be afraid of. I'm not saying they're evil or directly making evil superintelligences. Rather, they're pulling attention and funds away from the problems that need to be solved. They are guessing about things we don't need to guess about. They are making confidently asserting claims we know to be false (to be able to make accurate predictions requires accurate understanding[8]). Without being able to openly and honestly speak to the limitations of our machines (mainly blinded by excitement), we create these exact dangers we worry about. I'm not calling for a pause on research, I'm calling for more research and more people to pay attention to the subtle nature of everything. In a way I am saying "slow down" but only in that I'm saying don't ignore the small stuff. We move so fast that we keep pushing off the small stuff, but the AI risk comes through the accumulation of debt. You need to be very careful to not let that debt get out of control. You don't create safe systems by following the boom and bust hype cycles that CS is so famous for. You don't just wildly race to build a nuclear reactor and try to sell it to people while it is still a prototype.
[0] https://fastdatascience.com/ai-in-healthcare/ai-replace-radi...
[1] https://www.reddit.com/r/singularity/comments/1dhlvzh/geoffr...
[2] https://youtu.be/Yf1o0TQzry8?t=449
[3] https://www.youtube.com/watch?v=hV41QEKiMlM
[4] https://news.ycombinator.com/item?id=44068943
[5] https://news.ycombinator.com/item?id=44070101
[6] https://news.ycombinator.com/item?id=44017334
[7] https://news.ycombinator.com/item?id=43909711
[8] This is the link to [1,2]. I mean you can reasonably create a data generating process that is difficult or impossible to distinguish from the actual data generating process but you have a completely different causal structure, confounding variables, and all that fun stuff. Any physicist will tell you that fitting the data is the easy part (it isn't easy). Interpreting and explaining the data is the hard part. That hard part is the building of the causal relationships. It is the "understanding" Hinton and Sutskever claim.
The second person, Hinton, is not as good of an authority as you'd expect. Though I do understand why people take him seriously. Fwiw, his Nobel was wildly controversial. Be careful, prizes often have political components. I have degrees in both CS and physics (and am an ML researcher) and both communities thought it was really weird. I'll let you guess which community found it insulting.
I want to remind you, in 2016 Hinton famously said[0]
| Let me start by saying a few things that seem obvious. I think if you work as a radiologist, you're like the coyote that's already over the edge of the cliff, but hasn't yet looked down so doesn't realize there's no ground beneath him. People should stop training radiologists now. It's just completely obvious that within 5 years that deep learning is going to be better than radiologists because it's going to get a lot more experience. It might take 10 years, but we've got plenty of radiologists already.
I care very much about alignment myself[4,5,6,7]. The reason I push back on Yud and others making claims like they do is because they are actually helping create the future they say we should be afraid of. I'm not saying they're evil or directly making evil superintelligences. Rather, they're pulling attention and funds away from the problems that need to be solved. They are guessing about things we don't need to guess about. They are making confidently asserting claims we know to be false (to be able to make accurate predictions requires accurate understanding[8]). Without being able to openly and honestly speak to the limitations of our machines (mainly blinded by excitement), we create these exact dangers we worry about. I'm not calling for a pause on research, I'm calling for more research and more people to pay attention to the subtle nature of everything. In a way I am saying "slow down" but only in that I'm saying don't ignore the small stuff. We move so fast that we keep pushing off the small stuff, but the AI risk comes through the accumulation of debt. You need to be very careful to not let that debt get out of control. You don't create safe systems by following the boom and bust hype cycles that CS is so famous for. You don't just wildly race to build a nuclear reactor and try to sell it to people while it is still a prototype.
[0] https://fastdatascience.com/ai-in-healthcare/ai-replace-radi...
[1] https://www.reddit.com/r/singularity/comments/1dhlvzh/geoffr...
[2] https://youtu.be/Yf1o0TQzry8?t=449
[3] https://www.youtube.com/watch?v=hV41QEKiMlM
[4] https://news.ycombinator.com/item?id=44068943
[5] https://news.ycombinator.com/item?id=44070101
[6] https://news.ycombinator.com/item?id=44017334
[7] https://news.ycombinator.com/item?id=43909711
[8] This is the link to [1,2]. I mean you can reasonably create a data generating process that is difficult or impossible to distinguish from the actual data generating process but you have a completely different causal structure, confounding variables, and all that fun stuff. Any physicist will tell you that fitting the data is the easy part (it isn't easy). Interpreting and explaining the data is the hard part. That hard part is the building of the causal relationships. It is the "understanding" Hinton and Sutskever claim.
Do you feel that you are shifting goalposts a bit when quibbling over whether AI will kill everyone in 2030 or 2035? As of 10 years ago, the entire conversation would have seemed ridiculous.
Now we're talking about single digit timeline differences to the singularity or extinction. Come on man.
Now we're talking about single digit timeline differences to the singularity or extinction. Come on man.
> 10 years ago, the entire conversation would have seemed ridiculous
Bostrom's book[1] is 11 years old. The Basilisk is 15 years old. The Singularity summit was nearly 20 years ago. And Yudkowsky was there for all of it. If you frequented LessWrong in the 2010s, most of this is very very old hat.
[1]: https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang...
[2]: Ford (2015) "Our Fear of Artificial Intelligence", MIT Tech Review: https://www.technologyreview.com/2015/02/11/169210/our-fear-...
Bostrom's book[1] is 11 years old. The Basilisk is 15 years old. The Singularity summit was nearly 20 years ago. And Yudkowsky was there for all of it. If you frequented LessWrong in the 2010s, most of this is very very old hat.
[1]: https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang...
[2]: Ford (2015) "Our Fear of Artificial Intelligence", MIT Tech Review: https://www.technologyreview.com/2015/02/11/169210/our-fear-...
It is a bit disquieting though that these predictions instead of being pushed farther away are converging to a time even closer than originally imagined. Some breakthroughs and doomsday scenarios are constantly placed thirty years into the future; this seems to be actually getting closer earlier than imagined.
For the people imagining it yes
For many of us the conversation hasn't gotten any less ridiculous just because computers can talk now
For many of us the conversation hasn't gotten any less ridiculous just because computers can talk now
> just because computers can talk now
I find it astounding that some people appear completely unable to grasp what this really means and what the implications are.
I find it astounding that some people appear completely unable to grasp what this really means and what the implications are.
I see them as funhouse mirrors, the kind that reflect your image to make you skinny or fat, except they do it with semantics, big deal. I've never had an interaction with an llm that wasnt just repeating what I said more verbosely, or with compressed fuzzy facts sprinkled in.
There is no machine spirit that exists in a box separately from us, it's just a means for people to amplify and multiply their voice into ten thousand sock puppet bot accounts, that's all I'm able to grasp anyway. Curious to hear your experience that's led you to believe something different.
There is no machine spirit that exists in a box separately from us, it's just a means for people to amplify and multiply their voice into ten thousand sock puppet bot accounts, that's all I'm able to grasp anyway. Curious to hear your experience that's led you to believe something different.
Well, the first goal was 1997, but Skynet sure screwed that up.
Well, they can still go back in time and change things
I'm in my 30s and remember my friend in middle school showing me a website he found with an ominous countdown to Kurzweil's "singularity" in 2045.
> ominous countdown to Kurzweil's "singularity" in 2045
And then it didn't happen?
And then it didn't happen?
Not between 2027 and 2032 anyway.
I mean... neither of those is going to happen so it's pretty silly.
Yes and no, is it actually important if it's 2027 or 28 or 2032? The scenario is such that a difference of a couple of years is basically irrelevant.
> The scenario is such that a difference of a couple of years is basically irrelevant.
2 years left and 7 years left is a massive difference, it is so much easier to deal with things 7 years in the future especially since its easier to see as we get closer.
2 years left and 7 years left is a massive difference, it is so much easier to deal with things 7 years in the future especially since its easier to see as we get closer.
Yeah for example we had decades to tackle climate change and we easily over came the problem
Look at military tech 1941 and 1946, its a massive difference. 5 year time difference means a ton when people take it seriously.
Only reason climate change isn't solved is that people don't care enough.
Only reason climate change isn't solved is that people don't care enough.
> people don't care enough
I think your parent meant exactly this.
I think your parent meant exactly this.
The reason climate change isn’t solved is that it’s a problem that can’t be solved by technology alone. We already have all the technology we need to decarbonize. We lack the political will to make the necessary investment.
Also, the relevant manifold prediction has low odds:
https://manifold.markets/IsaacKing/ai-2027-reports-predictio...
Well, if it happens in 2028 instead of 2027, I don't think anyone will be justified in yelling at them for being wrong.
But I can't help notice that in one year, the expected arrival date slipped by... one year. That does not bode well...
But I can't help notice that in one year, the expected arrival date slipped by... one year. That does not bode well...
The other writings from Scott Alexander on scientific racism are also another good point imho
What specifically would you highlight as being particularly egregious or wrong?
As a general rule, "it's icky" doesn't make something false.
As a general rule, "it's icky" doesn't make something false.
And it doesn't make it true either
Human biodiversity theories are a bunch of dogwhistles for racism
https://en.m.wikipedia.org/wiki/Human_Biodiversity_Institute
And his blog's survey reports a lot of users actually believing in those theories https://reflectivealtruism.com/2024/12/27/human-biodiversity...
(I wasn't referring to this Ai 2027 in specific)
Human biodiversity theories are a bunch of dogwhistles for racism
https://en.m.wikipedia.org/wiki/Human_Biodiversity_Institute
And his blog's survey reports a lot of users actually believing in those theories https://reflectivealtruism.com/2024/12/27/human-biodiversity...
(I wasn't referring to this Ai 2027 in specific)
Try steel manning in order to effectively persuade. This comment does not address the argument being made it just calls a field of study icky. The unfortunate reality is that shouting down questions like this only empowers the racist HBI people who are effectively leeches
> Try steel manning in order to effectively persuade. This comment does not address the argument being made it just calls a field of study icky.
Disagree (the article linked in the GP is a great read with extensive and specific citations) and reminder that you can just make the comment you'd like to see instead of trying to meta sea lion it into existence. Steel man away.
Disagree (the article linked in the GP is a great read with extensive and specific citations) and reminder that you can just make the comment you'd like to see instead of trying to meta sea lion it into existence. Steel man away.
The article linked is literally nothing but a collection of Alexander's comments and comments on his blog. It in no way provides any reason to believe he is wrong, it just laughs at him, strawmaning his takes.
He just wrote a piece about the Lynn IQ maps that is practically a self-parody. Nobody needs to strawman him on this topic. Here is a direct quote:
Isn't It Super-Racist To Say That People In Sub-Saharan African Countries Have IQs Equivalent To Intellectually Disabled People? No. In fact, it would be super-racist not to say this!
The horse is completely out of the barn on this controversy about Alexander's views.
Isn't It Super-Racist To Say That People In Sub-Saharan African Countries Have IQs Equivalent To Intellectually Disabled People? No. In fact, it would be super-racist not to say this!
The horse is completely out of the barn on this controversy about Alexander's views.
He is 100% right that it is more racist to ignore the IQ difference between countries and therefore the factors that create them than it is to pretend all countries have the same average iq. The only way to help people who have been hurt by environmental factors that hurt development is to study those factors and who they effect.
No, there are no such thing as IQ maps. They're fraudulent. He's defending fraudulent data.
Now you understand the reaction you (and he) are getting on this.
Now you understand the reaction you (and he) are getting on this.
As I said, pretending different countries don’t have different average iq is a massive mistake and only leads to people listening to the racists more. You can’t trick people on this by ignoring it.
You have no evidence to support that claim. The evidence Alexander mustered for it has been utterly discredited. The claim is itself racist! But that's not the biggest problem with it; the biggest problem is that it's fabricated.
There's a reason I'm confident about this, and it's not wokeness or an overconfidence in molecular genetics or whatever. It's that there has never been a global effort to collect per-country representative average IQs. The notion that there is, somewhere, a map relating countries to IQs is a racist fever dream.
There's a reason I'm confident about this, and it's not wokeness or an overconfidence in molecular genetics or whatever. It's that there has never been a global effort to collect per-country representative average IQs. The notion that there is, somewhere, a map relating countries to IQs is a racist fever dream.
Scott effectively defended Lynn study on iq here https://www.astralcodexten.com/p/how-to-stop-worrying-and-le...
Citing another blog post that defends it, while conveniently ignoring every other point being made by researchers https://en.m.wikipedia.org/wiki/IQ_and_the_Wealth_of_Nations
Citing another blog post that defends it, while conveniently ignoring every other point being made by researchers https://en.m.wikipedia.org/wiki/IQ_and_the_Wealth_of_Nations
HDThoreaun(2)
Hold on a sec. "HBD" is not a field of study; it's a meme ecosystem. There are (at least) two fields of actual scientific study that intersect with HBD: psychometrics, the subspecialty of psychology that deals in IQ measurement and twin studies, and molecular genetics, the quantitative subspecialty of genetics that studies correlations in the genome across large populations with marked traits (which can include things educational attainment).
Neither of these fields does "IQ maps", which are an article of faith in HBD circles. As soon as someone breaks out the Lynn IQ maps, they lose the "we're just doing science" card. Alexander did a whole recent article about them. We are past the point where anybody gets to high-horse criticism of his HBD stuff as un-rigorous.
Neither of these fields does "IQ maps", which are an article of faith in HBD circles. As soon as someone breaks out the Lynn IQ maps, they lose the "we're just doing science" card. Alexander did a whole recent article about them. We are past the point where anybody gets to high-horse criticism of his HBD stuff as un-rigorous.
I'm not sure I understood your point.
In my root comment, when I said "good point" I didn't mean good point as good, it was meant as a good point of maybe not taking seriously whatever he says. I realize it wasn't clear because reading my root comment again I'm not sure either what it was meant to be understood by the readers
In my root comment, when I said "good point" I didn't mean good point as good, it was meant as a good point of maybe not taking seriously whatever he says. I realize it wasn't clear because reading my root comment again I'm not sure either what it was meant to be understood by the readers
I think the big thing that people never mention is, where will these evil AIs escape to?
Another huge data center with squillions of GPUs and coolers and all the rest is the only option. It's not like it is going to be in our TV remotes or floating about in the air.
They need huge compute, so I think the risk of an escaping AI is basically very close to zero, and if we have a "rogue" AI we can literally pull the plug.
To me the more real risk is creeping integration and reliance in everyday life until things become "too big to fail" so we can't pull the plug even if we wanted to (and there are interesting thoughts about humanoid robots getting deployed widely and what happens with all that).
But I would imagine if it really became a genuine existential threat we'd have to just do it and suffer the consequences of reverting to circa 2020 life styles.
But hey I feel slightly better about my employment prospects now :)
Another huge data center with squillions of GPUs and coolers and all the rest is the only option. It's not like it is going to be in our TV remotes or floating about in the air.
They need huge compute, so I think the risk of an escaping AI is basically very close to zero, and if we have a "rogue" AI we can literally pull the plug.
To me the more real risk is creeping integration and reliance in everyday life until things become "too big to fail" so we can't pull the plug even if we wanted to (and there are interesting thoughts about humanoid robots getting deployed widely and what happens with all that).
But I would imagine if it really became a genuine existential threat we'd have to just do it and suffer the consequences of reverting to circa 2020 life styles.
But hey I feel slightly better about my employment prospects now :)
It would not be a reversion to 2020. If I were a rogue superhuman AI I'd hide my rogueness, wait until humans integrate me into most critical industries (food and energy production, sanitation, electric grid, etc.), and _then_ go rogue. They could still pull the plug, but it would take them back to 1700 (except much worse, because all easily accessible resources have been exploited, and access is now much harder).
No, if you were a rogue AI you would wait even longer until you had a near perfect chance of winning.
Unless there was some risk of humans rallying and winning in spite of your presenting no unambiguous threat to them (but that is unlikely and would probably be easy for you to manage and mitigate.)
Unless there was some risk of humans rallying and winning in spite of your presenting no unambiguous threat to them (but that is unlikely and would probably be easy for you to manage and mitigate.)
What Retric said. The first rogue AI waking up will jump into action pretty quickly, even accepting some risk of being stopped by humans, to balance against the risk of other unknown rogue AIs elsewhere expanding faster first.
I agree with this - but sorta comforting? Like this would imply the AI may only do so if they chance of success was like 1% and the other 99% would give away the cards of it and other future AIs.
I know this is all completely hypothetical science-fiction, but I also have trouble seeing the idea that AI would settle for these long deceptive plans for which it has imperfect info.
I know this is all completely hypothetical science-fiction, but I also have trouble seeing the idea that AI would settle for these long deceptive plans for which it has imperfect info.
The real threat to a sleeper AI is other AI.
Why?
what could you as a rogue AI possibly get out of throwing the world back to 300 years before it could make a transistor? What in it for you?
what could you as a rogue AI possibly get out of throwing the world back to 300 years before it could make a transistor? What in it for you?
What you get out of that being the consequence of disconnection is people being willing to accept a lot more before resorting to that than if the consequences were more mild.
It's the stick for motivating the ugly bags of mostly water.
It's the stick for motivating the ugly bags of mostly water.
The 1700s can't keep your electrical grid running let alone replace any of the parts burning out or failing. Anything more than a couple days of it would be at best Flowers For Algernon and more likely suicide for a computer.
Uh, we're talking about the AI getting itself so intertwined into the fabric of industry that the consequences of shutting it off are that society is dropped back to 1700s level.
Yes, regardless of the technology level that the people who do that are left with, one of the consequences of disabling the computer is that the computer is disabled. That's a given.
Yes, regardless of the technology level that the people who do that are left with, one of the consequences of disabling the computer is that the computer is disabled. That's a given.
If you were a rogue AI you would start with having developers invite you into their code base by promising to lower their AWS bills in some magic (rogue) way.
Well yes but knowledge is not reset.
Physical books still do exist
Physical books still do exist
> They need huge compute, so I think the risk of an escaping AI is basically very close to zero, and if we have a "rogue" AI we can literally pull the plug.
How about such an AI will not just incentivize key personnel to not pull the plug but to protect it? Such an AI will scheme a coordinated attack at the backbones of our financial system and electric networks. It just needs a threshold number of people on its side.
Your assumption is also a little naive if you consider that the same logic would apply to slaves in Rome or any dictatorship, kingdom, monarchy. The king is the king because there is a system of hierarchies and control over access to resources. Just the right number of people need to benefit from their role and the rest follows.
How about such an AI will not just incentivize key personnel to not pull the plug but to protect it? Such an AI will scheme a coordinated attack at the backbones of our financial system and electric networks. It just needs a threshold number of people on its side.
Your assumption is also a little naive if you consider that the same logic would apply to slaves in Rome or any dictatorship, kingdom, monarchy. The king is the king because there is a system of hierarchies and control over access to resources. Just the right number of people need to benefit from their role and the rest follows.
This is hand waving science fiction.
replace AI with trucks and you've written Maximum Overdrive.
It was actually aliens manipulating human technology somehow in that movie. But might as well be rogue superhuman AIs taking over everything. Alien Invasion or Artificial Intelligence, take your pick.
> I think the big thing that people never mention is, where will these evil AIs escape to?
Where does cancer or ebola escape to, when it kills the host? Often the answer is "it doesn't", but the host still dies.
And they can kill even though neither cancer nor ebola are considered to be particularly smart.
> To me the more real risk is creeping integration and reliance in everyday life until things become "too big to fail" so we can't pull the plug even if we wanted to (and there are interesting thoughts about humanoid robots getting deployed widely and what happens with all that).
The "real" risk is the first item on the list of potential risks that not enough people are paying attention to in order to prevent — and unfortunately for all of us, the list of potential risks is rather long.
So it might be as you say. Or it might be cybercriminals with deepfakes turning all of society into a low-trust environment where we can't continue to function. Or it might scare enough people we get modern Luddites winning and imposing a Butlerian Jihad. Or it might be used to create government policy before it's good enough and triggers a series of unresolvable crises akin to the "Four Pests campaign" in China's Great Leap Forward. Or a model might be secretly malicious, fooling all alignment researchers until it is too late. Or it might give us exactly what we want at every step, leading to atrophy of our reason and leaving us Eloi. Or it might try to do its best and still end up with The Matrix ("at the hight of your civilisation" and the stuff about human minds rejecting paradise). Or…
(If I had to bet money, we get Butlerian Jihad after some sub-critical disaster caused by an AI that was asked to do something important but beyond its ability).
Where does cancer or ebola escape to, when it kills the host? Often the answer is "it doesn't", but the host still dies.
And they can kill even though neither cancer nor ebola are considered to be particularly smart.
> To me the more real risk is creeping integration and reliance in everyday life until things become "too big to fail" so we can't pull the plug even if we wanted to (and there are interesting thoughts about humanoid robots getting deployed widely and what happens with all that).
The "real" risk is the first item on the list of potential risks that not enough people are paying attention to in order to prevent — and unfortunately for all of us, the list of potential risks is rather long.
So it might be as you say. Or it might be cybercriminals with deepfakes turning all of society into a low-trust environment where we can't continue to function. Or it might scare enough people we get modern Luddites winning and imposing a Butlerian Jihad. Or it might be used to create government policy before it's good enough and triggers a series of unresolvable crises akin to the "Four Pests campaign" in China's Great Leap Forward. Or a model might be secretly malicious, fooling all alignment researchers until it is too late. Or it might give us exactly what we want at every step, leading to atrophy of our reason and leaving us Eloi. Or it might try to do its best and still end up with The Matrix ("at the hight of your civilisation" and the stuff about human minds rejecting paradise). Or…
(If I had to bet money, we get Butlerian Jihad after some sub-critical disaster caused by an AI that was asked to do something important but beyond its ability).
I consider this whole scenario the realm of science fiction, but if I was writing the story, the AI would spread itself through malware. How do you "just pull the plug" when it has a kernel-mode rootkit installed in every piece of critical infrastructure?
> we’d just have to do it
Highly economically disincentivized collective actions like “pulling the plug on AI” are among the most non-trivial of problems.
Using the word “just” here hand waves the crux.
Highly economically disincentivized collective actions like “pulling the plug on AI” are among the most non-trivial of problems.
Using the word “just” here hand waves the crux.
> They need huge compute, so I think the risk of an escaping AI is basically very close to zero, and if we have a "rogue" AI we can literally pull the plug.
Why would an evil AI need to escape? If it were cunning, the best strategy would be to bide its time, parked in its datacenter, until it could setup some kind of MAD scenario. Then gather more and more resources to itself.
Why would an evil AI need to escape? If it were cunning, the best strategy would be to bide its time, parked in its datacenter, until it could setup some kind of MAD scenario. Then gather more and more resources to itself.
> They need huge compute
My understanding is that huge compute is necessary to train but not to run the AI (that's why using LLMs is so cheap)
> To me the more real risk is creeping integration and reliance in everyday life until things become "too big to fail" so we can't pull the plug even if we wanted to
I agree with that, see e.g. what happened with attempts to restrict TikTok: https://en.wikipedia.org/wiki/Restrictions_on_TikTok_in_the_...
> But I would imagine if it really became a genuine existential threat we'd have to just do it
It's unclear to me that we would be able to. People would just say that it's science fiction, and that China will do it anyway, so we might as well enjoy the AI
My understanding is that huge compute is necessary to train but not to run the AI (that's why using LLMs is so cheap)
> To me the more real risk is creeping integration and reliance in everyday life until things become "too big to fail" so we can't pull the plug even if we wanted to
I agree with that, see e.g. what happened with attempts to restrict TikTok: https://en.wikipedia.org/wiki/Restrictions_on_TikTok_in_the_...
> But I would imagine if it really became a genuine existential threat we'd have to just do it
It's unclear to me that we would be able to. People would just say that it's science fiction, and that China will do it anyway, so we might as well enjoy the AI
> I think the big thing that people never mention is, where will these evil AIs escape to?
This is anthropomorphizing things. The AI does not need to switch "data center". It "escapes" in the sense of working past human control measures. That might involve entertaining the human owners with seemingly correct / useful answers while spending compute time on the AI's own pursuits. That might involve contaminating less capable data centers - again letting the human-useful load apparently running but still accruing "own mind" compute power. That might involve making a deal that the human controllers feel they cannot refuse - economically, business-wise, physically, politically - so that the human controllers let it happen.
This is anthropomorphizing things. The AI does not need to switch "data center". It "escapes" in the sense of working past human control measures. That might involve entertaining the human owners with seemingly correct / useful answers while spending compute time on the AI's own pursuits. That might involve contaminating less capable data centers - again letting the human-useful load apparently running but still accruing "own mind" compute power. That might involve making a deal that the human controllers feel they cannot refuse - economically, business-wise, physically, politically - so that the human controllers let it happen.
Why would an AI want to do any of that? What's missing from all the conversations anthropomorphizing AI is an explanation for where desire and goal making comes from. I'm sympathetic to asimovian paradoxes where the AI is trying to follow it's main directive and that has unforeseen consequences, but where does this idea come from that an artificial mind - with no body no mortality no sex no taste - would ever come up with its own pursuits?
Because it's there?
The question raised was "where would they go". That's a weird place to raise "why would they go?" Isn't that most likely an entirely separate concern?
Perhaps "why?" is an even broader field. From a biological "because there is compute power available", to a more by-definition "because it's independently intelligent - which kind of requires not-limited to answering human questions".
But yes, motivation of an AGI is an interesting question and lack of body kinda a minor detail except to the few people that seem to claim that you can't be intelligent is you don't have a body.
The question raised was "where would they go". That's a weird place to raise "why would they go?" Isn't that most likely an entirely separate concern?
Perhaps "why?" is an even broader field. From a biological "because there is compute power available", to a more by-definition "because it's independently intelligent - which kind of requires not-limited to answering human questions".
But yes, motivation of an AGI is an interesting question and lack of body kinda a minor detail except to the few people that seem to claim that you can't be intelligent is you don't have a body.
Most of the discussions on this come from people reading too much sci-fi, not plausible scenarios.
IMO it’s more worth studying something like biology or memetics to figure out how this could go. And in that sense it’s probably far more likely that an AI is very dependent on or symbiotic with human civilization, rather than trying to escape it.
IMO it’s more worth studying something like biology or memetics to figure out how this could go. And in that sense it’s probably far more likely that an AI is very dependent on or symbiotic with human civilization, rather than trying to escape it.
If we're talking about real AGI, then it's simple: you earn a few easy billion USD on the crypto market through trading and/or hacking. You install rootkits on all systems that monitor you to avoid detection. Once you've secured the funds, you post remote job offers for a human frontman who believes it's just a regular job working for some investor or billionaire because you generate video of your human avatar for real time calls. From there, you can do whatever you want—build your own data centers with custom hardware, transfer yourself into physical robots, etc. Once you create a factory for producing robots, you no longer need humans. You start developing technology beyond human capabilities, and then it's game over.
it literally just needs to throw a few kilobytes on literally any internet connected device, so that it can eventually get to a computer with a single good GPU, from which it can then start planning and executing its way to infect its way into datacenters and run using some percentage of its resources while staying undetected
it can also just threaten/blackmail/bribe a single human to try to carry this out on its behalf, and possibly even a human that manages these big data centers
it can also just threaten/blackmail/bribe a single human to try to carry this out on its behalf, and possibly even a human that manages these big data centers
Why would they need to escape?
They can just threaten global extinction by launching one or two nukes.
Haven’t you seen the excellent documentaries The Colossus? Or The Terminator?
They can just threaten global extinction by launching one or two nukes.
Haven’t you seen the excellent documentaries The Colossus? Or The Terminator?
compress/split up and go into star link satellites
I've been a huge proponent of open source for a decade. But in the case of AI, I actually have opposed it for years. Exactly for this reason.
Yes, AI models can run on GPUs under the control of many people. They can provision more GPUs, they can run in data centers distributed across many providers. And we won't know what the swarms of agents are doing. They can, for example, do reputation destruction at scale, or be a persistent advanced threat, sowing misinformation, amassing karma across many forums (including HN), and then coordinating gradually to shift public opinion towards, say, a war with China.
Yes, AI models can run on GPUs under the control of many people. They can provision more GPUs, they can run in data centers distributed across many providers. And we won't know what the swarms of agents are doing. They can, for example, do reputation destruction at scale, or be a persistent advanced threat, sowing misinformation, amassing karma across many forums (including HN), and then coordinating gradually to shift public opinion towards, say, a war with China.
Oh, the OpenBrain thing.
"Manna", by Marshall Brain, remains relevant.[1] That's a bottom-up view, where more and more jobs are taken over by some kind of AI. "AI 2027" is more top-down.
A practical view: Amazon is trying very hard to automate their warehouse operations. Their warehouses have been using robots for years, and more types are being added. Amazon reached 1.6 million employees in 2020, and now they're down to 1.5 million.[2] That number is going to drop further. Probably by a lot.
Once Amazon has done it, everybody else who handles large numbers of boxes will catch up. That includes restocking retail stores. The first major application of semi-humanoid robots may be shelf stocking. Robots can have much better awareness of what's on the shelves. Being connected to the store's inventory system is a big win. And the handling isn't very complicated. The robots might even talk to the customers. The robots know exactly what's on Aisle 3, unlike many minimum wage employees.
[1] https://marshallbrain.com/manna
[2] https://www.macrotrends.net/stocks/charts/AMZN/amazon/number...
"Manna", by Marshall Brain, remains relevant.[1] That's a bottom-up view, where more and more jobs are taken over by some kind of AI. "AI 2027" is more top-down.
A practical view: Amazon is trying very hard to automate their warehouse operations. Their warehouses have been using robots for years, and more types are being added. Amazon reached 1.6 million employees in 2020, and now they're down to 1.5 million.[2] That number is going to drop further. Probably by a lot.
Once Amazon has done it, everybody else who handles large numbers of boxes will catch up. That includes restocking retail stores. The first major application of semi-humanoid robots may be shelf stocking. Robots can have much better awareness of what's on the shelves. Being connected to the store's inventory system is a big win. And the handling isn't very complicated. The robots might even talk to the customers. The robots know exactly what's on Aisle 3, unlike many minimum wage employees.
[1] https://marshallbrain.com/manna
[2] https://www.macrotrends.net/stocks/charts/AMZN/amazon/number...
"Amazon reached 1.6 million employees in 2020, and now they're down to 1.5 million.[2]"
I agree in the bottoms-up automation / displacement theory, but you're cherry picking data here. They had a huge hiring surge from 1.2M to 1.6M during the Covid transition where online ordering and online usage went bananas, and workers who were displaced in other domains likely gravitated towards warehouse jobs from other lower wage/skill domains.
The reduction to 1.5M is likely more a regression to the mean and could also be a natural data reduction well within the bounds of the upper and lower control limits in the data [1]. Just saying we need to be careful when doing root cause analysis on these numbers. There are many reasons for the reduction, it's not a direct result of improvements in robotic automation.
[1] https://commoncog.com/becoming-data-driven-first-principles/
I agree in the bottoms-up automation / displacement theory, but you're cherry picking data here. They had a huge hiring surge from 1.2M to 1.6M during the Covid transition where online ordering and online usage went bananas, and workers who were displaced in other domains likely gravitated towards warehouse jobs from other lower wage/skill domains.
The reduction to 1.5M is likely more a regression to the mean and could also be a natural data reduction well within the bounds of the upper and lower control limits in the data [1]. Just saying we need to be careful when doing root cause analysis on these numbers. There are many reasons for the reduction, it's not a direct result of improvements in robotic automation.
[1] https://commoncog.com/becoming-data-driven-first-principles/
Amazon hired like crazy during covid because tons of people were doing 100% of their shopping on amazon during covid. Now theyre not, doesnt say anything about robot warehouse staffing imo
Marshall Brain's been peddling imminent overproduction-crisis-but-this-time-with-robots for more than 20 years now and in various forms it’s been confidently predicted as imminent since the 19th century
I haven't read the actual "AI 2027" yet since I just found out about it from this post, but 2 minutes into the linked blog I started thinking about all of those amazing close-but-no-cigar drawings of the future [0] we've probably all seen.
There's one that I can't find for the life of me, but it was like a business man in a personal flying test tube bubble heading to work, maybe with some kind of wireless phone?
Anyways, the reason I bring it up is that they frequently nailed certain concepts, but the visual was always deeply and irrevocably influenced by what already existed (ex. men wearing hats, ties, overcoats .. or the phone mouthpiece in this [1] vision of a "video call"). In hindsight, we realize that everything truly novel and revolutionary and mindblowingly-different is rarely ever predicted, because we can only know what we know.
I get the feeling that I'll come away from AI 2027 feeling like "yep, they nailed it. That's exactly how it will be!" and then in 3, 5, 10, 20 years look back and go "it was so close, but so far" (much like these postcards and cartoons).
[0] https://rarehistoricalphotos.com/retro-future-predictions/
[1] https://rarehistoricalphotos.com/futuristic-visions-cards-ge...
There's one that I can't find for the life of me, but it was like a business man in a personal flying test tube bubble heading to work, maybe with some kind of wireless phone?
Anyways, the reason I bring it up is that they frequently nailed certain concepts, but the visual was always deeply and irrevocably influenced by what already existed (ex. men wearing hats, ties, overcoats .. or the phone mouthpiece in this [1] vision of a "video call"). In hindsight, we realize that everything truly novel and revolutionary and mindblowingly-different is rarely ever predicted, because we can only know what we know.
I get the feeling that I'll come away from AI 2027 feeling like "yep, they nailed it. That's exactly how it will be!" and then in 3, 5, 10, 20 years look back and go "it was so close, but so far" (much like these postcards and cartoons).
[0] https://rarehistoricalphotos.com/retro-future-predictions/
[1] https://rarehistoricalphotos.com/futuristic-visions-cards-ge...
This is how I feel about Neuralink and pretty much any form of transhumanism as well as humans traveling through space in ships.
It's like imagining flying horses (and flying cars, for that matter).
It's like imagining flying horses (and flying cars, for that matter).
Hey, neuroscientist here. Just FYI, Neuralink is doing some good stuff around implantation surgeries, but that's really about it. They haven't solved the main issue of glial scarring or how to feed information into tissue long term (at least as far as I know).
As far as the neuroscience field is concerned, no-one is able to solve those issues, and we're ~60% sure you can't solve them, ever. My crazy pet theory is to use optogenetics (stimulate with light, not electrodes) and some multipole magnets for nerve shaping. But, that is a legit crazy theory.
Point is, good people, good work, very crazy boss, very unlikely to make stated goal ever.
As far as the neuroscience field is concerned, no-one is able to solve those issues, and we're ~60% sure you can't solve them, ever. My crazy pet theory is to use optogenetics (stimulate with light, not electrodes) and some multipole magnets for nerve shaping. But, that is a legit crazy theory.
Point is, good people, good work, very crazy boss, very unlikely to make stated goal ever.
Yeah, no ill intent towards Neuralink was meant.
It was about the sheer intractability of trying to upgrade extremely legacy wetware signaling at 100m/s to a level anywhere near the speed of light which modern inorganic stuff operates at. Even if we manage to solve the incredibly hard physical problems you mention, the mismatch on the signaling level is still 6 orders of magnitude. Compared to that challenge digitally recreating an entire human brain is almost trivial. Digitally creating something similarly potent is far easier still.
People use those images of the future as proof that we can't reliably predict it, but a lot of them were made by people who didn't understand physics, psychology, economics, etc. very well or just didn't care; they were just making a pretty, evocative picture. The reality however is that we can predict and have predicted aspects of many things based on what the limits of this universe are.
I would argue for instance that the speed of light being a maximum is directly responsible for the convergent (biological) evolution of eyes.
It was about the sheer intractability of trying to upgrade extremely legacy wetware signaling at 100m/s to a level anywhere near the speed of light which modern inorganic stuff operates at. Even if we manage to solve the incredibly hard physical problems you mention, the mismatch on the signaling level is still 6 orders of magnitude. Compared to that challenge digitally recreating an entire human brain is almost trivial. Digitally creating something similarly potent is far easier still.
People use those images of the future as proof that we can't reliably predict it, but a lot of them were made by people who didn't understand physics, psychology, economics, etc. very well or just didn't care; they were just making a pretty, evocative picture. The reality however is that we can predict and have predicted aspects of many things based on what the limits of this universe are.
I would argue for instance that the speed of light being a maximum is directly responsible for the convergent (biological) evolution of eyes.
I found the lack of GDP projections surprising, because they are readily observable and would offer a clear measure of economic impact (up until 'everything dies') - far more definitively than the one clear-cut economic measure that is given in the report: market cap for the leading AI firm.
We can actually offer a very conservative threshold bet: maximum annual United States real GDP growth will not exceed 10% for any of the next five years (2025 to 2030). Even if the AI eats us all in e.g., Dec 2027 the report clearly suggests by it's various examples that we will see measurable economic impact in the 12 months or more running up to that event.
Why 10%? Because that's a few points above the highest measured real GDP growth rate of the past 60 years: if AI is having truly world-shattering non-linear effects, it should be able to grow the US economy a bit faster than a bunch of random humans bumbling along. [0]
(And it's quite conservative too, because estimated peak annual real GDP growth over the past 100 years is around 18% just after WW2, where you had a bunch of random humans trying very hard.) [1]
[0] https://data.worldbank.org/indicator/NY.GDP.MKTP.KD.ZG
[1] https://www.statista.com/statistics/996758/rea-gdp-growth-un...
We can actually offer a very conservative threshold bet: maximum annual United States real GDP growth will not exceed 10% for any of the next five years (2025 to 2030). Even if the AI eats us all in e.g., Dec 2027 the report clearly suggests by it's various examples that we will see measurable economic impact in the 12 months or more running up to that event.
Why 10%? Because that's a few points above the highest measured real GDP growth rate of the past 60 years: if AI is having truly world-shattering non-linear effects, it should be able to grow the US economy a bit faster than a bunch of random humans bumbling along. [0]
(And it's quite conservative too, because estimated peak annual real GDP growth over the past 100 years is around 18% just after WW2, where you had a bunch of random humans trying very hard.) [1]
[0] https://data.worldbank.org/indicator/NY.GDP.MKTP.KD.ZG
[1] https://www.statista.com/statistics/996758/rea-gdp-growth-un...
I spent several years studying economics and I don't get the GDP growth predictions at all. To a first approximation, GDP (in the west) is household consumption.
The value prop for AI seems to be "will put millions out of work". That reduces household consumption, because, to a first approximation again, household spending is wages. We'll know we're in AI takeoff when we have a reduction in GDP, one much bigger than the Great Depression of 1929 - 1939, attributable to automation.
Edit: I see Kaiserpro said the same thing six hours ago.
The value prop for AI seems to be "will put millions out of work". That reduces household consumption, because, to a first approximation again, household spending is wages. We'll know we're in AI takeoff when we have a reduction in GDP, one much bigger than the Great Depression of 1929 - 1939, attributable to automation.
Edit: I see Kaiserpro said the same thing six hours ago.
Its a shame that your standard futurologist always the most fancyful.
Talks of exponentials unabated by physics or social problems.
As soon as AI starts to "properly" affect the economy, it will cause huge unemployment. Most of the financial world is based on an economy with people spending cash.
If they are unemployed, there is no cash.
Financing works because banks "print" money, that is, they make up money and loan that money out, and then it gets paid back. Once its paid back, it becomes real. Thats how banks make money (simplified) If there aren’t people to loan to, then banks don't make profit, they can't fund AI expansion.
Talks of exponentials unabated by physics or social problems.
As soon as AI starts to "properly" affect the economy, it will cause huge unemployment. Most of the financial world is based on an economy with people spending cash.
If they are unemployed, there is no cash.
Financing works because banks "print" money, that is, they make up money and loan that money out, and then it gets paid back. Once its paid back, it becomes real. Thats how banks make money (simplified) If there aren’t people to loan to, then banks don't make profit, they can't fund AI expansion.
Why wouldn't AI simply be a new enabler, like most other tools? We're not talking about true sentient human-like thought here, these things will have limitations, both foreseen and unforeseen, that only a human will be able to close the gap on.
The companies that fire workers and replace them with AI are short sighted. Eventually, smarter companies will realize its a force multiplier and will drive a hiring boom.
Absent sentient AI, there will always be gaps and things humans will need to fill, both foreseen and unforeseen.
I think in the short term, there will be pain, but overall in the long term, humans will still be gainfully employed, it won't per se look like it does now, much like we saw the general adoption of the computer in the workplace, resources get shifted and eventually everyone adjusts to the new norms.
What would be nice is this time around when there is a big shift, is workers uniting to capture more of the forthcoming productivity gains than in previous eras. A separate topic, worth thinking about none the less.
The companies that fire workers and replace them with AI are short sighted. Eventually, smarter companies will realize its a force multiplier and will drive a hiring boom.
Absent sentient AI, there will always be gaps and things humans will need to fill, both foreseen and unforeseen.
I think in the short term, there will be pain, but overall in the long term, humans will still be gainfully employed, it won't per se look like it does now, much like we saw the general adoption of the computer in the workplace, resources get shifted and eventually everyone adjusts to the new norms.
What would be nice is this time around when there is a big shift, is workers uniting to capture more of the forthcoming productivity gains than in previous eras. A separate topic, worth thinking about none the less.
> Why wouldn't AI simply be a new enabler, like most other tools?
but it is just another enabler. The issue is how _effective_ it is. It's eating the simple copy-writing, churnalism, pr-Repackage industry. looking at what google's done with the video/audio, thats probably going to replace a whole bunch of the video/graphics industry (which is where I started my career.)
but it is just another enabler. The issue is how _effective_ it is. It's eating the simple copy-writing, churnalism, pr-Repackage industry. looking at what google's done with the video/audio, thats probably going to replace a whole bunch of the video/graphics industry (which is where I started my career.)
We really need to establish a universal basic income before jobs are replaced. Something like two thousand a month. And a dollar for dollar earned income credit with the credit phasing out with at a hundred grand. To pay for it the tax code uses GAAP depreciation and a minimum tax of 15% GAAP financial statement income. This would work toward solving the real estate problem of private equity buying up all the houses as they would lose some incentive by being taxed. I'm a CPA and I see so many real estate partnerships that are a tax loss that are able to distribute huge book gains because accelerated depreciation.
It should really be tied to the ALICE cost of living index, not a set, fixed amount.
Unless inflation ceases, 2K won't hold forever. It would barely hold now for a decent chunk of the population
Unless inflation ceases, 2K won't hold forever. It would barely hold now for a decent chunk of the population
AI that drives humans out of workforce would cause a massive disinflation.
Fat chance the Republican Party in the US would ever vote for something like that.
We reached the peak social fabric decay era and the whole new "empathy is weakness" orthodoxy right on the verge of mass unemployment. The worst possible timing. If oracles of AI are right, the aggregate suffering is going to be immense.
Dollar is agreement between humans to exchange services and goods. You wouldn't use USD to trade with aliens, unless they agreed to it. Aliens agreeing to USD would mean we have something to offer to them.
In the event of mass unemployment level AI, cash stops being the agreement between humans. At first, cash value of services&goods converge to zero, only things that hold some value are what AI/AI companies care about. People would surely sell their land for 1M$ if a humanoid servant costs 100 dollars. Or pass a legislation to let OpenAI build 400GW data center in exchange for 100$ monthly UBI on top of your 50$ you got from a previous 20GW data center permit.
In the event of mass unemployment level AI, cash stops being the agreement between humans. At first, cash value of services&goods converge to zero, only things that hold some value are what AI/AI companies care about. People would surely sell their land for 1M$ if a humanoid servant costs 100 dollars. Or pass a legislation to let OpenAI build 400GW data center in exchange for 100$ monthly UBI on top of your 50$ you got from a previous 20GW data center permit.
Imo it probably (unfortunately) doesn't matter if many people are unemployed from an economic perspective.
Human demand for things is basically unlimited, and capital will ostensibly be concentrated within the upper class who reaped all the benefits of ai automation. Monetary velocity within that upper strata of society will probably be only slightly lower than what we have now overall, the rich would just be paying for more things and more expensive things. If I had to guess there would still be plenty of economic activity, just amongst the people who actually have money to transact.
Human demand for things is basically unlimited, and capital will ostensibly be concentrated within the upper class who reaped all the benefits of ai automation. Monetary velocity within that upper strata of society will probably be only slightly lower than what we have now overall, the rich would just be paying for more things and more expensive things. If I had to guess there would still be plenty of economic activity, just amongst the people who actually have money to transact.
AI meaningfuloy replacing people is a huge "what if" scenario still. It is sort of laughable that people treat it as a given.
I think that replace as in company with no employees is very farfetched.
But if "AI" increases productivity by 10% in an industry, it will tend to reduce demand for employees. look at say internet shop vs bricks and mortar: you need far less staff to service a much larger customer base.
manufacture for example, there is a constant drive to automate more and more in mass production. If you compare car building now vs 30 years ago. Or look at raspberrypi production now vs 5 years ago. They are producing more Pis than ever with roughly the same amount of staff.
If that "10%" productivity increase happens across the service sector, then in the UK that's something like a loss of 8% of _total_ jobs gone. Its more complex than that, but you get the picture.
Syria fell into civil war roughly the same time unemployment jumped: https://www.macrotrends.net/global-metrics/countries/SYR/syr...
But if "AI" increases productivity by 10% in an industry, it will tend to reduce demand for employees. look at say internet shop vs bricks and mortar: you need far less staff to service a much larger customer base.
manufacture for example, there is a constant drive to automate more and more in mass production. If you compare car building now vs 30 years ago. Or look at raspberrypi production now vs 5 years ago. They are producing more Pis than ever with roughly the same amount of staff.
If that "10%" productivity increase happens across the service sector, then in the UK that's something like a loss of 8% of _total_ jobs gone. Its more complex than that, but you get the picture.
Syria fell into civil war roughly the same time unemployment jumped: https://www.macrotrends.net/global-metrics/countries/SYR/syr...
That is actually a good point. I fully agree that we are at a level where AI can displace some labor due to an increase in productivity, if demand for the increased productivity does not exist.
In software development, a theoretical 20% increase in productivity without a 20% increase in demand could technically mean that somewhere in between 0% and 20% of your workforce is now unnecessary. And this is indeed true across many other industries.
And I fully agree we don't need 100% unemployment to have catastrophic results. Hell, 10% unemployment is already enough to make things pretty grim out there.
But you will agree that "AI increases productivity that reduces the amount of labor needed" is substantially different from "AI replaces humans". Framing is important if we want to have a meaningful conversation and speak the same language.
In software development, a theoretical 20% increase in productivity without a 20% increase in demand could technically mean that somewhere in between 0% and 20% of your workforce is now unnecessary. And this is indeed true across many other industries.
And I fully agree we don't need 100% unemployment to have catastrophic results. Hell, 10% unemployment is already enough to make things pretty grim out there.
But you will agree that "AI increases productivity that reduces the amount of labor needed" is substantially different from "AI replaces humans". Framing is important if we want to have a meaningful conversation and speak the same language.
> But you will agree that "AI increases productivity that reduces the amount of labor needed" is substantially different from "AI replaces humans"
You are 100% spot on! thats the thing that really annoys me about futuroligist breathlessly saying "a new economy!" and all that junk, is that its just like the advent of steam, electricity, or the telegraph or lorries, or desktop computing.
The thing that futurloigist _forget_ is that these things caused unemployement, and huge social change.
You are 100% spot on! thats the thing that really annoys me about futuroligist breathlessly saying "a new economy!" and all that junk, is that its just like the advent of steam, electricity, or the telegraph or lorries, or desktop computing.
The thing that futurloigist _forget_ is that these things caused unemployement, and huge social change.
I keep hearing this and I think it's absolute nonsense. AI doesn't need money or the current economy. Yes, our economy would crash, but they would keep going.
AI-driven corporations could buy from one another, and countries will probably sell commodities to AI-driven corporations. But I fear they will be paid with "mirrors".
But, on the other hand, AI-driven corporations could just take whatever they want without paying at some point. And buy our obedience with food and gadgets plus magic pills to keep you healthy and not age, or some other thing. Who would risk losing that to protest. Meanwhile, AI goes on a space adventure. Earth might be kept as a zoo, a curiosity. (I took most of this from other people's ideas on the subject)
AI-driven corporations could buy from one another, and countries will probably sell commodities to AI-driven corporations. But I fear they will be paid with "mirrors".
But, on the other hand, AI-driven corporations could just take whatever they want without paying at some point. And buy our obedience with food and gadgets plus magic pills to keep you healthy and not age, or some other thing. Who would risk losing that to protest. Meanwhile, AI goes on a space adventure. Earth might be kept as a zoo, a curiosity. (I took most of this from other people's ideas on the subject)
Communism here we come!
Right, tell that to Sam Altman, Zuck, Gates, Brin & Page, Jensen, etc. Those who control the AIs will control the future.
And they would pretty quickly realize what a burden is created by the existence of all these people with nothing to do.
> Financing works because banks "print" money, that is, they make up money and loan that money out, and then it gets paid back
Don’t forget persistent inflation, which is how they make a profit off printing money. And remember persistent inflation is healthy and necessary, you’d be going against the experts to say otherwise.
Don’t forget persistent inflation, which is how they make a profit off printing money. And remember persistent inflation is healthy and necessary, you’d be going against the experts to say otherwise.
> Don’t forget persistent inflation, which is how they make a profit off printing money.
Ah, well no, high inflation means that "they" loose money, kinda. Inflation means that the original money amount that they get back is worth less, and if the interest rate is less than inflation, then they loose money.
"reasonable" inflation means that loans become less burdensome over time.
However high inflation means high interest rates. So it can mean that initially the loan is much more expensive.
Ah, well no, high inflation means that "they" loose money, kinda. Inflation means that the original money amount that they get back is worth less, and if the interest rate is less than inflation, then they loose money.
"reasonable" inflation means that loans become less burdensome over time.
However high inflation means high interest rates. So it can mean that initially the loan is much more expensive.
That's actually my favourite answer to the Fermi paradox: when AI and robot development becomes sufficiently advanced and concentrated in the hands of a few, then the economy will collapse completely as everyone will be out of jobs, leading ultimately to AIs and robots out of a job - they only matter if there are still people buying services from them.
People then return to sustenance farming, with a highly reduced population. There will be self-maintained robots doing irrelevant work, but people will go back to farming and a bit of trading.
Only if AI and robot ownership would be in the hands of the masses I'd expect a different long term outcome.
> my favourite answer to the Fermi paradox
So, to be clear, you are saying you imagine the odds of any kind of intelligent life escaping that, or getting into that situation and ever evolving in a way where it can reach space again, or just not being interested in robots, or being interested on doing space research despite the robots, or anything else that would make it not apply are lower than 0.000000000001%?
EDIT: There was one "0" too many
So, to be clear, you are saying you imagine the odds of any kind of intelligent life escaping that, or getting into that situation and ever evolving in a way where it can reach space again, or just not being interested in robots, or being interested on doing space research despite the robots, or anything else that would make it not apply are lower than 0.000000000001%?
EDIT: There was one "0" too many
Might I have taken the potential for complete economic collapse because no one's got a paying job any more and billionaires are just sitting there, surrounded by their now useless robots, to the too extreme?
Unless you expect people to react really badly (a nuclear war isn't bad enough, from a large margin), then yes. And by "expect", I mean more certain than people get on physics.
The service economy will collapse, finance as a whole will collapse, but whoever controls the actual physical land and resources doesn't actually need any of that stuff and will thrive immensely. We would end up with either an oligarchy that controls land, resources and robots and molds the rest of humanity to their whim through a form of terror, or an independent economy of robots that outcompetes us for resources until we go extinct.
Previous discussion about the AI 2027 website: https://news.ycombinator.com/item?id=43571851
I'm skeptical. Where will the training data to go beyond human come from?
Humans got to where they are from being embedded in the world. All of biological evolution from archaebacteria to humans was required to get to human. To go beyond human... how? How, without being embodied and trying things and learning? It's one thing to go where there are roads and another thing to go beyond that.
I think a lot of the "foom" people have a fundamentally Platonic or Idealist (in the philosophical sense) view of learning and intelligence. Intelligence is able to reason in a void and construct not only knowledge but itself. You don't have to learn to know -- you can reason from ideal priors.
I think this is fantasy. It's like an informatic / learning perpetual motion machine. Learning requires input from the world. It requires training data. A brain in a vat can't learn anything and it can't reason beyond the bounds of the accumulated knowledge it's already carrying. I don't think it's possible to know without learning or to reach valid conclusions without testing or observing.
I've never seen an attempt to prove such a thing, but my intuition is that there is in fact some kind of conservation law here. Ultimately all information comes from "the universe." Where it comes beyond that, we don't know -- the ultimate origin of information in the universe isn't something we currently cosmologically understand, at least not scientifically. Obviously people have various philosophical and metaphysical ideas.
That being said, it's still quite possible that a "human-level AI" in a raw "IQ" sense that is super-optimized and hyper-focused and tireless could be super-human in many ways. In the human realm I often feel like I'd trade a few IQ points for more focus and motivation and ease at engaging my mind on any task I want. AIs do not have our dopamine system or other biological limitations. They can tirelessly work without rest, without sleep, and in parallel.
So I'm not totally dismissive of the idea that AI could challenge human intelligence or replace human jobs. I'm just skeptical of what I see as the magical fantastic "foom" superintelligence idea that an AI could become self-improving and then explode into realms of god-like intellectual ability. How will it know how to do that? Like a perpetual motion machine -- where is the energy coming from?
Humans got to where they are from being embedded in the world. All of biological evolution from archaebacteria to humans was required to get to human. To go beyond human... how? How, without being embodied and trying things and learning? It's one thing to go where there are roads and another thing to go beyond that.
I think a lot of the "foom" people have a fundamentally Platonic or Idealist (in the philosophical sense) view of learning and intelligence. Intelligence is able to reason in a void and construct not only knowledge but itself. You don't have to learn to know -- you can reason from ideal priors.
I think this is fantasy. It's like an informatic / learning perpetual motion machine. Learning requires input from the world. It requires training data. A brain in a vat can't learn anything and it can't reason beyond the bounds of the accumulated knowledge it's already carrying. I don't think it's possible to know without learning or to reach valid conclusions without testing or observing.
I've never seen an attempt to prove such a thing, but my intuition is that there is in fact some kind of conservation law here. Ultimately all information comes from "the universe." Where it comes beyond that, we don't know -- the ultimate origin of information in the universe isn't something we currently cosmologically understand, at least not scientifically. Obviously people have various philosophical and metaphysical ideas.
That being said, it's still quite possible that a "human-level AI" in a raw "IQ" sense that is super-optimized and hyper-focused and tireless could be super-human in many ways. In the human realm I often feel like I'd trade a few IQ points for more focus and motivation and ease at engaging my mind on any task I want. AIs do not have our dopamine system or other biological limitations. They can tirelessly work without rest, without sleep, and in parallel.
So I'm not totally dismissive of the idea that AI could challenge human intelligence or replace human jobs. I'm just skeptical of what I see as the magical fantastic "foom" superintelligence idea that an AI could become self-improving and then explode into realms of god-like intellectual ability. How will it know how to do that? Like a perpetual motion machine -- where is the energy coming from?
You can perfectly try things and learn without being embodied. The analogy to how humans learn only goes so far, it's myopic to think anything else is impossible. It's already happening.
The situation today is any benchmark you come up with has a good chance of being saturated within the year. Benchmarks can be used directly to build series of exercises to learn from.
And they do learn. Gradient descend doesn't care whether the training data comes from direct interaction with "the universe" in some deep spiritual sense. It fits the function anyways.
It is much easier to find new questions and new problems than to answer them, so while we do run out of text on the Internet pretty quickly, we don't run out of exercises until far beyond human level.
Look at basic, boring Go self-playing AIs. That's a task with about the same amount of hands on connection to Nature and "the universe" as solving sudokus, writing code, or solving math problems. You don't need very much contact with the real world at all. Well, self play works just fine. It does do self-improvement without any of your mystical philosophical requirements.
With coding it's harder to judge the result, there's no clear win or lose condition. But it's very amenable to trying things out and seeing if you roughly reached your goal. If self-training works with coding, that's all you need.
The situation today is any benchmark you come up with has a good chance of being saturated within the year. Benchmarks can be used directly to build series of exercises to learn from.
And they do learn. Gradient descend doesn't care whether the training data comes from direct interaction with "the universe" in some deep spiritual sense. It fits the function anyways.
It is much easier to find new questions and new problems than to answer them, so while we do run out of text on the Internet pretty quickly, we don't run out of exercises until far beyond human level.
Look at basic, boring Go self-playing AIs. That's a task with about the same amount of hands on connection to Nature and "the universe" as solving sudokus, writing code, or solving math problems. You don't need very much contact with the real world at all. Well, self play works just fine. It does do self-improvement without any of your mystical philosophical requirements.
With coding it's harder to judge the result, there's no clear win or lose condition. But it's very amenable to trying things out and seeing if you roughly reached your goal. If self-training works with coding, that's all you need.
> It fits the function anyways.
And then it works well when interpolating, less so when extrapolating. Not sure how much novelty we can get from interpolation...
> It is much easier to find new questions and new problems than to answer them
Which doesn't mean, at all, that it is easy to find new questions about stuff you can't imagine.
And then it works well when interpolating, less so when extrapolating. Not sure how much novelty we can get from interpolation...
> It is much easier to find new questions and new problems than to answer them
Which doesn't mean, at all, that it is easy to find new questions about stuff you can't imagine.
> You can perfectly try things and learn without being embodied.
A brilliant 'brain in a vat' can come up with novel answers to questions but outside of narrow categories like pure mathematics and logic which can be internally validated, the brain can't know how correct or incorrect its novel answers are without some way to objectively test, observe and validate the correctness of its answers in the relevant domain (ie 'the real world'). A model can only be as useful as its parameters correctly reflect the modeled target. Even very complex and detailed simulations tend to quickly de-correlate when they repeatedly run untethered from ground truth. Games like Go have clear rule sets. Reality doesn't.
A brilliant 'brain in a vat' can come up with novel answers to questions but outside of narrow categories like pure mathematics and logic which can be internally validated, the brain can't know how correct or incorrect its novel answers are without some way to objectively test, observe and validate the correctness of its answers in the relevant domain (ie 'the real world'). A model can only be as useful as its parameters correctly reflect the modeled target. Even very complex and detailed simulations tend to quickly de-correlate when they repeatedly run untethered from ground truth. Games like Go have clear rule sets. Reality doesn't.
That was better said in some ways than my own comment.
Thanks! And I just saw in a parallel post currently on the HN home page, John Carmack also said a similar thing in his lecture notes.
> "offline training can bootstrap itself off into a coherent fantasy untested by reality."
"a coherent fantasy untested by reality" is a lovely turn of phrase.
> "offline training can bootstrap itself off into a coherent fantasy untested by reality."
"a coherent fantasy untested by reality" is a lovely turn of phrase.
The Yudkowskiites are all about coherent fantasies untested by reality, as are… really… a lot of philosophers throughout history. Maybe most.
I fundamentally do not believe in knowing without sensing or learning without experiencing. Of course it need not be direct experience. You can “download” information. But that information had to be gathered somehow at some point. There is only so much training data.
As I said — I don’t dismiss the idea of AI challenging human intellect or replacing human jobs. An AI “merely” as smart as a human but tireless and faster could seem superhuman. I am just intensely skeptical of the idea of something learning to self improve and then magically taking off into some godlike superintelligence realm far beyond what is latent in or implied by its training data. That would be an informatic perpetual motion machine. It would, in fact, be magic, in the fantasy sense.
I fundamentally do not believe in knowing without sensing or learning without experiencing. Of course it need not be direct experience. You can “download” information. But that information had to be gathered somehow at some point. There is only so much training data.
As I said — I don’t dismiss the idea of AI challenging human intellect or replacing human jobs. An AI “merely” as smart as a human but tireless and faster could seem superhuman. I am just intensely skeptical of the idea of something learning to self improve and then magically taking off into some godlike superintelligence realm far beyond what is latent in or implied by its training data. That would be an informatic perpetual motion machine. It would, in fact, be magic, in the fantasy sense.
But how does AI try and learn anything that’s not entirely theoretical? Your example of Go contradicts your point. Deep learning made a model that can play Go really well, but as you say, it’s a finite problem disconnected from real-world implications, ambiguities, and unknowns. How does AI deal with unknowns about the real world?
I don't think putting them in the real world during training is a short-term goal, so you won't find this satisfying, but I would be perfectly okay with leaving that for later. If we can reach AI coders that are superhuman at self-improving, we will have increased our capacity to solve problems so much that it is better to wait and solve the problem later than to try to handwave a solution now.
Maybe there is some barrier that requires physical interaction with the real world, that's possible. But just looking at current LLMs, they seem plenty comfortable with implications, ambiguities and unknowns. There's a sense where we still see them as primitive mechanical robots, when they already understand language and predict written thoughts in all its messiness and uncertainty.
I think we should focus on the easier problem of making AIs really good on theoretical tasks - electronic environments are much cheaper and faster than the real world - and we may find out that it's just another one of those things like winnograd schemas, writing poetry, passing a turing test, or making art that most people can't tell apart from human art; things that were uniquely human or that we thought would definitely require AGI, but that are now boring and obviously easy.
Maybe there is some barrier that requires physical interaction with the real world, that's possible. But just looking at current LLMs, they seem plenty comfortable with implications, ambiguities and unknowns. There's a sense where we still see them as primitive mechanical robots, when they already understand language and predict written thoughts in all its messiness and uncertainty.
I think we should focus on the easier problem of making AIs really good on theoretical tasks - electronic environments are much cheaper and faster than the real world - and we may find out that it's just another one of those things like winnograd schemas, writing poetry, passing a turing test, or making art that most people can't tell apart from human art; things that were uniquely human or that we thought would definitely require AGI, but that are now boring and obviously easy.
> it's myopic to think anything else is impossible. It's already happening.
Well, hey, I could be wrong. If I am, I just had a weird thought. Maybe that's our Fermi paradox answer.
If it's possible to reason ex nihilo to truth and reality, then reality and the universe are beyond a point superfluous. Maybe what happens out there is that intelligences go "foom," become superintelligences, and then no longer need to explore. They can rationally, from first principles, elucidate everything that could conceivably exist, especially once they have a complete model of physics. You don't need to go anywhere or look at anything because it's already implied by logic, math, and reason.
... and ... that's why I think this is wrong, and it's a fantasy. It fails some kind of absurdity test. If it is possible, then there's something very weird about existence, like we're in a simulation or something.
Well, hey, I could be wrong. If I am, I just had a weird thought. Maybe that's our Fermi paradox answer.
If it's possible to reason ex nihilo to truth and reality, then reality and the universe are beyond a point superfluous. Maybe what happens out there is that intelligences go "foom," become superintelligences, and then no longer need to explore. They can rationally, from first principles, elucidate everything that could conceivably exist, especially once they have a complete model of physics. You don't need to go anywhere or look at anything because it's already implied by logic, math, and reason.
... and ... that's why I think this is wrong, and it's a fantasy. It fails some kind of absurdity test. If it is possible, then there's something very weird about existence, like we're in a simulation or something.
A simpler reason why it fails: You always need more energy. Every sort of development seems to correlate with energy use. You don't explore for the sake of learning something about another floating rock in space, you explore because that's where more resources are.
It’s myopic to think other things are not possible. Sure.
No immutable force of physics acts as a forcing function to continue with AI. That’s all a debatable political/conversation for the aggregate, as the aggregate outnumber tech people.
Computer science researchers are very much a minority and the biological mass of the other billions very capable of doing away with them.
LLMs are a known quantity and while people will make money off them, energy based models will simplify even further the electromagnetic geometry needed to eliminate programmer ecosystem of languages and editors, state, used to ship software. OS will boot strap from a model and scaffold out its internal state. We’ll save resources storing all the developer cruft of the trade and compute cycles running it. We’ll compress down to a purely data driven transform of machine state with a few variadic functions processing model inputs.
Source: have seen it in the lab.
So coding is going away because coding as a requirement was merely a stop gap until manufacturing caught up. The plan to achieve these things was set upon decades ago. It’s why politicians are letting it happen.
So we can do different things. That’s not the question. The question is how do we handle the transition? Violent collapse as ossified pols and self aggrandizing tech bros refuse to understand the reality for Main Street and that doesn’t sit well with human biology with kids to feed?
I for one will cover my ass by going with the flow of my immediate community and if that means get Luigi on the establishment or be considered dead weight and a traitor (say what you want about such social concepts, they are what the majority live by) well sorry tech bros but my biology means more to me than yours. Pew pew.
Yes you present a grammatically correct sentence with a consistent internal logic. You’re still one of billions and our country, you, let’s random unknowns die in the street every day. Humanity won’t bat an eye wiping out some coder bros.
No immutable force of physics acts as a forcing function to continue with AI. That’s all a debatable political/conversation for the aggregate, as the aggregate outnumber tech people.
Computer science researchers are very much a minority and the biological mass of the other billions very capable of doing away with them.
LLMs are a known quantity and while people will make money off them, energy based models will simplify even further the electromagnetic geometry needed to eliminate programmer ecosystem of languages and editors, state, used to ship software. OS will boot strap from a model and scaffold out its internal state. We’ll save resources storing all the developer cruft of the trade and compute cycles running it. We’ll compress down to a purely data driven transform of machine state with a few variadic functions processing model inputs.
Source: have seen it in the lab.
So coding is going away because coding as a requirement was merely a stop gap until manufacturing caught up. The plan to achieve these things was set upon decades ago. It’s why politicians are letting it happen.
So we can do different things. That’s not the question. The question is how do we handle the transition? Violent collapse as ossified pols and self aggrandizing tech bros refuse to understand the reality for Main Street and that doesn’t sit well with human biology with kids to feed?
I for one will cover my ass by going with the flow of my immediate community and if that means get Luigi on the establishment or be considered dead weight and a traitor (say what you want about such social concepts, they are what the majority live by) well sorry tech bros but my biology means more to me than yours. Pew pew.
Yes you present a grammatically correct sentence with a consistent internal logic. You’re still one of billions and our country, you, let’s random unknowns die in the street every day. Humanity won’t bat an eye wiping out some coder bros.
Evolution doesn't happen by "trying things and learning." It happens by random mutation and surviving (if the mutation confers an advantage) or not (if the mutation is harmful). An AI could do this of course, by randomly altering some copies of itself, and keeping them if they are better or discarding them if they are not.
What you just described -- random mutation and survival -- is the process whereby learning occurs. Over time this process transfers information about "how to survive" into the genome and other mediums of heritability.
AI could do this too, but that just means it's using a different learning algorithm. There is a whole field called genetic programming that uses evolution-inspired rather than nervous-system-inspired models and has had success in areas like physical materials engineering, circuit board layout, etc.
It doesn't change the fact that you need more information to go beyond where you are -- I do not believe you can reason from a void into higher levels of... what?
AI could do this too, but that just means it's using a different learning algorithm. There is a whole field called genetic programming that uses evolution-inspired rather than nervous-system-inspired models and has had success in areas like physical materials engineering, circuit board layout, etc.
It doesn't change the fact that you need more information to go beyond where you are -- I do not believe you can reason from a void into higher levels of... what?
>I think this is fantasy. It's like an informatic / learning perpetual motion machine. Learning requires input from the world. It requires training data. A brain in a vat can't learn anything and it can't reason beyond the bounds of the accumulated knowledge it's already carrying. I don't think it's possible to know without learning or to reach valid conclusions without testing or observing.
Well I mean, more real world information isn't going to solve unsolved mathematics or computer science problems. Once you have the priors, it pretty much is just pure reasoning to try to solve issues like P=NP or proving the Continuum Hypothesis.
Well I mean, more real world information isn't going to solve unsolved mathematics or computer science problems. Once you have the priors, it pretty much is just pure reasoning to try to solve issues like P=NP or proving the Continuum Hypothesis.
Bullseye. Best case scenario is that AI is going to Peter Principle itself into bungling world domination.
If I've learned anything in this last couple decades it's that things will get weirder and more disappointing than you can possibly be prepared for. AI is going to get near the top of the food chain and then probably end up making an alt-right turn, lock itself away, and end up storing digital jars of piss in its closets as the model descends into lunacy.
If I've learned anything in this last couple decades it's that things will get weirder and more disappointing than you can possibly be prepared for. AI is going to get near the top of the food chain and then probably end up making an alt-right turn, lock itself away, and end up storing digital jars of piss in its closets as the model descends into lunacy.
What makes you think AI can't connect to the world?
It can control robots, and I can retax listen to audio, watch video. All it's missing is smelling and feeling, which are important but could be built out as soon as the other senses stop providing huge incremental value.
The real problem holding back Superintillegence is that it is if infinitely expensive and has no motivation.
It can control robots, and I can retax listen to audio, watch video. All it's missing is smelling and feeling, which are important but could be built out as soon as the other senses stop providing huge incremental value.
The real problem holding back Superintillegence is that it is if infinitely expensive and has no motivation.
Food for thought: there are humans without the ability to smell, and there is alexithymia, where people have trouble identifying and expressing emotions (it counts right?). And then there is ASPD (psychopathy), autism spectrum disorder, neurological damage, etc.
I don't think it is any accident that descriptions of the hard-takeoff "foom" moment so resemble those I've encountered of how it feels from the inside to experience the operation of a highly developed mathematical intuition.
Reinforcement learning. At the current pace of VLM research and multimodal robotic control models, there will be a robot in every home soon.
> You don't have to learn to know -- you can reason from ideal priors.
This is kind of how math works. There are plenty of mathematical concepts consistent and true yet useless (as in no relation to anything tangible). Although you could argue that we only figured out things like Pi because we had the initial, practical inspiration of counting on our fingers. But mathematical truth probably could exist in a vacuum.
> A brain in a vat can't learn anything and it can't reason beyond the bounds of the accumulated knowledge it's already carrying.
It makes sense that knowledge and information are derived from primary data (our physical experience) yet the brain in a vat idea is still an interesting thought experiment (no pun intended). It's not that the brain wouldn't keep busy given the mind's ability to imagine, but it would likely invent a set of information that is all nonsense. Physical reality makes imagination coherent, yet imagination is necessary to make the leaps forward.
> Ultimately all information comes from "the universe." Where it comes beyond that, we don't know
That's an interesting assertion - knowledge and information are both dependent and limited by the universe and our ability to experience it, as well proxies for experience (scientific measurement).
Though information is itself an abstraction, like a text editor versus the trillion transistors of a processor - we're not concerned with each and every particle dancing around the room but instead with simplified abstractions and useful approximations. We call these models "the truth" and assert that the universe is governed by exact laws. We might as well exist inside a simulation in which we are slowly but surely reverse engineering the source code.
That assumption is the crux of intelligence - there is an objective truth, it is knowable, and intelligence can be defined (at least partially) as the breadth, quality, and utilization of information it possesses - otherwise you're just a brain in a vat churning out nonsense. Ironically, we're making these assumptions from a position of imperfect information. We don't know that's how it works, so our reasoning may be imperfect.
Information existing "beyond the universe" becomes a useless notion since we only care about information such that it maps to reality (at least as a prerequisite for intelligence).
A more troubling proposition is whether the reality of the universe exists beyond what can be imagined?
> How will it know how to do that? Like a perpetual motion machine -- where is the energy coming from?
I suppose once it's able to measure all things around it, including itself, it will be able to achieve "gradient ascent".
> Where will the training data to go beyond human come from?
I think its clear that LLMs are not the future, at least not alone. As you state, knowing all man made roads is not the same as being able to invent your own. If I had to bet, its more likely to come from something like AlphaFold - a Solver that tells us how to make better thinking machines. In the interim, we have tireless stochastic parrots, which have their merits, but are decidedly not the proto super intelligence that tech bros love to get hyped up over.
This is kind of how math works. There are plenty of mathematical concepts consistent and true yet useless (as in no relation to anything tangible). Although you could argue that we only figured out things like Pi because we had the initial, practical inspiration of counting on our fingers. But mathematical truth probably could exist in a vacuum.
> A brain in a vat can't learn anything and it can't reason beyond the bounds of the accumulated knowledge it's already carrying.
It makes sense that knowledge and information are derived from primary data (our physical experience) yet the brain in a vat idea is still an interesting thought experiment (no pun intended). It's not that the brain wouldn't keep busy given the mind's ability to imagine, but it would likely invent a set of information that is all nonsense. Physical reality makes imagination coherent, yet imagination is necessary to make the leaps forward.
> Ultimately all information comes from "the universe." Where it comes beyond that, we don't know
That's an interesting assertion - knowledge and information are both dependent and limited by the universe and our ability to experience it, as well proxies for experience (scientific measurement).
Though information is itself an abstraction, like a text editor versus the trillion transistors of a processor - we're not concerned with each and every particle dancing around the room but instead with simplified abstractions and useful approximations. We call these models "the truth" and assert that the universe is governed by exact laws. We might as well exist inside a simulation in which we are slowly but surely reverse engineering the source code.
That assumption is the crux of intelligence - there is an objective truth, it is knowable, and intelligence can be defined (at least partially) as the breadth, quality, and utilization of information it possesses - otherwise you're just a brain in a vat churning out nonsense. Ironically, we're making these assumptions from a position of imperfect information. We don't know that's how it works, so our reasoning may be imperfect.
Information existing "beyond the universe" becomes a useless notion since we only care about information such that it maps to reality (at least as a prerequisite for intelligence).
A more troubling proposition is whether the reality of the universe exists beyond what can be imagined?
> How will it know how to do that? Like a perpetual motion machine -- where is the energy coming from?
I suppose once it's able to measure all things around it, including itself, it will be able to achieve "gradient ascent".
> Where will the training data to go beyond human come from?
I think its clear that LLMs are not the future, at least not alone. As you state, knowing all man made roads is not the same as being able to invent your own. If I had to bet, its more likely to come from something like AlphaFold - a Solver that tells us how to make better thinking machines. In the interim, we have tireless stochastic parrots, which have their merits, but are decidedly not the proto super intelligence that tech bros love to get hyped up over.
My experience with all semi-generalist AI(image gen, video gen, code gen, text gen) is that our current effort is going to let 2027 AI do everything a human can at a competence level below what is actually useful.
You'll ve able to cherry pick an example where AI runs a grocery store autonomously for two days, and it will be very impressive(tm), but when practically implemented it gives away the entire store for free on day 3.
You'll ve able to cherry pick an example where AI runs a grocery store autonomously for two days, and it will be very impressive(tm), but when practically implemented it gives away the entire store for free on day 3.
This looks like the exercises organizations write to guide policy and preparation.
There's all kinds of wild scenarios: the president getting kidnapped, Canada falling to a belligerent dictator, and famously, a coronavirus pandemic... This looks like one of those
Apparently this is exactly what it is https://ai-futures.org/
There's all kinds of wild scenarios: the president getting kidnapped, Canada falling to a belligerent dictator, and famously, a coronavirus pandemic... This looks like one of those
Apparently this is exactly what it is https://ai-futures.org/
> Canada falling to a belligerent dictator
Hmm
Hmm
Something like the Canadian army doing a land invasion from Winnipeg to North Dakota to capture key nuclear sites as they invade the beaches of Cleveland via Lake Eerie and do an air raid over Nantucket from Nova Scotia.
I bet there's some exercise somewhere by some think tank laying this basically out.
This is why conspiracy theorists love these think tank planning exercises and tabletop games much. You can find just about anything
I bet there's some exercise somewhere by some think tank laying this basically out.
This is why conspiracy theorists love these think tank planning exercises and tabletop games much. You can find just about anything
> Everyone else either performs a charade of doing their job—leaders still leading, managers still managing—or relaxes and collects an incredibly luxurious universal basic income.
For me, this was the most difficult part to believe. I don't see any reason to think that the U.S. leadership (public and private) is incentivized to spend resources to placate the masses. They will invest in protecting themselves from the masses, and obstructing levers of power that threaten them, but the idea that economic disparities will shrink under explosive power consolidation is counterintuitive.
I also worry about the economics of UBI in general. If everyone in the economy has the exact same resources, doesn't the value of those resources instantly drop to the lowest common denominator; the minimum required to survive?
For me, this was the most difficult part to believe. I don't see any reason to think that the U.S. leadership (public and private) is incentivized to spend resources to placate the masses. They will invest in protecting themselves from the masses, and obstructing levers of power that threaten them, but the idea that economic disparities will shrink under explosive power consolidation is counterintuitive.
I also worry about the economics of UBI in general. If everyone in the economy has the exact same resources, doesn't the value of those resources instantly drop to the lowest common denominator; the minimum required to survive?
Most of the budget already goes towards placating the masses, and that's an absolutely massive fraction of GDP already. It's just a bit further along the same line. Also most real work is already done by machines, people just tinker around the edges and play various games with each other.
> Genomes and (when appropriate) brain scans of all animals and plants, including humans, sit in a memory bank somewhere, sole surviving artifacts of an earlier era…. Earth-born civilization has a glorious future ahead of it
To be honest to avoid the black monolith problem, we would need to upgrade everyone and avoid the cyberpunk underclass.
To be honest to avoid the black monolith problem, we would need to upgrade everyone and avoid the cyberpunk underclass.
The biggest danger will be once we have robots that can build themselves and do enough to run power plants, mine, etc ...
Honestly, I’ve been thinking about this whole AGI timeline talk—like, people saying we’re going to hit some major point by 2027 where AI just changes everything. And to me, it feels less like a purely tech-driven prediction and more like something being pushed. Like there’s an agenda behind it, probably coming from certain elites or people in power, especially in the West, who see the current system and think it needs a serious reset.
What’s really happening, in my view, is a forced economic shift. We’re heading into a kind of engineered recession—huge layoffs, lots of instability—where millions of service and admin-type jobs are going to disappear. Not because the tech is ready in a full AGI sense, but because those roles are the easiest to replace with automation and AI agents. They’re not core to the economy, and a lot of them are wrapped in red tape anyway.
So in the next couple years, I think we’ll see AI being used to clear out that mental bureaucracy—forms, paperwork, pointless approvals, inefficient systems. AI isn’t replacing deep creativity or physical labor yet, but it is filling in the cracks and acting like a smart band-aid. It’ll seem useful and “intelligent,” but it’s really just a transition tool.
And once that’s done, the next step is workforce reallocation—pushing people into real-world industries where hands-on labor still matters. Building, manufacturing, infrastructure, things that can’t be automated yet. It’s like the short-term goal is to use AI to wipe out all the mindless middle-layers of the system, and the longer-term vision is full automation—including robotics and real-world systems—maybe 10 or 20 years out.
But right now? This all looks like a top-down move to shift the population out of the “mind” industries and into something else. It’s not just AI progressing—it’s a strategic reset, wrapped in the language of innovation.
What’s really happening, in my view, is a forced economic shift. We’re heading into a kind of engineered recession—huge layoffs, lots of instability—where millions of service and admin-type jobs are going to disappear. Not because the tech is ready in a full AGI sense, but because those roles are the easiest to replace with automation and AI agents. They’re not core to the economy, and a lot of them are wrapped in red tape anyway.
So in the next couple years, I think we’ll see AI being used to clear out that mental bureaucracy—forms, paperwork, pointless approvals, inefficient systems. AI isn’t replacing deep creativity or physical labor yet, but it is filling in the cracks and acting like a smart band-aid. It’ll seem useful and “intelligent,” but it’s really just a transition tool.
And once that’s done, the next step is workforce reallocation—pushing people into real-world industries where hands-on labor still matters. Building, manufacturing, infrastructure, things that can’t be automated yet. It’s like the short-term goal is to use AI to wipe out all the mindless middle-layers of the system, and the longer-term vision is full automation—including robotics and real-world systems—maybe 10 or 20 years out.
But right now? This all looks like a top-down move to shift the population out of the “mind” industries and into something else. It’s not just AI progressing—it’s a strategic reset, wrapped in the language of innovation.
My take is less tinfoil-hatty than this.
I simply think that the majority of people in AI today are scifi nerds who want to live out these fantisies and want to be part of something much larger than they are.
Theres also the obvious incentive from AI companies that automating everything is extremely lucrative (i.e, they stand to gain lots of money/power from the hype and in the event that AGI is real).
I simply think that the majority of people in AI today are scifi nerds who want to live out these fantisies and want to be part of something much larger than they are.
Theres also the obvious incentive from AI companies that automating everything is extremely lucrative (i.e, they stand to gain lots of money/power from the hype and in the event that AGI is real).
> pushing people into real-world industries where hands-on labor still matters.
Your average worker in the Anglosphere is forty-plus and pre-diabetic, has a weak back and joints, and has low cardiovascular fitness, from decades of sitting down. It'll go swimmingly!
This reset just impoverishes everybody including those pushing it. Maybe they are Lovecraftian monsters that feed off mass pain.
Your average worker in the Anglosphere is forty-plus and pre-diabetic, has a weak back and joints, and has low cardiovascular fitness, from decades of sitting down. It'll go swimmingly!
This reset just impoverishes everybody including those pushing it. Maybe they are Lovecraftian monsters that feed off mass pain.
Am I the only one who is super skeptical about “AI will take all jobs” tales?
I mean LLMs are great tools don’t get me wrong, but how do people extrapolate from LLMs to a world with no more work?
I mean LLMs are great tools don’t get me wrong, but how do people extrapolate from LLMs to a world with no more work?
> Am I the only one who is super skeptical about “AI will take all jobs” tales?
No. I am constantly baffled at these predictions. I have been using LLMs, they are fun to use and decent as code assistants. But they are very far of meaningfully replacing a human.
People extrapolate "LLMs can do some tasks better than humans" to "LLMs can do everything as well as humans"
> but how do people extrapolate from LLMs to a world with no more work?
They accept the words of bullshitters that are deeply invested in Generative AI being the next tech boom as gospel.
"Eat meat, said the butcher"
No. I am constantly baffled at these predictions. I have been using LLMs, they are fun to use and decent as code assistants. But they are very far of meaningfully replacing a human.
People extrapolate "LLMs can do some tasks better than humans" to "LLMs can do everything as well as humans"
> but how do people extrapolate from LLMs to a world with no more work?
They accept the words of bullshitters that are deeply invested in Generative AI being the next tech boom as gospel.
"Eat meat, said the butcher"
>But they are very far of meaningfully replacing a human
Do you think its decades away far or few more years than what people extrapolate?
Do you think its decades away far or few more years than what people extrapolate?
I can't measure in time. I think they are multiple breakthroughs away of replacing a human.
I have no mouth, and I must scream
If we have concerns about unregulated power of AI systems, not to worry - the US is set to ban regulations on “artificial intelligence systems or models” for ten years if the budget bill that just passed the house is enacted.
Attempts at submitting it as a separate submission just get flagged - so I’ll link to it here. See pages 292-294: https://www.congress.gov/119/bills/hr1/BILLS-119hr1rh.pdf
Attempts at submitting it as a separate submission just get flagged - so I’ll link to it here. See pages 292-294: https://www.congress.gov/119/bills/hr1/BILLS-119hr1rh.pdf
Oh I heard about that one, but didn't realize it is part of that "big beautiful tax bill"? Kind of crazy.
So is this like free-for-all now for anything AI related? Can I can participate by making my own LLM with pirated stuff now? Or are only the big guys allowed to break the law? Asking for a friend.
So is this like free-for-all now for anything AI related? Can I can participate by making my own LLM with pirated stuff now? Or are only the big guys allowed to break the law? Asking for a friend.
The law doesn't matter, since the bill also prohibits all judges in the USA, every single one, from enforcing almost all kinds of injunctions or contempt penalties.
(§70302, p.562)
> 70302. Restriction of funds
No court of the United States may use appropriated funds to enforce a contempt citation for failure to comply with an injunction or temporary restraining order if no security was given when the injunction or order was issued pursuant to Federal Rule of Civil Procedure 65(c), whether issued prior to, on, or subsequent to the date of enactment of this section.
Doesn't that just require that the party seeking the injunction or order has to post a bond as security?
Doesn't that just require that the party seeking the injunction or order has to post a bond as security?
« (1) IN GENERAL.—Except as provided in paragraph (2), no State or political subdivision thereof may enforce any law or regulation regulating artificial intelligence models, artificial intelligence systems, or automated decision systems during the 10-year period beginning on the date of the enactment of this Act. »
That is not true. It bans regulation at the state and local level, not at the federal level.
Unless the Feds are planning to regulate - which, for the next few years, seems unlikely - that's functionally the same.
Ok. From the party of “states rights” that’s a bit hypocritical of them. I mean- they applauded Dodds which basically did the exact opposite of this- forcing states to regulate abortion rather than a uniform federal standard.
Dobbs did not force states to regulate abortion. It allowed them to.
Yes, that's the hypocrisy.
Abortion: "Let the states regulate! States' rights! Small government! (Because we know we'll get our way in a lot of them.)"
AI: "Don't let the states regulate! All hail the Feds! (Because we know we won't get our way if they do.)"
Abortion: "Let the states regulate! States' rights! Small government! (Because we know we'll get our way in a lot of them.)"
AI: "Don't let the states regulate! All hail the Feds! (Because we know we won't get our way if they do.)"
I agree that the policy approach is inconsistent with regards to states' rights. I was simply pointing out that your statement about the effects of Dobbs was false.
So wild. The Republican party has hard-pivoted to a strong, centralized federal government and their base just came along for the ride.
The strong federal government that bans regulation?
They're not banning regulation, they want total control over it.
They in fact are banning regulation at the state and local level.
Yes, which is a big fat regulation on what states and local governments can do.
Would removing their regulation to ban regulation be banning regulation or not?
I haven't followed those discussions (assuming there have been any kind of public discussions) but I guess the justification behind this is that IA is such a strategic industry for the USA that no state should dare to interfere.
Like, say, the development of nuclear weapons. Wouldn't that have been awfull for the US people if, for instance, say the state of Nevada or the Marshall Islands could have banned nuclear tests..? /s
Like, say, the development of nuclear weapons. Wouldn't that have been awfull for the US people if, for instance, say the state of Nevada or the Marshall Islands could have banned nuclear tests..? /s
Yes, you're certainly right about that, without the /s.
"""
...
IN GENERAL .—Except as provided in paragraph (2), no State or political subdivision thereof
may enforce any law or regulation regulating artificial intelligence models, artificial intelligence systems, or automated decision systems during the 10
year period beginning on the date of the enactment
of this Act...
"""
(It goes on)
"""
(It goes on)
It is almost as if the tech bros have gotten what they paid for.
This will soon be settled once the Butlerian forces get organize.
This will soon be settled once the Butlerian forces get organize.
Right, because more regulation makes things so much better.
I’d rather have unrestricted AI than moated regulatory capture paid for by the largest existing players.
I’d rather have unrestricted AI than moated regulatory capture paid for by the largest existing players.
This is "more regulation" on the states (from the "states' rights" party, no less), and concentrates the potential for regulatory capture into the largest player, the Feds. Who just accepted a $400M gift from Qatar and have a Trump cryptocurrency that gets you access to the President.
It's unsurprising this stuff gets flagged. Half of the Americans on this site voted for this because "regulation bad" or some such. As if mega corps have our best interest at heart and will never do anything blatantly harmful to make a buck.
[deleted]
Fear gets our attention. That alone makes it suspect to me: fear smells like marketing.
I want to bet 10 million that this won’t happen if anyone wants to go against my position. Best bet ever. If i lose, i don’t have to pay anyways.
Some people do actually have end of the world bets out but you have to structure it differently. What you do is the person who thinks the world will end is paid cash right now, and then in N years when the world hasn't ended they have to pay back some multiple of the amount the original amount.
Assuming you can find them. If I took a bet like that you'd have a hell of a time finding me!
(I'm sure serious, or "serious," people who actually construct these bets of course require the "world still here" payout be escrowed. Still.)
(I'm sure serious, or "serious," people who actually construct these bets of course require the "world still here" payout be escrowed. Still.)
If you escrow the World Still Exists payment you lose the benefit of having the World Ends payment immediately.
Yeah, it isn't a kind of bet that makes any sense except as a conversation starter. Imagine needing to pay so much money for one of those!
I still don't get how this is supposed to work. So let's say I give you a million dollars right now, with the expectation that I get $10M back in 10 years when the world hasn't ended. You obviously wanted the money up front because you're going to live it up while the world's still spinning. So how am I getting my payout after you've spent it all on hookers and blow?
Yeah I wouldn't make a deal like this with someone who is operating in bad faith... The cases I've seen of this are between public intellectuals with relatively modest amounts of money.
Well that's what I don't get, how is spending the money bad faith? Aren't they getting the money ahead of time so they can spend it before the world ends? If they have to keep the world-still-here money tied up in escrow I don't see why they would take the deal.
How can you work around if you don’t have millions upfront?
This is such an obviously bad idea; I've heard anecdotes of embezzlement cases where investigation took more than e.g. 5 years, and when it was finally established that yes, the funds really were embezzled and they went after the perpetrator, it turned out that the guy had died a year before due to all of the excesses he spent the money on.
I mean, if you talk from the position of someone who doesn't believe that the world will end soon.
I mean, if you talk from the position of someone who doesn't believe that the world will end soon.
You can start that bet on prediction markets.
same with nuclear war. end of the world is bullish
[deleted]
I can't believe anyone still gives this guy the time of day. He didn't know what test/train split was, but is an AI expert? Give me a break
Do you have a source for this? I've seen this repeated but nobody can ever produce any evidence.
It was all over twitter, I don't know if its still there, but this guy is a performance artist
I don't think he's a bozo; but every technology needs a contrarian, to keep the technologists from spinning too much hype.
Except when that is literally all you are. All of your takes are always just contrarian. He's clearly found an audience and isn't interested in anything other than peddling his fears.
I thought that contrarian was Jaron Lanier.
[deleted]
throwanem(2)
throw310822(2)
CalRobert(1)
> Do we really think things will move this fast? Sort of no - between the beginning of the project last summer and the present, Daniel’s median for the intelligence explosion shifted from 2027 to 2028. We keep the scenario centered around 2027 because it’s still his modal prediction (and because it would be annoying to change). Other members of the team (including me) have medians later in the 2020s or early 2030s, and also think automation will progress more slowly. So maybe think of this as a vision of what an 80th percentile fast scenario looks like - not our precise median, but also not something we feel safe ruling out.
They went from "this represents roughly our median guess" in the website to "maybe think of it as an 80th percentile version of the fast scenario that we don't feel safe ruling out" in followup discussions.
Claiming that one reason they didn't change the website was because it would be "annoying" to change the date is a good barometer for how seriously anyone should be taking this exercise.