If Anyone Builds IT, Everyone Dies
7 comments
"If Anyone Build IT..." is a cool-sounding trope. Which pushes lots of human emotional buttons. Making that trope a great thing to push, if you goal is to get ahead in our attention economy.
From my PoV, our "shortish-term profits are the ONLY metric that matters" global techno-capitalist system is vastly more dangerous than AI.
From my PoV, our "shortish-term profits are the ONLY metric that matters" global techno-capitalist system is vastly more dangerous than AI.
>if you goal is to get ahead in our attention economy.
If a person wants a reputation as smart and able to make accurate predictions, why would he focus most of his public statments on a prediction that if he is right, no one will find out definitively that he was right because everyone will be dead?
If a person wants a reputation as smart and able to make accurate predictions, why would he focus most of his public statments on a prediction that if he is right, no one will find out definitively that he was right because everyone will be dead?
Yes, other motives are possible. Though "get ahead in our attention economy" provides a strong positive feedback loop. And if the trope is very widely circulated, it will find and activate most of its potential Apostles - people who will both strongly believe it, and also zealously try to spread it.
Bigger picture - I see AI as just another bale of straw on the camel's back. Sure, it's possible that AI will end up looking like the fatal bale. But that could come from so much money being invested in AI that we get a global financial meltdown when the AI bubble bursts, leading to a nuclear WWIII.
Bigger picture - I see AI as just another bale of straw on the camel's back. Sure, it's possible that AI will end up looking like the fatal bale. But that could come from so much money being invested in AI that we get a global financial meltdown when the AI bubble bursts, leading to a nuclear WWIII.
... or "you can get ahead through telling people what they want to hear" or "people mistake dark personality traits for high agency", etc.
What gets me about 'rationalism' is how ahistorical it is. No reference to Korzybski or Teilhard de Chardin. There never was the movie Colossus the Forbin Project or the 1960s speculation that progress in computing would lead to accelerated progress in computing by an equation like
What gets me about 'rationalism' is how ahistorical it is. No reference to Korzybski or Teilhard de Chardin. There never was the movie Colossus the Forbin Project or the 1960s speculation that progress in computing would lead to accelerated progress in computing by an equation like
dx 2
-- = x
dt
>No reference to Korzybski or Teilhard de Chardin
Pointing a search engine at site:lesswrong.com finds many references to either thinker.
Pointing a search engine at site:lesswrong.com finds many references to either thinker.
I thought it said "If anyone builds IT, everyone dies".
Maybe I take my job too seriously.
I wouldn't be surprised if Yudkowsky would agree that the initial creation of "Information Technology" was an almost direct stepping stone (in evolutionary terms) on the path to AI, which in turn (he argues and I agree) will lead us towards superintelligence. I suppose that Kurzweil might take this even further, arguing that the path was set as soon as an early hominin picked up a stick and intentionally drew an arrow towards berry bushes in the ground.
Oh, by the way, Google is building a system to allow AI to spend money.
https://www.youtube.com/watch?v=8s6nGMcyr7k
Read it. Weep. I'm not sure if the authors have met humanity.
https://www.amazon.com/Anyone-Builds-Everyone-Dies-Superhuman/dp/0316595640