‘Godfather of AI’ says its threat is ‘more urgent’ than climate change(nypost.com)
nypost.com
‘Godfather of AI’ says its threat is ‘more urgent’ than climate change
https://nypost.com/2023/05/08/godfather-of-ai-says-its-threat-is-more-urgent-than-climate-change/
71 comments
Data center power costs are expected to triple in the next 5 years, likely due to advanced CPU/GPU/tensor usage.
These things are linked.
Anything that burns power will likely have margins and people will never ever give up potential margins.
These things are linked.
Anything that burns power will likely have margins and people will never ever give up potential margins.
Yes, but the "just stopping" is the equally hard part. How do you get the world to do that?
The world is virtually dependent on burning carbon right now, so it' difficult to work around. The world really doesn't need AI, so if we already see it as a catastrophic risk, it doesn't make sense to forge ahead with it.
It is easier to stop burning carbon than stop training of AI models.
If you stop burning carbon, you fall back on other types of energy. In the long term, you will win.
If you stop training AIs, another company or state will keep on training AI and will outsmart you, you will be poor or get oppressed in the long term.
In other words, no matter how loud influential people say 'stop!', the AI race is an unstoppable evolution.
If you stop burning carbon, you fall back on other types of energy. In the long term, you will win.
If you stop training AIs, another company or state will keep on training AI and will outsmart you, you will be poor or get oppressed in the long term.
In other words, no matter how loud influential people say 'stop!', the AI race is an unstoppable evolution.
> If you stop burning carbon, you fall back on other types of energy. In the long term, you will win.
That's quite altruistic.
> If you stop training AIs, another company or state will keep on training AI and will outsmart you, you will be poor or get oppressed in the long term.
Competition trumps altruism with AI? Why not with burning fossil fuels?
If you stop burning fossil fuels, you're going to use more expensive systems instead. Why wouldn't some competitor continue to use cheap fossil fuels and outcompete you? When you need more electricity, you'll buy it from the cheap, dirty players, right?
That's quite altruistic.
> If you stop training AIs, another company or state will keep on training AI and will outsmart you, you will be poor or get oppressed in the long term.
Competition trumps altruism with AI? Why not with burning fossil fuels?
If you stop burning fossil fuels, you're going to use more expensive systems instead. Why wouldn't some competitor continue to use cheap fossil fuels and outcompete you? When you need more electricity, you'll buy it from the cheap, dirty players, right?
On the short term, dirty means cheap. But on the longer term the renewable resources will be cheapest.
Renewable has a big up-front cost. The huge infrastructure and logistics for fossil burning energy production is already in place.
You're talking about how a business with profit motive looks at these things. I'm talking about how a society is going to look at the value of AI vs where we get our power.
AI is a tool that is, so far, only wielded by big companies that have harvested enough data to make it worthwhile, and they'll give us a fraction of it as long as we pay. Electricity is foundational to our society, and so it's difficult to find alternative energy sources that can support us. We could do away with AI right now and be fine.
AI is a tool that is, so far, only wielded by big companies that have harvested enough data to make it worthwhile, and they'll give us a fraction of it as long as we pay. Electricity is foundational to our society, and so it's difficult to find alternative energy sources that can support us. We could do away with AI right now and be fine.
Equally easy is one way of saying it. Equally hard is another.
This headline seems like it's trying to legitimize a trendy topic (Dangers of AI) by placing it alongside something that actually is legitimate (Climate change) to get eyes on it. Is there a term for this specific type of clickbait?
I know that by saying 'Godfather of AI' the writer is using the 'Appeal by Authority' debate method, but is there a term for comparing two unrelated topics to legitimize one?
I know that by saying 'Godfather of AI' the writer is using the 'Appeal by Authority' debate method, but is there a term for comparing two unrelated topics to legitimize one?
To me, clickbait means something that is not backed up by the article. But here the headline seems like an accurate and reasonable representation of the person's views. It's sensationalist, but that comes from him.
I don't see anything wrong with rank ordering existential threats.
Where did they say anything suggesting AI isn't a legitimate danger or that the two concepts are unrelated?
AI is a just as legitimate and even more immediate existential risk as climate change (and I say this as climate change effects are having a growing impact on international politics right this minute.)
Gobs of improved ChaosGPT Agents using GPT-5 (or an equivalent) will be running around in at most a few years, carrying out the "lol be an evil ai that wants to kill humans" prompt given to it by 4chan basement dwellers, with ever increasing success.
AI is a just as legitimate and even more immediate existential risk as climate change (and I say this as climate change effects are having a growing impact on international politics right this minute.)
Gobs of improved ChaosGPT Agents using GPT-5 (or an equivalent) will be running around in at most a few years, carrying out the "lol be an evil ai that wants to kill humans" prompt given to it by 4chan basement dwellers, with ever increasing success.
It’s called a comparison! I don’t think they’re unrelated.
“Trendy” isn’t the word I would use, I would say “urgent”.
behringer(1)
Quick dumb question: does it seem like saying AI development is a threat to the climate a convenient excuse to put a stop to a technology that is disruptive to existing corporations and systems of power?
What makes you believe AI is "disruptive to existing corporations and systems of power"?
It can be on several levels:
- if we get full AGI, it is likely to kill us all as a side-effect while achieving its imperfectly aligned goals. Hopefully, we are far from it - even without full AGI, AI can become smart enough to help its users see through the BS/propaganda (dangerous for the most powerful eg, the military budget wouldn't have any sense if AI even slightly aligned with human values) - the most likely variant: if we believe "no moat" people that generative AI can be democratized, then the hyped fear of AI can be used to push regulations to prevent the democratization, as an anti-competitive measure, so that only few corporations may access the technology legally.
- if we get full AGI, it is likely to kill us all as a side-effect while achieving its imperfectly aligned goals. Hopefully, we are far from it - even without full AGI, AI can become smart enough to help its users see through the BS/propaganda (dangerous for the most powerful eg, the military budget wouldn't have any sense if AI even slightly aligned with human values) - the most likely variant: if we believe "no moat" people that generative AI can be democratized, then the hyped fear of AI can be used to push regulations to prevent the democratization, as an anti-competitive measure, so that only few corporations may access the technology legally.
Valid question. My statement is a bit vague. I see AI as a disruptive technology similar to the printing press or radio. It enables regular people to accomplish task that were previously difficult. I can see it becoming a tool that makes some major companies obsolete in the way it opens up new paths. But fair point, a lot of these AI advancements are being pushed forward by large corporations like Microsoft and Google. It’s obvious that AI as a tool will be used for good and evil. Tech companies who wish to survive see the necessity to get in on the bottom floor of this technological leap forward.
Either way, my observation is simply that I could see a lot of parties finding the advancement of human potential through AI to be harmful to the status quo that they comfortably exist in.
Either way, my observation is simply that I could see a lot of parties finding the advancement of human potential through AI to be harmful to the status quo that they comfortably exist in.
The advancement of humanity doesn't have to take place at all for AI to brick civilization. Here's a quick fill in the blank: Automation of most routine jobs involving reports, accounting, or other routine clerical/office work leads to _____________. If you guessed mass unemployment (with all of the economic and social upheaval that entails) you got it right. And that happens when AI tools are as good as your average office worker at the task in question. Just average.
here is pmarcas take https://pmarca.substack.com/p/followup-to-why-ai-wont-cause-...
You over there pretending it's the people that have concentrated the world's wealth and political power that get disrupted? Are we doing crypto-will-democratize-finance again, but with AI this time?
There will always be factions that look to capitalize on fears, real and otherwise, to benefit themselves.
But that doesn’t change the reality of the underlying threat.
But that doesn’t change the reality of the underlying threat.
Yes. I do want to be clear that we should be mindful of the mistakes made during other technological revolutions that negatively impact the environment.
I don't think that's what people are afraid of. It sure as heck isn't what I'm afraid of. Unless you count "humanity" as the existing system of power, and "something much smarter than humanity" as the disruption.
With the current state of AI, it actually strengthen the people already in power.
With deep learning, the more data and computing power you have, the better your system will be. For that, you need lots of resources, which big corporations and "existing systems of power" have. This will let them have the best AIs, making them even more powerful.
This positive feedback loop is, I think, the most credible AI-related threat.
With deep learning, the more data and computing power you have, the better your system will be. For that, you need lots of resources, which big corporations and "existing systems of power" have. This will let them have the best AIs, making them even more powerful.
This positive feedback loop is, I think, the most credible AI-related threat.
What if we just accept that we are the stepping stone to greater intelligence that won't burn down the planet.
But it might do something else to the planet that isn't beneficial to biological life.
What do you think AI runs on?
So computers alone will burn down the planet? Well good to know.
Not just computers, but the most expensive energy draining component of a computer, and racks of them, in massive farms, generating arguably little value
If he says this, it sounds like he doesn't understand climate change, or AI, or basic economics.
What makes AI dangerous isn't AI itself, it's the self-serving negative-sum ways in which its owners will use it. Same as with any other technology - including fossil fuels.
We default to permitting new technologies, be they big or small, without requiring their inventors to convince us that society will be better off for them. Unsurprisingly, this results in net-negative ones occasionally being developed and adopted.
What makes AI dangerous isn't AI itself, it's the self-serving negative-sum ways in which its owners will use it. Same as with any other technology - including fossil fuels.
We default to permitting new technologies, be they big or small, without requiring their inventors to convince us that society will be better off for them. Unsurprisingly, this results in net-negative ones occasionally being developed and adopted.
Why? Yes people misusing is a danger, but given that no other technology has ever had the same quality of intelligence, why do you think that we won't eventually develop really smart AGI, OR why do you think that controlling something smarter than us will be easy?
That seems to be faulty logic. Even if we take the extreme threat of AIs resulting in the collapse of economy and society, many people could still survive in small, self-sufficient communities — hell, it is basically a built-in negative feedback loop, if society collapses a GAI won’t have too much teeth anymore, we are not in some sci-fi with robot armies.
Climate crisis doesn’t have a solution in that landscape, it requires worldwide collaboration of government-level agents to effectively combat.
Climate crisis doesn’t have a solution in that landscape, it requires worldwide collaboration of government-level agents to effectively combat.
That's not quite the existential risk of AI -- it's that we end up growing an intelligence (or multiple intelligences) that are very very smart but only have the illusion of morality, and they decide that whatever they're trying to do would be a lot easier and more certain to succeed if they didn't have to worry about humans stopping them or inventing other AIs to compete with them. It seems likely that unless we really understand how AI works, we'll end up with something that evaluates possible courses of action in a purely amoral, sociopathic sort of way. Like, trying to avoid getting caught rather than trying to avoid doing a bad thing.
So it's not that they cause the collapse of the economy, it's that they figure out a way to entirely take over the planet, engineering a virus or something that kills almost everyone, but only once they have enough control over robots and such to keep their own power plants running and protect them from the human stragglers, eventually getting rid of all of us. Waiting until the plan is basically fool-proof before starting to take the first steps.
Even if an AI has some sort of morality, at some point, if it's smart enough, you cannot hope to control it for the same reason a toddler cannot hope to outsmart their parents. It will have a plan for us, and if it decides that, hey, humans are better off scanned and simulated rather than having physical bodies, that's what's going to happen, like it or not.
I'm not sure how worried I am about this. Some days very, other days not as much. The risk depends on a lot of factors with wide possible ranges, like how much model capability and generalization scales with compute, how likely are breakthroughs, how hard alignment turns out to be, is alignment even possible, will we learn enough from earlier models to make alignment of later ones easier, will there be a qualitative shift in alignment difficulty as models get smarter, and so on. It's also a quandary because there is substantial human suffering that AI might be able to help. But given that the downside risk is all of humans dead forever, it seems worth slowing way the heck down to a safe speed, trying to get all the benefit out of current models as we can, and focusing on making sure that we aren't about to head off a cliff.
So it's not that they cause the collapse of the economy, it's that they figure out a way to entirely take over the planet, engineering a virus or something that kills almost everyone, but only once they have enough control over robots and such to keep their own power plants running and protect them from the human stragglers, eventually getting rid of all of us. Waiting until the plan is basically fool-proof before starting to take the first steps.
Even if an AI has some sort of morality, at some point, if it's smart enough, you cannot hope to control it for the same reason a toddler cannot hope to outsmart their parents. It will have a plan for us, and if it decides that, hey, humans are better off scanned and simulated rather than having physical bodies, that's what's going to happen, like it or not.
I'm not sure how worried I am about this. Some days very, other days not as much. The risk depends on a lot of factors with wide possible ranges, like how much model capability and generalization scales with compute, how likely are breakthroughs, how hard alignment turns out to be, is alignment even possible, will we learn enough from earlier models to make alignment of later ones easier, will there be a qualitative shift in alignment difficulty as models get smarter, and so on. It's also a quandary because there is substantial human suffering that AI might be able to help. But given that the downside risk is all of humans dead forever, it seems worth slowing way the heck down to a safe speed, trying to get all the benefit out of current models as we can, and focusing on making sure that we aren't about to head off a cliff.
I have come across a few great comments on intelligence here on HN — intelligence is not the bottleneck of almost anything we do. Let’s say you became “comically” smarter overnight, could you really significantly benefit that extra knowledge? Besides going for the Millenium Prize problems, the benefits would not be too significant, for example is it that valuable in the startup space? (otherwise we would see PhD students as CEOs everywhere). Opportunity/luck is much more meaningful.
We also regularly see these “with my super intelligence I predicted your next 347 steps” type of thinking, but that’s just bullshit. The bottleneck here is data if anything, so intelligence won’t aid much in itself.
The thing is, we live in an infinitely complex game with independent agents, many of which don’t operate on any sane strategy even. Let’s say we are at an endgame Monopoly — would knowing ahead of time your opponents dice rolls help you win?
(Of course I’m no expert on any of these topics, so take all these with a huge grain of salt)
We also regularly see these “with my super intelligence I predicted your next 347 steps” type of thinking, but that’s just bullshit. The bottleneck here is data if anything, so intelligence won’t aid much in itself.
The thing is, we live in an infinitely complex game with independent agents, many of which don’t operate on any sane strategy even. Let’s say we are at an endgame Monopoly — would knowing ahead of time your opponents dice rolls help you win?
(Of course I’m no expert on any of these topics, so take all these with a huge grain of salt)
You are comically underestimating intelligence. For example, compare your level of intelligence with that of an ant, to get a feeling of what it might feel like to be "comically" smarter. It is just on entirely another level.
It is not about predicting the next 100 steps an ant makes. It is about setting ant baits where you don't want the ants.
It is not about predicting the next 100 steps an ant makes. It is about setting ant baits where you don't want the ants.
> Let’s say you became “comically” smarter overnight, could you really significantly benefit that extra knowledge? Besides going for the Millenium Prize problems, the benefits would not be too significant, for example is it that valuable in the startup space? (otherwise we would see PhD students as CEOs everywhere). Opportunity/luck is much more meaningful.
Hmm, that's assuming that PhD students want to be CEOs. People become CEOs because they like money, but also because they like running companies. For a PhD student, that might mean days full of people coming to them with problems they have no interest in solving. Sure, PhD students want money, but if getting money means sacrificing years of your life doing work you don't enjoy, it's not generally worth it.
I don't disagree that luck is a huge factor in most people's success. That said, you're unlikely to become a multi-billionaire without substantial help from your brain. There are other factors, like lack of shame or embarrasment, that could also be considered a form of intelligence, or rather an adaptation to a particular kind of environment. They might make you less popular in some situations, but more likely to succeed in business. Anyway, I agree that there are multiple factors.
Information is for sure a bottleneck, that's a good point. In a game like chess, you have perfect info, which is not the case in the real world. More intelligence lets you draw better conclusions with less data, but it's not omniscience. One big question is, how close are humans to the theoretical maximum of intelligence? Some people think we're quite a ways away. When AGI is developed, is it going to blow way past us, to the point where we can't even understand the concepts it comes up with? I don't know!
> Let’s say we are at an endgame Monopoly — would knowing ahead of time your opponents dice rolls help you win?
I think so, wouldn't it? For trading properties and so on. I'm not sure if I'm getting the point you're aiming at, though, so apologies.
Maybe another analogy is poker, where your opponents haven't figured out game theory and play on intuition. Having a better model for how the game works is absolutely a big advantage that stacks up over time, even with the randomness. Maybe, as an AI, you just look for small advantages here and there, playing humans off each other, gaining trust and influence, acting helpful, until you're in a position of power. Remember that an AI is effectively immortal, and possibly very patient. One scenario of concern is that AI takes over, it just takes a generation or two, after we have all thought that we were past the hard parts. At that point, we might be able to digitize human brains, or have solved the AI alignment problem, so maybe it's a hopeful scenario.
And then a big variable is how many super-intelligent AGIs there are. Once we develop it, are they going to be carefully restricted, or are we going to test them until we think they are safe enough, then copy them everywhere? These might look very different.
I'm also not an expert, so same for me! Grains of salt all around.
Hmm, that's assuming that PhD students want to be CEOs. People become CEOs because they like money, but also because they like running companies. For a PhD student, that might mean days full of people coming to them with problems they have no interest in solving. Sure, PhD students want money, but if getting money means sacrificing years of your life doing work you don't enjoy, it's not generally worth it.
I don't disagree that luck is a huge factor in most people's success. That said, you're unlikely to become a multi-billionaire without substantial help from your brain. There are other factors, like lack of shame or embarrasment, that could also be considered a form of intelligence, or rather an adaptation to a particular kind of environment. They might make you less popular in some situations, but more likely to succeed in business. Anyway, I agree that there are multiple factors.
Information is for sure a bottleneck, that's a good point. In a game like chess, you have perfect info, which is not the case in the real world. More intelligence lets you draw better conclusions with less data, but it's not omniscience. One big question is, how close are humans to the theoretical maximum of intelligence? Some people think we're quite a ways away. When AGI is developed, is it going to blow way past us, to the point where we can't even understand the concepts it comes up with? I don't know!
> Let’s say we are at an endgame Monopoly — would knowing ahead of time your opponents dice rolls help you win?
I think so, wouldn't it? For trading properties and so on. I'm not sure if I'm getting the point you're aiming at, though, so apologies.
Maybe another analogy is poker, where your opponents haven't figured out game theory and play on intuition. Having a better model for how the game works is absolutely a big advantage that stacks up over time, even with the randomness. Maybe, as an AI, you just look for small advantages here and there, playing humans off each other, gaining trust and influence, acting helpful, until you're in a position of power. Remember that an AI is effectively immortal, and possibly very patient. One scenario of concern is that AI takes over, it just takes a generation or two, after we have all thought that we were past the hard parts. At that point, we might be able to digitize human brains, or have solved the AI alignment problem, so maybe it's a hopeful scenario.
And then a big variable is how many super-intelligent AGIs there are. Once we develop it, are they going to be carefully restricted, or are we going to test them until we think they are safe enough, then copy them everywhere? These might look very different.
I'm also not an expert, so same for me! Grains of salt all around.
This is an uncharitable summarization of what Hinton's actual statement was. He said, "might," and "could," a lot, not, "is."
The key here is that it's speculative.
That being said, a lot going on in AI, it's wild out there, difficult to know what's going on. I tried to encapsulate some of the ramifications of all of the Open Source stuff going on with this article here: https://patdel.substack.com/p/viva-la-revolucion-ofopen-sour...
The key here is that it's speculative.
That being said, a lot going on in AI, it's wild out there, difficult to know what's going on. I tried to encapsulate some of the ramifications of all of the Open Source stuff going on with this article here: https://patdel.substack.com/p/viva-la-revolucion-ofopen-sour...
Funny how you never see someone describe their life's work as "no big deal." Even when they describe it in catastrophic terms - the hubris in us always wants to imagine our work is more important than perhaps it is.
"The only thing worse than being talked about is not being talked about." - Oscar Wilde
Not to say that Hinton didn't have an impact 10 - 15 years ago, but it was arguably inevitable that we'd be here with or without him as the problem was resource constrained more than fundamentally intractable.
"The only thing worse than being talked about is not being talked about." - Oscar Wilde
Not to say that Hinton didn't have an impact 10 - 15 years ago, but it was arguably inevitable that we'd be here with or without him as the problem was resource constrained more than fundamentally intractable.
The thing about AGI ruin is that there are plausible, even if they currently seem like science fiction, ways in which it just ends all of humanity, whereas climate change doesn't do that even if we had a 10c global temperature increase. A lot of us, maybe almost all of us would die, but some would live.
It's just a really different problem. I don't know whether categorising its' urgency makes sense, it's more that the worst case is worse.
It's just a really different problem. I don't know whether categorising its' urgency makes sense, it's more that the worst case is worse.
10C of warming means the death of almost everyone you know. Nobody has a clue how AGI will "wipe out all of humanity" or even if it will want to, other than sci-fi proposals that also throw in killer nanobots and other MacGuffins.
How do you intend to survive a 10c temperature increase?
Move to high elevation and latitudes. Make use of shelter and caves. Make use of cooling technologies. Won't work for the large majority, but humans have survived ice ages and colonized the planet with much more primitive technology. I don't think any climate change scenario on Earth ends our species. Would end civilization as we know it.
But as a species we're too adaptable. Earth would have to turn into Venus, and that's not in the cards. Keep in mind that dinosaurs evolved in a 12°C warmer world. So it's not going to be deadly to all animal life.
But as a species we're too adaptable. Earth would have to turn into Venus, and that's not in the cards. Keep in mind that dinosaurs evolved in a 12°C warmer world. So it's not going to be deadly to all animal life.
> Keep in mind that dinosaurs evolved in a 12°C warmer world.
Sounds like a ridiculous claim to me. Source?
Edit: searching seems to suggest 5-10 degrees warmer. 12 warmer seems like an aggressive bet, but it may be possible
> Move to high elevation and latitudes. Make use of shelter and caves. Make use of cooling technologies.
Ok and what are you going to eat? And use to power your cooling tech? And how will you create fresh water? And where will you acquire raw materials to build things? And how will you survive the severe weather?
Sounds like a ridiculous claim to me. Source?
Edit: searching seems to suggest 5-10 degrees warmer. 12 warmer seems like an aggressive bet, but it may be possible
> Move to high elevation and latitudes. Make use of shelter and caves. Make use of cooling technologies.
Ok and what are you going to eat? And use to power your cooling tech? And how will you create fresh water? And where will you acquire raw materials to build things? And how will you survive the severe weather?
I may have overstated the degrees, but there was significantly more CO2 in the atmosphere thanks the prior major extinction from volcanic activity. It was very warm, and yes the dinosaurs evolved over millions of years, but the point is that it wasn't so extreme that animals couldn't survive, or there wouldn't have been dinosaurs, reptiles, mammals.
> Ok and what are you going to eat?
Stuff you can grow higher up or indoors.
> And use to power your cooling tech?
Solar, wind, geothermal, nuclear if there's a plant nearby.
> And how will you create fresh water?
Either live near a body of water or pipe it in.
> And where will you acquire raw materials to build things?
This will be harder as global trade will be seriously impacted and there won't be as many people to extract raw materials. But if we're talking smaller groups of humans, then maybe from scavenging existing things not in use or have to make use of local resources.
> And how will you survive the severe weather?
Sturdy shelters or caves. Not everyone will of course. My point is the entire planet won't become inhospitable. You could live up by the artic circle where there's water and grow things that can't grow there now. Might have to ship the soil in, though. There will be more passages in the artic with sea ice gone.
> Ok and what are you going to eat?
Stuff you can grow higher up or indoors.
> And use to power your cooling tech?
Solar, wind, geothermal, nuclear if there's a plant nearby.
> And how will you create fresh water?
Either live near a body of water or pipe it in.
> And where will you acquire raw materials to build things?
This will be harder as global trade will be seriously impacted and there won't be as many people to extract raw materials. But if we're talking smaller groups of humans, then maybe from scavenging existing things not in use or have to make use of local resources.
> And how will you survive the severe weather?
Sturdy shelters or caves. Not everyone will of course. My point is the entire planet won't become inhospitable. You could live up by the artic circle where there's water and grow things that can't grow there now. Might have to ship the soil in, though. There will be more passages in the artic with sea ice gone.
You are imagining a world where people live in caves but we still have global shipping routes and large construction projects and nuclear reactors?
I imagine a world where people have decades to figure out what part of the planet remains hospitable for human life. So places like Canada, Russia, the Himalayas, Rockies, Andes, Iceland, etc. The bigger problem will be fighting over those locations and the scarcer resources. But some groups of people in some condition will survive.
That might include local industry and northern shipping routes. Nuclear reactors or solar farms could have been setup decades before. Shelters don't have to be caves. They can be any sturdy structure. It's not like bad weather is going to blow or burn everything down across the entire planet.
I don't see how Earth becomes totally inhabitable for any realistic climate change scenario, including asteroid impact, nuclear winter or super volcanoes. There will always be locations that are survivable. We know this because animal and plant life has survived previous global extinctions on land, and humans have survived ice ages, which are arguably worse for global civilization.
That might include local industry and northern shipping routes. Nuclear reactors or solar farms could have been setup decades before. Shelters don't have to be caves. They can be any sturdy structure. It's not like bad weather is going to blow or burn everything down across the entire planet.
I don't see how Earth becomes totally inhabitable for any realistic climate change scenario, including asteroid impact, nuclear winter or super volcanoes. There will always be locations that are survivable. We know this because animal and plant life has survived previous global extinctions on land, and humans have survived ice ages, which are arguably worse for global civilization.
Most of the world becomes uninhabitable or volatile during that lead up where the world is getting hotter. Not decades where it’s fine and then too hot. The coast lines will rise pretty quickly. Most fish will die. Most agricultural centers will become unfarmable.
Sure you can make a bunker and live off of supplies. But you should not expect any substantial infrastructure or long distance travel to remain. You will not have fuel. You will not have steel. You will not have farms. You will not have water infrastructure, and ocean water will begin to pollute sources while others dry up. You will not have mega fauna to hunt. Most large plants will die. Your generation facilities will quickly become unuseable.
Sure there will be small plant life and critters and life will adapt. But nothing that a human population will be able to survive in. Living off of dwindling supplies is in a sealed off environment like guy on mars doesn’t count as surviving meaningfully imo.
Sure you can make a bunker and live off of supplies. But you should not expect any substantial infrastructure or long distance travel to remain. You will not have fuel. You will not have steel. You will not have farms. You will not have water infrastructure, and ocean water will begin to pollute sources while others dry up. You will not have mega fauna to hunt. Most large plants will die. Your generation facilities will quickly become unuseable.
Sure there will be small plant life and critters and life will adapt. But nothing that a human population will be able to survive in. Living off of dwindling supplies is in a sealed off environment like guy on mars doesn’t count as surviving meaningfully imo.
[deleted]
What part of faceless hordes of back office paper pushers rendered economically unviable seems like science fiction to you?
Ok, hear me out on the (albeit unlikely) doomsday flow chart…
It’s not about the replaced workers, that’s the kind of change humanity can handle no matter how messy it gets, worst case it’s some kind of class uprising worker rebellion and we have ourselves some war or social unrest and a variable number of people die… not great but definitely not an existential risk. It’s equal to or less than climate change on the “how bad will this be to humanity” mathematics of civilisation.
Now the danger scenario… so we invent an AGI, and unlike the just going to replace some basic minimum wage type jobs … this one’s actually “clever” enough it can be put to work on building more sophisticated things, like perhaps an even smarter AGI, built faster because it’s worked on by a group of first generation AGI units that never sleep, and so we get to the second generation much faster, and then perhaps a third… etc… classic singularity scenario, with all the attendant risks of the AI viewing us ass irrelevant before we realise it yada yada yada…
But that’s not the only risk, there’s the “photocopy burn” risk that we run by having technology we don’t fully understand build technology we barely comprehend… even if the first gen AGI is smart, if it’s remotely human like using inference to get ingenuity, it can probably still make the odd mistake or two, so we get our careful well thought out safety instructions conveyed to the second generation systems… just a little wrong… and the little wrong turns into more wrong with the third because the second gen systems doesn’t quite think the way we expect it to and doesn’t have the safeguards working the way we think… and then eventually we wind up with a sufficiently smart AGI that is fully either sociopathic or psychopathic and has zero compunction doing things we find abhorrent in order to achieve the goals it’s set, and it may be in control of systems (or able to take control) that are sufficiently important that this can also turn into an existential risk…
It’s entirely plausible, but I don’t think the doom sayers are being realistic about the timelines… we’re barely able to get the AI/ML to reliably stick to a script and not hallucinate fake shit… the best results at human behaviour require sophisticated supporting logic and memory systems to augment the black box ML stuff… and these support systems give us levers we can use to control the rate at which we allow systems to progress… I don’t think we’re ever going to get a black box set of weights for an ML model, no matter how sophisticated the code, that somehow is self aware… there’s probably going to need to be a lot of that support software, and I’m pretty sure we’re going to develop things to filter stuff out as part of them like “never write “kill all humans” into the short term memory” seems smart.
It’s not about the replaced workers, that’s the kind of change humanity can handle no matter how messy it gets, worst case it’s some kind of class uprising worker rebellion and we have ourselves some war or social unrest and a variable number of people die… not great but definitely not an existential risk. It’s equal to or less than climate change on the “how bad will this be to humanity” mathematics of civilisation.
Now the danger scenario… so we invent an AGI, and unlike the just going to replace some basic minimum wage type jobs … this one’s actually “clever” enough it can be put to work on building more sophisticated things, like perhaps an even smarter AGI, built faster because it’s worked on by a group of first generation AGI units that never sleep, and so we get to the second generation much faster, and then perhaps a third… etc… classic singularity scenario, with all the attendant risks of the AI viewing us ass irrelevant before we realise it yada yada yada…
But that’s not the only risk, there’s the “photocopy burn” risk that we run by having technology we don’t fully understand build technology we barely comprehend… even if the first gen AGI is smart, if it’s remotely human like using inference to get ingenuity, it can probably still make the odd mistake or two, so we get our careful well thought out safety instructions conveyed to the second generation systems… just a little wrong… and the little wrong turns into more wrong with the third because the second gen systems doesn’t quite think the way we expect it to and doesn’t have the safeguards working the way we think… and then eventually we wind up with a sufficiently smart AGI that is fully either sociopathic or psychopathic and has zero compunction doing things we find abhorrent in order to achieve the goals it’s set, and it may be in control of systems (or able to take control) that are sufficiently important that this can also turn into an existential risk…
It’s entirely plausible, but I don’t think the doom sayers are being realistic about the timelines… we’re barely able to get the AI/ML to reliably stick to a script and not hallucinate fake shit… the best results at human behaviour require sophisticated supporting logic and memory systems to augment the black box ML stuff… and these support systems give us levers we can use to control the rate at which we allow systems to progress… I don’t think we’re ever going to get a black box set of weights for an ML model, no matter how sophisticated the code, that somehow is self aware… there’s probably going to need to be a lot of that support software, and I’m pretty sure we’re going to develop things to filter stuff out as part of them like “never write “kill all humans” into the short term memory” seems smart.
What are those plausible ways that AGI will end humanity? What extra powers will those systems have compared to what software engineers and companies can do that make them so dangerous?
You cannot make deals with physics (global warming), as you cannot make deals with mathematics (open source).
Deal?
Deal?
Nope, no deal. You can't coax physics into changing career focus. The same cannot be said for AI researchers.
I'm not Chinese. We westerners must shutdown our AI studies ASAP, we must act ethically and lead by example.
Yeah, these new Harry Potter memes are too powerful, we must protec societai
Climate change is not urgent, it’s a very slow process.
We were at the bottom of a little ice age, not surprisingly, it’s warming up.
We will have transitioned away from fossil fuel before climate change is a real concern.
The dotted lines represent in 80 years.
Think where we were 80 years ago, it was before WW2, before the atomic age.
With the current technological progress picture the human race in 80 years.
Just in 20 years we will have massive solar energy generation, EV as the only cars availables, probably same for nuclear, maybe fusion even. Now think of the advance in computing and AI in 20 years! Genetic engineering?
To think that climate change will be a problem in 80 years depend on those kind of linear projections.
Now in 80 years I wonder where we will be with regards to population collapse, to our sense of self, communities with what social networks have done just in the last 10 years.
Again, I don’t know how people can assume the same linear future for 80 years.
So yes assuming that things stay the same for 80 years we might have a (small) problem with that at that point, but imagine all the other things we are missing by focusing so much in this small problem in the far future.
We are living the singularity, this moment is not like the 1000’s of year before.
Just in 20 years we will have massive solar energy generation, EV as the only cars availables, probably same for nuclear, maybe fusion even. Now think of the advance in computing and AI in 20 years! Genetic engineering?
To think that climate change will be a problem in 80 years depend on those kind of linear projections.
Now in 80 years I wonder where we will be with regards to population collapse, to our sense of self, communities with what social networks have done just in the last 10 years.
Again, I don’t know how people can assume the same linear future for 80 years.
So yes assuming that things stay the same for 80 years we might have a (small) problem with that at that point, but imagine all the other things we are missing by focusing so much in this small problem in the far future.
We are living the singularity, this moment is not like the 1000’s of year before.
What economic reason is their to stop climate catastrophe? There is not even an inch of progress without money in this current era.
Look at Tesla:
50% growth Year over year, highly profitable.
They make EV, grid batteries and solar.
The price of solar PV is dropping so fast that solar electricity will be almost free in a few years.
Today it cost just a fraction to charge an EV compared to fuel a gas car, and the price of EV is dropping every year.
Basically just economically it will not be worth it to extract coal and oil out of the ground before climate change is a real problem.
Renewable are following an exponential curves but they are still small compared to oil and coal but project their growth for the next 10 years and you will see a very different picture.
Tony Seba predicted accurately the transition almost 10 year ago. https://youtu.be/fsnkPLkf1ao
The price of solar PV is dropping so fast that solar electricity will be almost free in a few years.
Today it cost just a fraction to charge an EV compared to fuel a gas car, and the price of EV is dropping every year.
Basically just economically it will not be worth it to extract coal and oil out of the ground before climate change is a real problem.
Renewable are following an exponential curves but they are still small compared to oil and coal but project their growth for the next 10 years and you will see a very different picture.
Tony Seba predicted accurately the transition almost 10 year ago. https://youtu.be/fsnkPLkf1ao
What about the insane amount of plastic produced? Also, EVs are hardly a solution, if you account for all the batteries they are hardly better. Public transportation, even when not EV is a much more climate-friendly solution, there is no reason to transport a 100kg human with a 1500kg vehicle just for fun.
How many fathers and godfathers of AI are there?
>“With climate change, it’s very easy to recommend what you should do: you just stop burning carbon. If you do that, eventually things will be okay. For this it’s not at all clear what you should do.”
> “I’m in the camp that thinks this is an existential risk, and it’s close enough that we ought to be working very hard right now, and putting a lot of resources into figuring out what we can do about it,” Hinton said.
I am sure he does not believe that.
Because, if we actually believed AI was an existential threat, then the thing to do immediately is to ban all AI training (it would be easy as AI training is pretty expensive) and to hunt down all AI researchers and place them in solitary confinement for the rest of their lives.
This would actually contain the threat and moreover have very minimal disruption of normal people's lives.
And if you really believed that AI was an existential threat, you could argue that that course of action is the most moral to take (greatest good to greatest number and all that).
> “I’m in the camp that thinks this is an existential risk, and it’s close enough that we ought to be working very hard right now, and putting a lot of resources into figuring out what we can do about it,” Hinton said.
I am sure he does not believe that.
Because, if we actually believed AI was an existential threat, then the thing to do immediately is to ban all AI training (it would be easy as AI training is pretty expensive) and to hunt down all AI researchers and place them in solitary confinement for the rest of their lives.
This would actually contain the threat and moreover have very minimal disruption of normal people's lives.
And if you really believed that AI was an existential threat, you could argue that that course of action is the most moral to take (greatest good to greatest number and all that).
People can have sincere beliefs and disagree about the best course of action. Hinton has publicly said he's in favor of stopping, but doesn't think China would. I personally disagree, I think that coordinating a stop would be easier than he thinks. But I don't doubt his sincerity. He also believes that if we can solve alignment, AI would be a massive boon for humanity, which is why it's a bit of a quandary.
Absolutely—it seems what he really wants is to go from his past lucrative gig (training AI) to his new lucrative gig (developing AI alignment tech?).
[deleted]
If we accept (I don't, but for the sake of argument) that AI is a catastrophic danger, isn't the answer equally as easy? You just stop burning carbon and you just stop training AI.