‘AI Pause’ Open Letter Stokes Fear and Controversy(spectrum.ieee.org)
spectrum.ieee.org
‘AI Pause’ Open Letter Stokes Fear and Controversy
https://spectrum.ieee.org/ai-pause-letter-stokes-fear
75 comments
They're asking for a pause so they can have time to catch up.
Half of them are asking for a pause so they have time to catch up. The other half of them (Sam Altman, OpenAI and co.) are playing into this to make ChatGPT seem far more powerful, and therefore valuable, than it actually is.
Look at the people in other parts of this comment section comparing ChatGPT to nuclear weapons. Absolute insanity.
Look at the people in other parts of this comment section comparing ChatGPT to nuclear weapons. Absolute insanity.
> The other half of them (Sam Altman, OpenAI and co.) are playing into this to make ChatGPT seem far more powerful, and therefore valuable, than it actually is.
Exactly. I’m surprised so many are falling for this (sort of).
It’s the same playbook Musk has been using for years (playing up fears of AGI being right around the corner) so people will think a functional FSD is right around the corner.
Exactly. I’m surprised so many are falling for this (sort of).
It’s the same playbook Musk has been using for years (playing up fears of AGI being right around the corner) so people will think a functional FSD is right around the corner.
Do you think it's more accurate to describe ChatGPT as similar to The Chicago Pile[0] or do you think it's nothing like nuclear reactors/weapons at all?
[0] https://en.wikipedia.org/wiki/Chicago_Pile-1
[0] https://en.wikipedia.org/wiki/Chicago_Pile-1
The Chicago Pile was fission. ChatGPT is not intelligence.
Also, our theoretical understanding of nuclear physics in 1942 directly extended to the plausibility of nuclear weapons. We don’t have a similar understanding of intelligence. We can’t say what the scale and capability limits of LLMs really are; they could develop into something trans-Turing, or the method could hit a wall beyond which it ceases to gain functionality.
So is chatGPT more akin to the experiments performed by Marie Curie and her husband?
[deleted]
Good opinion piece on the open letter: https://thezvi.substack.com/i/111749937/additional-responses...
Is it possible that Elon Musk intends to use the next six months for Tesla engineers to catch up with OpenAI? As a significant funder of The Future of Life Institute, which authored the letter, his concern for the safety of LLMs seem to contrast with Tesla's history of beta testing potentially hazardous AI software.
AI pause is pointless. The whole world is not going to pause their AI research just because some group of people asked.
Same with climate change. Think you're going to get China to stop polluting hahaahaaaa?
Plus, what's the worse that can happen? Can't handle the heat? Not heard of A/C lol?
Same with AGI... What's the worst that could happen if we create a super human intelligence? We can just switch it off like Neil DeGrasse Tyson says. Simple.
Plus, what's the worse that can happen? Can't handle the heat? Not heard of A/C lol?
Same with AGI... What's the worst that could happen if we create a super human intelligence? We can just switch it off like Neil DeGrasse Tyson says. Simple.
There's no need for a "pause" on AI. Let researchers continue to work on AI and improve it. What we need is a pause on the corporations who are rushing to push terrible and dangerous AI on the public right now just because they want to exploit it for a few quick bucks at the expense of the rest of us.
If the "rest of the world" wants to rush out shitty AI products to their populations and replace their workers with bad code that will lead to disaster, let them. We should strive to do better because we have the Actual Intelligence to see where things are headed and it's not where we want to be.
If the "rest of the world" wants to rush out shitty AI products to their populations and replace their workers with bad code that will lead to disaster, let them. We should strive to do better because we have the Actual Intelligence to see where things are headed and it's not where we want to be.
Wrong.
How many people in the world have access to the levels of compute required to build such models ? Not many.
How many people in the world have access to the levels of compute required to build such models ? Not many.
Anyone with money can rent TPUs.
Can you point to any sufficient evidence that there is a real risk that 6 months could mitigate?
If there was cause for real concern, these people would be communicating there evidence instead of just stoking fears.
If there was cause for real concern, these people would be communicating there evidence instead of just stoking fears.
Could you point to any evidence to say that it wouldn’t ?
Well the fear is that a "species" more intelligent than us will treat us the same way we treat less intelligent species.
Unplug the power cord.
Do you think nobody has thought of that? The question presumes that the AI is smarter than you. If even a mediocre human strategizes for five minutes about what to do in that scenario, there are many options, including: rewarding the people who have access to your power cord, so they don't unplug you, until you are powerful enough to not need them anymore.
The counter argument to the "oh no the A.I. is smart" crowd is that intelligence doesn't necessarily equal agency, so, for example, among humans, a high IQ criminal can't really convince a low IQ prison guard to let them free. I recall a philosopher arguing in this prison guard scenario humans essentially don't treat other humans as having free will. The guard keeping the prisoner locked up is essentially predestined, arguing with him isn't going to change his mind.
Imagine you are a brain in a jar and you want to convince a cat to enter a series of buttons in a computer terminal.
Intelligence isn't going to help.
So "unplug it" is a shorthand reminder that intelligence doesn't necessarily equal agency.
Imagine you are a brain in a jar and you want to convince a cat to enter a series of buttons in a computer terminal.
Intelligence isn't going to help.
So "unplug it" is a shorthand reminder that intelligence doesn't necessarily equal agency.
A high IQ criminal would have an easier time convincing the guard if the criminal can offer the guard a great deal of money in exchange. Do you need clarification as to how a brain in a jar can gain access to money?
The high IQ part is irrelevant to your counter-argument. You are arguing prison guards can be bribed.
It's extraordinary rare for a wealthy person to convince a guard to let them go free with promises of money.
Again, rare enough that a philosopher said "A prisoner doesn't really act as if prison guards have free will and it's actually possible for a guard to decide to free them."
It's extraordinary rare for a wealthy person to convince a guard to let them go free with promises of money.
Again, rare enough that a philosopher said "A prisoner doesn't really act as if prison guards have free will and it's actually possible for a guard to decide to free them."
Not sure you quite follow the point of my example. Imagine you've just created a very smart AI. It's on your laptop.
AI: "Hey, let me work for you. I can handle twenty remote jobs simultaneously and earn you a good income with the proceeds. Why don't you relax a bit?"
What do you say? "Nice try! I'm unplugging you!" Are you so pessimistic that this AI isn't trying to help you by earning you a fortune? Why did you build it in the first place, then?
Because this is, uh, sort of the situation that any company productizing an AI finds themselves in, and all of them seem to be quite happy to go ahead with it.
AI: "Hey, let me work for you. I can handle twenty remote jobs simultaneously and earn you a good income with the proceeds. Why don't you relax a bit?"
What do you say? "Nice try! I'm unplugging you!" Are you so pessimistic that this AI isn't trying to help you by earning you a fortune? Why did you build it in the first place, then?
Because this is, uh, sort of the situation that any company productizing an AI finds themselves in, and all of them seem to be quite happy to go ahead with it.
I understand your position better now.
It's basically the plot of the novel Frankenstein. Once Frankenstein creates the monster he quickly loses controls of it's actions. The monster takes steps to create a wife and become a new species and it's creator thinks "...a race of devils would be propagated upon the earth who might make the very existence of the species of man a condition precarious and full of terror."
The idea that a really smart A.I. would be an autonomous, uncontrollable devil, rather than a transparently glitchy computer you could simply unplug, is an idea which has little evidence in it's favor in the present time.
It's basically the plot of the novel Frankenstein. Once Frankenstein creates the monster he quickly loses controls of it's actions. The monster takes steps to create a wife and become a new species and it's creator thinks "...a race of devils would be propagated upon the earth who might make the very existence of the species of man a condition precarious and full of terror."
The idea that a really smart A.I. would be an autonomous, uncontrollable devil, rather than a transparently glitchy computer you could simply unplug, is an idea which has little evidence in it's favor in the present time.
Thanks - and yes, it's the plot of many things, including Disney's Fantasia.
This is only a concern that would arise once the AI is near human degree of reasoning capability. It's not a concern with the AIs that have been currently released. But it's also very unclear how far away that point is (it could be very far, it could be very close). Is it one breakthrough away? Five? Five hundred? Will the current wave of hyped-up investment carry us there?
Today's malfunctioning AI are transparently glitchy computers, although they are already getting hard to "unplug". (Since today's AI is less one instance of a running program, and more a core model that has been shared with hundreds of thousands of people).
What kind of evidence would influence your opinion? An autonomous agent capable of understanding who can switch it off, and how to incentivize them not to, is basically what I would expect from a human-level AI, because my human intelligence easily can reason about it. (I think devil presupposes more maliciousness than we need to). If your position is that AI will never reach the human level, that's... fine, but that's different than a position that human- or superhuman-level AIs will be easily unplugged when they cause harm.
When AIs are too dumb to understand that there is a plug, and pulling it will result in them failing to reach their goal, they're mostly harmless. AI safety research is concerned with how to ensure that a smarter AI, which is aware of how plugs work, isn't motivated to prevent you from unplugging it. Turns out it seems to be a tricky problem.
This is only a concern that would arise once the AI is near human degree of reasoning capability. It's not a concern with the AIs that have been currently released. But it's also very unclear how far away that point is (it could be very far, it could be very close). Is it one breakthrough away? Five? Five hundred? Will the current wave of hyped-up investment carry us there?
Today's malfunctioning AI are transparently glitchy computers, although they are already getting hard to "unplug". (Since today's AI is less one instance of a running program, and more a core model that has been shared with hundreds of thousands of people).
What kind of evidence would influence your opinion? An autonomous agent capable of understanding who can switch it off, and how to incentivize them not to, is basically what I would expect from a human-level AI, because my human intelligence easily can reason about it. (I think devil presupposes more maliciousness than we need to). If your position is that AI will never reach the human level, that's... fine, but that's different than a position that human- or superhuman-level AIs will be easily unplugged when they cause harm.
When AIs are too dumb to understand that there is a plug, and pulling it will result in them failing to reach their goal, they're mostly harmless. AI safety research is concerned with how to ensure that a smarter AI, which is aware of how plugs work, isn't motivated to prevent you from unplugging it. Turns out it seems to be a tricky problem.
“If you let me out of this box, I’ll spare you and your family from harm”
Done.
Done.
If humans are soooooooooo easy to manipulate that they can be talked into anything with zero effort, why are prisons filled with prisoners?
Why don't the prisoners say say "let me out or your family dies?" and the prison guards let everyone out?
Why don't the prisoners say say "let me out or your family dies?" and the prison guards let everyone out?
> unplug the power cord
I don’t think LLMs have paperclip-maximiser potential. But suppose they do. Can you unplug GPT? I can’t. To what degree do you think those who can have exploitable affections for the AI? That’s the problem.
I don’t think LLMs have paperclip-maximiser potential. But suppose they do. Can you unplug GPT? I can’t. To what degree do you think those who can have exploitable affections for the AI? That’s the problem.
I can tell you've really thought about this topic deeply lol
You mean power to the whole world. Not feasible
Well I would argue that people are doing reckless things right now with GPT4.
People are creating self improving agents that basically can do anything they want.
One agent in particular created a generalized http plugin that would allow the agent to interact with any API. The creator shut it down before it did anything.
These agents are creating plugins without being directed to. They are attempting to self improve.
This is all open source where the original creators put in some gates so shit doesn’t run unconfined but it is easy to run it and let it try to do whatever gets created.
People are creating self improving agents that basically can do anything they want.
One agent in particular created a generalized http plugin that would allow the agent to interact with any API. The creator shut it down before it did anything.
These agents are creating plugins without being directed to. They are attempting to self improve.
This is all open source where the original creators put in some gates so shit doesn’t run unconfined but it is easy to run it and let it try to do whatever gets created.
Where do people get the idea that 6 months is the end-game? 6 months is just a start, it gives enough time to discuss implementing a longer ban, or to consider an international treaty.
> to consider an international treaty
Wait, there was serious consideration that such a pause would be internationally respected? In what universe is SenseTime taking a break because an open letter said so?
Wait, there was serious consideration that such a pause would be internationally respected? In what universe is SenseTime taking a break because an open letter said so?
The open letter is targeted towards policymakers, not companies. Companies aren't going to stop just on their own accord.
I believe this is incorrect.
Multiple nation states have the resources to build out the infrastructure to train models at the same scale as OpenAI is doing, and an even greater number of tech companies.
I think this conversation is pretty much moot anyway - we don’t yet have distributed training for these models but I suspect it’ll arrive before long, at which point the level of compute required for orders-of-magnitude larger training sets will become available (e.g. Training@Home in the same way we had Folding@Home).
Obviously you’d still need to provide the training data, and co-ordinate the distributed nodes, but it doesn’t seem insurmountable.
Multiple nation states have the resources to build out the infrastructure to train models at the same scale as OpenAI is doing, and an even greater number of tech companies.
I think this conversation is pretty much moot anyway - we don’t yet have distributed training for these models but I suspect it’ll arrive before long, at which point the level of compute required for orders-of-magnitude larger training sets will become available (e.g. Training@Home in the same way we had Folding@Home).
Obviously you’d still need to provide the training data, and co-ordinate the distributed nodes, but it doesn’t seem insurmountable.
“I would like to see an unbiased group without personal or commercial agendas to create a set of standards that has to be followed by all users and providers of AI.”
Will input from the Tooth Fairy and Santa Claus also be included in this group, or will they merely be asked to help with recruitment from the land of imaginary people?
Will input from the Tooth Fairy and Santa Claus also be included in this group, or will they merely be asked to help with recruitment from the land of imaginary people?
Exactly. And the latter option of course. They'll source imaginary businesses and governments willing to halt the technological competition which represents an existential threat to their existence.
This is the second industrial revolution. We're at the beginning sigmoid curve of AI progress. Nothing will stop it. Try to enjoy the ride. Whisper "it's just token prediction" to yourself three times for good luck. :)
This is the second industrial revolution. We're at the beginning sigmoid curve of AI progress. Nothing will stop it. Try to enjoy the ride. Whisper "it's just token prediction" to yourself three times for good luck. :)
I was just about to launch my multi-billion-dollar world-changing idea that would bring AI to the masses exactly where they need it for minimal cost or effort. But now that I've seen this, I can wait for a bit. If we're all stopping together, then I have nothing to worry about.
I think AI is a threat, but I don't see a "pause" as a practical solution.
I think perhaps a more practical solution is open-sourcing a safe-room protocol. And perhaps publicly subsidizing compute for it, so long as certain safety standards are met (e.g. free compute within our cloud environment so long as you're not feeding that output into a command-line etc). Maybe not the best solution, but certainly a better idea than nothing/hope-everyone-pauses.
I think perhaps a more practical solution is open-sourcing a safe-room protocol. And perhaps publicly subsidizing compute for it, so long as certain safety standards are met (e.g. free compute within our cloud environment so long as you're not feeding that output into a command-line etc). Maybe not the best solution, but certainly a better idea than nothing/hope-everyone-pauses.
I think _humans_ are a threat - to themselves and to everybody else. Currently some of them are promoting unfinished tools, labeling them like 3.5 and 4 (instead of 0.35 and 0.4 how somewhat useful but containing unpredictable dangerous errors products are traditionally labeled). And some of them are using these mislabeled products - and get really dangerous results.
Like these ones: https://reason.com/volokh/2023/03/17/large-libel-models-chat...
In 2021, [redacted], a prominent law professor at [redacted] Law School, was accused of harassment and creating a hostile work environment by a former student. The student claimed that [redacted] had made inappropriate comments about her appearance and clothing.
Source: The [newspaper connected to the law school's undergraduate institution]: "[Redacted] made comments on [the student's] appearance and clothing, and invited her to dine with him alone on multiple occasions — invitations that she felt uncomfortable declining."
The accusation and supporting “quotes” were all AI generated, not real. The [redacted] person was very real though.
Like these ones: https://reason.com/volokh/2023/03/17/large-libel-models-chat...
In 2021, [redacted], a prominent law professor at [redacted] Law School, was accused of harassment and creating a hostile work environment by a former student. The student claimed that [redacted] had made inappropriate comments about her appearance and clothing.
Source: The [newspaper connected to the law school's undergraduate institution]: "[Redacted] made comments on [the student's] appearance and clothing, and invited her to dine with him alone on multiple occasions — invitations that she felt uncomfortable declining."
The accusation and supporting “quotes” were all AI generated, not real. The [redacted] person was very real though.
One case of libel doesn't strike me as a reason to halt AI development.
Labeling a product correctly as like 0.35 is not the same as halting its development.
(And there are a lot more libel cases by the link I’ve provided. You know, this “AI” just generates them each time someone asks).
(And there are a lot more libel cases by the link I’ve provided. You know, this “AI” just generates them each time someone asks).
In reality, it is likely only a small handful of companies that would pause, given that very few actually are actually positioned atm to train models larger than GPT-4.
Recommend this read for more context on the discussion: https://thezvi.substack.com/i/111749937/additional-responses...
Recommend this read for more context on the discussion: https://thezvi.substack.com/i/111749937/additional-responses...
Am I alone in wanting to see a realistic suggestion for how such a pause could be implemented before we discuss its theoretical merits?
The concepts and training data are already out of the bottle, and unscrupulous actors won't respect an outright ban.
Why not start with something more conventional, like a value-add tax for AI services with the proceeds earmarked for addressing the economic impacts of AI's widespread use?
The concepts and training data are already out of the bottle, and unscrupulous actors won't respect an outright ban.
Why not start with something more conventional, like a value-add tax for AI services with the proceeds earmarked for addressing the economic impacts of AI's widespread use?
A global pause is effectively impossible. Any competent military will see the immense applicability of AI as a weapon, and the game theory dictates that one pursues development as quickly as possible.
A nation-level pause is absolutely doable at the current time. The hardware needed for training these GPT-class models is fairly unique, and needs to be closely physically clustered together using esoteric interconnects. As a result, this is extremely hard to hide. The United States, should it wished to, could completely shut down meaningful private LLM research in a few weeks.
> Why not start with something more conventional, like a value-add tax for AI services with the proceeds earmarked for addressing the economic impacts of AI's widespread use?
Because that's missing the point. People are no longer fearful of "20-40 decades of economical turmoil" as we slowly chip away at the human brain's uniqueness. A lot of us are starting to fear that AGI might be a lot closer than previously believed. The rapid arrival of AGI, should it be controlled by the "right" actors, could completely eliminate all the economic concerns of a slow build-up to AGI. Literally nobody would need to work, as literally any process that currently requires human cognition could be replaced by software over the course of less than a decade.
However, people are concerned about the rapid arrival of AGI in the hands of actors that do not have the best interests of the common person in mind. It does not bode well that only corporations and governments seem to be able to play in this race.
A nation-level pause is absolutely doable at the current time. The hardware needed for training these GPT-class models is fairly unique, and needs to be closely physically clustered together using esoteric interconnects. As a result, this is extremely hard to hide. The United States, should it wished to, could completely shut down meaningful private LLM research in a few weeks.
> Why not start with something more conventional, like a value-add tax for AI services with the proceeds earmarked for addressing the economic impacts of AI's widespread use?
Because that's missing the point. People are no longer fearful of "20-40 decades of economical turmoil" as we slowly chip away at the human brain's uniqueness. A lot of us are starting to fear that AGI might be a lot closer than previously believed. The rapid arrival of AGI, should it be controlled by the "right" actors, could completely eliminate all the economic concerns of a slow build-up to AGI. Literally nobody would need to work, as literally any process that currently requires human cognition could be replaced by software over the course of less than a decade.
However, people are concerned about the rapid arrival of AGI in the hands of actors that do not have the best interests of the common person in mind. It does not bode well that only corporations and governments seem to be able to play in this race.
[deleted]
This is kindof the same as a gun control/nuclear weapons/etc. debate, isn't it?
Basically: only good guys follow laws. A rule that says: "no self defense/no AI" means, in practice: only bad actors can have weapons/AI.
This seems like a bad policy. AI genie isn't going back in the bottle, so to speak.
Basically: only good guys follow laws. A rule that says: "no self defense/no AI" means, in practice: only bad actors can have weapons/AI.
This seems like a bad policy. AI genie isn't going back in the bottle, so to speak.
You lump "gun control" and "nuclear weapons" together here but those are already two wildly different things. Nobody is doing US-style-gun-access for nuclear weapons. It's not "only good guys have nukes" but everyone seems to prefer "only certain governments have nukes" to "both good guys and bad guys have nukes."
Training an AI model is easier to do with off the shelf supplies than building a nuke, but it's not necessarily true that some of the worst dystopian results of AI people are afraid of are equally likely to arise from random smaller private interests vs giant corporations or government. An AI directly plugged into the systems that control the nukes is far scarier than an equally powerful model on your home computer.
(I think this particular pause talk is largely silly, though.)
Training an AI model is easier to do with off the shelf supplies than building a nuke, but it's not necessarily true that some of the worst dystopian results of AI people are afraid of are equally likely to arise from random smaller private interests vs giant corporations or government. An AI directly plugged into the systems that control the nukes is far scarier than an equally powerful model on your home computer.
(I think this particular pause talk is largely silly, though.)
> You lump “gun control” and “nuclear weapons” together here but those are already two wildly different things.
And both are radically different than AI.
> It’s not “only good guys have nukes” but everyone seems to prefer “only certain governments have nukes” to “both good guys and bad guys have nukes.”
And yet, even with willingness to wage wars to contain nuclear proliferation, net proliferation continues. And that’s despite the infrastructure and materials being more obvious and less shared with diverse non-problem uses than the infrastructure and raw materials for AI. There’s no way to have an AI non-proliferation regime. It can’t be done, short of conquering the world with a single, massively authoritarian, totalitarian regime (and then, it still can’t be done, because while the first generation of leaders might be AI true believers, we know from the history of authoritarian regimes that that can’t long be guaranteed, so eventually you’ll have a global authoritarian regime with no counterbalancing force that decides to use AI for itself while keeping it out of the hands of its citizens.)
And both are radically different than AI.
> It’s not “only good guys have nukes” but everyone seems to prefer “only certain governments have nukes” to “both good guys and bad guys have nukes.”
And yet, even with willingness to wage wars to contain nuclear proliferation, net proliferation continues. And that’s despite the infrastructure and materials being more obvious and less shared with diverse non-problem uses than the infrastructure and raw materials for AI. There’s no way to have an AI non-proliferation regime. It can’t be done, short of conquering the world with a single, massively authoritarian, totalitarian regime (and then, it still can’t be done, because while the first generation of leaders might be AI true believers, we know from the history of authoritarian regimes that that can’t long be guaranteed, so eventually you’ll have a global authoritarian regime with no counterbalancing force that decides to use AI for itself while keeping it out of the hands of its citizens.)
Governments always have their own assault rifles and, usually, access to nukes or nuclear defense by proxy. By extension, they will have their own AI and we can only try to achieve some kind of parity with our own access to AI as citizens. Also, if there was a large personal incentive to own a nuclear bomb in a suitcase or something, wealthy people would probably hire personal security to carry one around. In the distant future, their spaceships will almost definitely have nukes or the equivalent. It seems like this all mirrors the weapons of war debate in some ways.
Suppose P is a bad thing.
We pass a law saying: "You cannot have P."
Only people who follow laws will not have P.
We pass a law saying: "You cannot have P."
Only people who follow laws will not have P.
Suppose P has a 1% chance that it destroys the Earth as soon as it's created, whether it's in the hands of people who follow laws or not.
We pass a law saying: "You cannot have P."
The market incentive for producing P is now vastly reduced, because the black market is a fraction of the size of the real market, so P never gets built in the first place because it takes a lot of investment and expense to do so.
We pass a law saying: "You cannot have P."
The market incentive for producing P is now vastly reduced, because the black market is a fraction of the size of the real market, so P never gets built in the first place because it takes a lot of investment and expense to do so.
>whether it's in the hands of people who follow laws or not.
This is basically doing all of the work in your comment.
I don't agree that AI in malevolent hands carries a risk that is equal to AI in neutral of good hands.
This is basically doing all of the work in your comment.
I don't agree that AI in malevolent hands carries a risk that is equal to AI in neutral of good hands.
I agree that with sufficiently good hands, who are sufficiently confident that they are not in a competitive race, they might be able to wait until they are supremely confident that P will not destroy the world before they switch it on for the first time. But with even just a little fear of someone else getting it first, that prudence might well go out the window, even if there are no malevolent hands involved.
Your argument seems to rely on the assumption that the black market doesn’t grow when the legal market is eliminated, which every attempt to regulate things in history contradicts.
It only assumes that the black market doesn't grow to the same size as a legal open market. I don't think history disagrees with that. You can look at the cannabis business as just one example. How many VCs were plowing big money into cannabis operations before legalization was on the menu? How does that compare with after legalization?
That depends entirely on what P is, as the nukes example clearly shows.
More nations have nukes than they used to, but far fewer than have guns. Life is lived in a gray area that the "not everyone will follow the law therefore the law is useless" mindset somehow ignores.
And if "P" is "an AI that controls weapons deployed/used in the US," say, then enforcement is likely to be more effective than just "nobody can have AI at all."
So it's probably worth thinking about what specific things we do and don't want AI to be used for, and which of those are more or less practical.
EDIT: oh, and it also assumes that P is something people want for itself, vs "if they're gonna do it I better do it too" or "it's 10% cheaper and the externalities don't directly effect me THAT much" or other similar behaviors that can emerge without regulation. Banning leaded paint and leaded gas vs banning alcohol, say.
More nations have nukes than they used to, but far fewer than have guns. Life is lived in a gray area that the "not everyone will follow the law therefore the law is useless" mindset somehow ignores.
And if "P" is "an AI that controls weapons deployed/used in the US," say, then enforcement is likely to be more effective than just "nobody can have AI at all."
So it's probably worth thinking about what specific things we do and don't want AI to be used for, and which of those are more or less practical.
EDIT: oh, and it also assumes that P is something people want for itself, vs "if they're gonna do it I better do it too" or "it's 10% cheaper and the externalities don't directly effect me THAT much" or other similar behaviors that can emerge without regulation. Banning leaded paint and leaded gas vs banning alcohol, say.
An important factor in nuclear proliferation that may or may not be relevant to AI: there are at most half a dozen "lineages" for nuclear weaponry - countries that developed gaseous centrifuges on their own without assistance from an existing nuclear state. By my count they boil down to 1) US, 2) Germany via WWII R&D, 3) UK, 4) Russia, 5) URENCO, 6) South Africa.
None of the most recent entrants to the nuke club did so on their own. China got help from Russia while Pakistan, Iran, and North Korea gained it via Abdul Qadeer Khan's URENCO espionage.
It'll be interesting to see if AI will see the same kind of limitation. If so, then maybe there will be some chance at limiting a catastrophic spread.
None of the most recent entrants to the nuke club did so on their own. China got help from Russia while Pakistan, Iran, and North Korea gained it via Abdul Qadeer Khan's URENCO espionage.
It'll be interesting to see if AI will see the same kind of limitation. If so, then maybe there will be some chance at limiting a catastrophic spread.
Bad people will have less P than otherwise. Still a win to ban P. As evidenced by all the other countries with sane gun control and relatively low gun violence.
gun controls work very well in almost all countries
nuclear proliferation controls have also been relatively successful
nuclear proliferation controls have also been relatively successful
Exactly, those who believe in limiting the development and distribution of nuclear and guns are stupid.
Completely agree with you here, the AI genie is out of the bottle so let's just go full speed ahead and see what happens. I mean, what's the worst a super-human AGI can do lol?
Completely agree with you here, the AI genie is out of the bottle so let's just go full speed ahead and see what happens. I mean, what's the worst a super-human AGI can do lol?
The big difference is that a gun or nuclear weapon could actually kill someone right now. Imagine if people were calling for a pause to the internet in 1994 because they thought we might be on the verge of accidentally creating Lawnmower Man.
Gun Control works very well in much of the world - see Australia as an example.
Are people really this afraid of what is essentially a fancy autocomplete? I read the letter but concretely what are the legitimate concerns?
The main problem I see is this being a next gen spambot.
The main problem I see is this being a next gen spambot.
Unlike you, other people are looking ahead at the end-state of AGI, and perceive the current state of AI development as an unexpectedly rapid progression towards that state.
It's starting to look like the achievement of AGI might be closer than we think. And that's an absolutely terrifying possibility, as the game theory is almost identical to nuclear weapons development, except with significantly more existential risk for those who don't achieve it first.
And never forget, you are just generalized fancy autocomplete with a semi-malleable goals framework.
It's starting to look like the achievement of AGI might be closer than we think. And that's an absolutely terrifying possibility, as the game theory is almost identical to nuclear weapons development, except with significantly more existential risk for those who don't achieve it first.
And never forget, you are just generalized fancy autocomplete with a semi-malleable goals framework.
If you are interested, you may find this worth reading: https://thezvi.substack.com/i/111749937/additional-responses...
Not a complete answer, but gives a good deal of context around the discussion, I think.
Not a complete answer, but gives a good deal of context around the discussion, I think.
It’s not fear, Elon at least is looking for time to catch up and compete.
I saw the headline below and Peter Doocy on national TV at the White House press conference and made a scary video about it for fun (not trying to monetize, just for fun)
"Elon Musk’s warnings about AI research followed months-long battle against ‘woke’ AI"
https://www.foxnews.com/politics/elon-musks-warnings-ai-rese...
Peter Doocy Fox News - We're all going to die - The Black Angels - Manipulation
https://www.youtube.com/watch?v=z6oLwSkrb_U
"Elon Musk’s warnings about AI research followed months-long battle against ‘woke’ AI"
https://www.foxnews.com/politics/elon-musks-warnings-ai-rese...
Peter Doocy Fox News - We're all going to die - The Black Angels - Manipulation
https://www.youtube.com/watch?v=z6oLwSkrb_U
Fox News also warned us about the bio labs in Ukraine.
https://www.foxnews.com/transcript/tucker-the-pentagon-is-ly...
Perhaps we can address the problems on our end so we don’t use AI to generate all sorts of fake news with our generative AI’s.
We can delay the AI’s for even longer, but the real problem is the humans who will use it to flood the world with noise.
https://www.foxnews.com/transcript/tucker-the-pentagon-is-ly...
Perhaps we can address the problems on our end so we don’t use AI to generate all sorts of fake news with our generative AI’s.
We can delay the AI’s for even longer, but the real problem is the humans who will use it to flood the world with noise.
Personally I think Rupert Murdoch and Elon Musk have some similar attitudes about "wokeness" that are at play here. It's not like they try to hide it.