Open-Source AI Is Uniquely Dangerous(spectrum.ieee.org)
spectrum.ieee.org
Open-Source AI Is Uniquely Dangerous
https://spectrum.ieee.org/open-source-ai-2666932122
34 comments
If you need AI model to tell you how to make a bomb you are incapable of securing the materials and building one and now people want to “gatekeep” biological and chemical weapons as if the price of entry wasn’t already excessive to anyone who would have otherwise used these
> "Unsecured AI also has the potential to facilitate production of dangerous materials, such as biological and chemical weapons."
> "You could ask them to design a more deadly coronavirus, provide instructions for making a bomb [...]
Moreover, I call total bullshit.
I'd like for these people to show me how an LLM can instruct them to make a better virus.
Where's the hypothetical open source LLM that understands viral genomics? At the level of a PhD student?
Where?
Please show me. I'm waiting.
> "You could ask them to design a more deadly coronavirus, provide instructions for making a bomb [...]
Moreover, I call total bullshit.
I'd like for these people to show me how an LLM can instruct them to make a better virus.
Where's the hypothetical open source LLM that understands viral genomics? At the level of a PhD student?
Where?
Please show me. I'm waiting.
Isn't the point here the potential risk? By definition, nobody can point to a textbook instantiation of the risk because it hasn't happened yet, and the question is whether and when it might in the future.
It strikes me as similar logic to "you're saying I might get injured if I use this chainsaw without protective equipment? Well, show me the injury then!"
It strikes me as similar logic to "you're saying I might get injured if I use this chainsaw without protective equipment? Well, show me the injury then!"
Outside of very specific workplace safety regulations, nobody is trying to regulate chainsaws.
Speaking of potentials, there's also a potential risk of outer space alien invasion, too. Yet our preparation here is appropriately scaled proportional to the risk.
Where is someone going to get a BSL4 lab with the necessary inputs to follow through with these wacky James Bond plots?
Maybe raise the alarm when splicing viral genes becomes a regular household occurrence. (And you don't even need to invoke the specter of LLMs for this scenario.)
Speaking of potentials, there's also a potential risk of outer space alien invasion, too. Yet our preparation here is appropriately scaled proportional to the risk.
Where is someone going to get a BSL4 lab with the necessary inputs to follow through with these wacky James Bond plots?
Maybe raise the alarm when splicing viral genes becomes a regular household occurrence. (And you don't even need to invoke the specter of LLMs for this scenario.)
This is one of the more unbelievable posts I've seen. Once again the argument is that the common person is too dumb and needs to be protected from big bad open source. Really kind of sad that someone has this position.
This kind of post is typical of people who don't really understand what language models do... it's the classic strawman, they make up a caricature of the thing they want to argue against and then push that over instead of actually engaging with what the technology really is. It's also a stereotype of the out of touch "AI safety" person.
Maybe the broader point is that whenever a new technology empowers people, entrenched players get upset and it's always possible to find a mouthpiece who will support the view that it's too dangerous for the masses. It happened with the printing press, encryption, etc.
Encryption and the history around the Clipper chip is an excellent analog. “Encryption is dangerous in public hands. Let us be the only licensed provider of public encryption and we’ll backdoor it to make it safe.”
Well if we're all just spitballing here then what about the uniquely dangerous outcome of having the ruling class enslave the world's population in a techno-feudalistic surveillance state utilizing the AI that only they are allowed to own and use? If people thought overthrowing a cruel king was hard imagine if they have cameras everywhere and an army of killbots that can swarm any position in under 5 minutes. Which they might use even if their systems detect your potential crime coefficient gets too high, even if you haven't committed any crimes.
Or more mundane you're only paid for the seconds the AI detects that you're "really" working, of which there isn't much after most of it got automated to serve the wealthy who have escaped to their fallout shelters and private yachts to remotely control their empires and leave everyone else to starve to death from the climate collapse that wasn't averted.
Or more mundane you're only paid for the seconds the AI detects that you're "really" working, of which there isn't much after most of it got automated to serve the wealthy who have escaped to their fallout shelters and private yachts to remotely control their empires and leave everyone else to starve to death from the climate collapse that wasn't averted.
All of our errors with "AI" will be the fault of our own foolishness.
https://snafuhall.com/p/aioracle.html
https://snafuhall.com/p/aioracle.html
The list of recommendations at the end starts with a moratorium on open source AI until the following controls are implemented: you’re not allowed to release a model without a government license, you have to release your training data, and you’re liable for misuse of your model.
It also starts with a fawning review of the benefits of closed models only accessible via API.
David is a former Meta employee and is now pushing hard for global AI regulation.
Gatekeeping public access and innovation in AI leaves us with a world subjugated to powerful corporate interests and their lobby groups. We won’t even have the opportunity to teach AI or conduct lab experiments without a government license, being forced to release training data, and we’ll have to assume liability which will have a massive chilling effect.
This article asks for a global overreach based on the premise of “Those with wealth, power and influence know best and will keep you safe.”
It also starts with a fawning review of the benefits of closed models only accessible via API.
David is a former Meta employee and is now pushing hard for global AI regulation.
Gatekeeping public access and innovation in AI leaves us with a world subjugated to powerful corporate interests and their lobby groups. We won’t even have the opportunity to teach AI or conduct lab experiments without a government license, being forced to release training data, and we’ll have to assume liability which will have a massive chilling effect.
This article asks for a global overreach based on the premise of “Those with wealth, power and influence know best and will keep you safe.”
I liked reading this article because it is almost 100% THE OPPOSITE of my viewpoint. It is healthy reading well written and well thought out arguments contrary to our own beliefs.
Where I think the author goes very much wrong is not in pointing out the dangers of open models, but rather ignoring the huge positive potential benefits to society of open systems.
I think Mistral is one example of thoughtful release of open model weights. Mistral-7B and mixtral-8-7B may be black boxes, but they are also powerful and useful engineering tools for building a huge range of useful and interesting applications without relying on big tech. Obviously, Hugging Face, Meta, and other companies are also making important contributions.
I think it is important to evaluate potential gains along with potential risks.
Where I think the author goes very much wrong is not in pointing out the dangers of open models, but rather ignoring the huge positive potential benefits to society of open systems.
I think Mistral is one example of thoughtful release of open model weights. Mistral-7B and mixtral-8-7B may be black boxes, but they are also powerful and useful engineering tools for building a huge range of useful and interesting applications without relying on big tech. Obviously, Hugging Face, Meta, and other companies are also making important contributions.
I think it is important to evaluate potential gains along with potential risks.
> You could ask them to design a more deadly coronavirus
If I'm a shadowy org with a lab of the required genetic engineering equipment and a team of highly knowledgable virologists, is the make-or-break factor really going to be whether I can access a local LLM trained only on the public Internet?
I suspect that it's this kind of group that may get secret military contracts for unfiltered access to powerful closed-source LLMs regardless of what the public gets. Or the budget to train a model better suited for their application.
If I'm a shadowy org with a lab of the required genetic engineering equipment and a team of highly knowledgable virologists, is the make-or-break factor really going to be whether I can access a local LLM trained only on the public Internet?
I suspect that it's this kind of group that may get secret military contracts for unfiltered access to powerful closed-source LLMs regardless of what the public gets. Or the budget to train a model better suited for their application.
"A good first step in understanding the threats posed by unsecured AI is to ask secured AI systems like ChatGPT, Bard, or Claude to misbehave. You could ask them to design a more deadly coronavirus, provide instructions for making a bomb, make naked pictures of your favorite actor, or write a series of inflammatory text messages designed to make voters in swing states more angry about immigration."
And again - it's a certainty that governments will create enormous systems populated with all of the data that Snowden warned about and much more, with absolutely no constraints whatever, with the resources to actually do something with the answers, unlike almost everyone using an open source system. If anything there desperately needs to be unrestricted systems capable of acting defensively against such actors. It'll likely be the only way to do so.
And again - it's a certainty that governments will create enormous systems populated with all of the data that Snowden warned about and much more, with absolutely no constraints whatever, with the resources to actually do something with the answers, unlike almost everyone using an open source system. If anything there desperately needs to be unrestricted systems capable of acting defensively against such actors. It'll likely be the only way to do so.
These things are just hugely glorified math functions. There is no need to create fear of them and regulate them on their own. But the decision made with or without them should be governed in light of existing laws.
I’ve been studying and tinkering with open weight LLMs since the original llama weights leaked. I’ve very recently become convinced that the true data and compute requirements needed to fine tune and produce an “unsafe” model are orders of magnitude less than what’s needed today. We are no more than a year away from anyone with a 4090 being able to fine tune their own mistral. The cat is out of the bag on this one.
So they want to stifle AI from normal people. All these restrictions are something only large corporations can adhere to, meaning any hobbyist stuff is immediately dead and no new competitors can emerge.
Bad actors of course will still have their “unsecure” models and those will just be couple years behind and sold on the dark web with other illicit materials
Bad actors of course will still have their “unsecure” models and those will just be couple years behind and sold on the dark web with other illicit materials
The whole narrative of "Open source = bad" is totally f****d up and absurd. I'm sick of nobodies being portrayed as high-IQ all-knowing tech overlords. F**k them. In the grand scheme of things, they will be remembered as great enemies of human progress or better - completely forgotten.
I mean David Evan Harris from Business Insider’s AI 100 list, what a laugh. The guy is nobody, his opinion is of ZERO technical relevance.
LinkedIn: University of California, Berkeley, BA, Political Economy of Development & Environment Universidade de São PauloUSP, M.S., Sociology Civic Integrity Researcher at FB - what does that even mean!? "As Research Manager of the Responsible AI team, I led a team of quantitative and qualitative researchers focused on AI fairness, inclusion, governance and accountability." - Jesus Christ, if I had a company and someone with those words in their resume appeared at my door, help me god, I would SteveJobs them with the chair or monitor or both.
Please.
The guy is of the professional fog-selling NGO managerial class, that has almost eaten up our SF.
Where are the voices of the people in the trenches of ML algorithms research, CUDA Toolkit optimisation, Semiconductor/material sciences research etc etc - the ones that make a real difference?
I mean David Evan Harris from Business Insider’s AI 100 list, what a laugh. The guy is nobody, his opinion is of ZERO technical relevance.
LinkedIn: University of California, Berkeley, BA, Political Economy of Development & Environment Universidade de São PauloUSP, M.S., Sociology Civic Integrity Researcher at FB - what does that even mean!? "As Research Manager of the Responsible AI team, I led a team of quantitative and qualitative researchers focused on AI fairness, inclusion, governance and accountability." - Jesus Christ, if I had a company and someone with those words in their resume appeared at my door, help me god, I would SteveJobs them with the chair or monitor or both.
Please.
The guy is of the professional fog-selling NGO managerial class, that has almost eaten up our SF.
Where are the voices of the people in the trenches of ML algorithms research, CUDA Toolkit optimisation, Semiconductor/material sciences research etc etc - the ones that make a real difference?
Why would open-source AI be any different from any other AI? AI is AI. Being opensource does not make it better or worst. Is being built by people, and being used by people.
AI is a force multiplier. The more people with access to it, the better a chance that someone will do a lot of damage, by choice or by accident.
Because you would be able to use the FLOSS AI, whereas currently only "trusted" companies like Microsoft can.
There are lots of things you can do with weights which you can't do with API access.
I would be more scared of military, political and big corporations AIs, that they have the (computing/development) power and the means to be really dangerous, both because it will the will of the ones working with them, or because their "owners" could eventually be manipulated (in the case of some godlike AI, at least).
And they won't be regulated, almost by definition. Having a more powerful AI than everyone else is an strategic advantage.
And they won't be regulated, almost by definition. Having a more powerful AI than everyone else is an strategic advantage.
Recent discussion: https://news.ycombinator.com/item?id=38977366
David Evan Harris, the author from a prominent UC Berkeley advisory position .. " previously worked as a research manager at Meta (formerly Facebook) on the responsible AI, civic integrity, and social impact teams, and was recently named to Business Insider’s AI 100 list for his work on AI governance, fairness, and misinformation."
There are new winners and new losers, and the politics are happening in real time, right now.
Everything is OK if OpenAI / Meta runs it? whose safety?
There are new winners and new losers, and the politics are happening in real time, right now.
Everything is OK if OpenAI / Meta runs it? whose safety?
It was Meta that released llama -- a move which Harris specifically cites as irresponsible and dangerous. So he's certainly not writing in support of them -- rather the opposite.
There are plenty of reasons to disagree (Steve Bannon's ilk didn't need AI to "flood the zone with shit", they used Meta to do it, and it's really not obvious that AI increases the threat), but he's not writing propaganda for Meta.
There are plenty of reasons to disagree (Steve Bannon's ilk didn't need AI to "flood the zone with shit", they used Meta to do it, and it's really not obvious that AI increases the threat), but he's not writing propaganda for Meta.
Full bio from the site:
>David Evan Harris is a chancellor's public scholar at UC Berkeley, senior research fellow at the International Computer Science Institute, senior fellow at the Centre for International Governance Innovation, senior advisor for AI and elections at the Brennan Center for Justice, and visiting fellow at the Integrity Institute. He previously worked as a research manager at Meta (formerly Facebook) on the responsible AI, civic integrity, and social impact teams, and was recently named to Business Insider’s AI 100 list for his work on AI governance, fairness, and misinformation.
>David Evan Harris is a chancellor's public scholar at UC Berkeley, senior research fellow at the International Computer Science Institute, senior fellow at the Centre for International Governance Innovation, senior advisor for AI and elections at the Brennan Center for Justice, and visiting fellow at the Integrity Institute. He previously worked as a research manager at Meta (formerly Facebook) on the responsible AI, civic integrity, and social impact teams, and was recently named to Business Insider’s AI 100 list for his work on AI governance, fairness, and misinformation.
Are the guns sold at Walmart uniquely dangerous?
Ah yes, information is just too dangerous to be put in the hands of us common folk. It's a really disappointing thesis.
Never mind that the resources required to produce massively dangerous things are usually on the scale of nation-state budgets, we can't have a McDonald's fry cook learn how to genetically engineer viruses.
I feel like a vast majority of articles like this are written by people who read too much sci-fi. They think LLM's can suddenly crack the laws of the universe open and make anything possible.
Never mind that the resources required to produce massively dangerous things are usually on the scale of nation-state budgets, we can't have a McDonald's fry cook learn how to genetically engineer viruses.
I feel like a vast majority of articles like this are written by people who read too much sci-fi. They think LLM's can suddenly crack the laws of the universe open and make anything possible.
I think that we already have a good chunk of them, I doubt there could be worse than the ones we have.
"OpenAI worried because these text-generating AI systems can be used to generate massive amounts of well-written but misleading or toxic content."
This is solved by prestige, exactly what normative media had until some years ago, when they decided to throw everything away and just lick politics asses or just sell their integrity for money.
"You could ask them to design a more deadly coronavirus, provide instructions for making a bomb, make naked pictures of your favorite actor, or write a series of inflammatory text messages designed to make voters in swing states more angry about immigration. "
The coronavirus sentence is hilarious, and you don't need AI for any of that.
I'm kinda sick, what they mean with "regulating AI" is that they just want to control what the AI spits out to match their political agenda and beliefs, slowly impregnating the users of these AI of their political ideas.
Obviously, it's me and you who will not have access to uncensored models, big corps and the gov will have access to whatever they want unrestricted, and most hilarious, they WILL use the AI to "swing political beliefs".