Sam Altman isn’t the answer to regulating artificial intelligence(bloomberg.com)
bloomberg.com
Sam Altman isn’t the answer to regulating artificial intelligence
https://www.bloomberg.com/opinion/articles/2023-05-17/ai-regulation-openai-s-sam-altman-is-the-wrong-person-to-ask
100 comments
Sam Altman is as much the answer to regulating AI as SBF was the answer to regulating crypto.
Yeah. Walking into Congress and agreeing with every barb thrown his way is a red flag to me. OpenAI has made AI fear mongering a central plank in their marketing platform. I think their goal is to convince the government to build a regulatory moat around their business.
Why? Because they see the writing on the wall. There’s no network effect here. Anyone can build AI. There are thousands of students going into grad school to study this stuff. The market is going to be flooded.
All of their head start will evaporate unless they can build a moat around it to keep new entrants out. What better way to do that than to have your rockstar pitch man walk into Congress and have them eating out of his hand? They practically begged him to head up a new regulatory agency!
Why? Because they see the writing on the wall. There’s no network effect here. Anyone can build AI. There are thousands of students going into grad school to study this stuff. The market is going to be flooded.
All of their head start will evaporate unless they can build a moat around it to keep new entrants out. What better way to do that than to have your rockstar pitch man walk into Congress and have them eating out of his hand? They practically begged him to head up a new regulatory agency!
They didn’t practically, they did. There was a specific exchange between Sam and members of the hearing in which a sitting US Senator asked him if he wanted the job. Sam replied no, he likes his job, but suggested they could recommend people. The Senator appeared happy with this.
I mostly agree that there is probably some self-interest at play when they ask for regulation. Though the fact that one of the existing firms in the industry wants regulation doesn't invalidate all the legitimate reason for regulation. For example, existing car manufacturers might benefit significantly from all the safety requirements for cars as they create a barrier to entry for new players, but I doubt that many people want to abolish all the testing and certification cars have to go through.
Arguably OpenAI's moat is the heavily subsidized Microsoft Azure infrastructure to train its models and serve traffic, but your point still stands: government regulation will be a more reliable moat. Especially as the latest LLM models (e.g. LLaMA) are shrinking in size while maintaining performance parity.
I wonder if government will end up enacting any legislation or this scrutiny will blow over until something egregious happens (e.g. Cambridge Analytica). They've shown zero willingness to legislate any consumer data privacy laws so far.
I wonder if government will end up enacting any legislation or this scrutiny will blow over until something egregious happens (e.g. Cambridge Analytica). They've shown zero willingness to legislate any consumer data privacy laws so far.
Isn't it also possible that Sam is being genuine about his concerns and his statements because he wants what is best for humanity?
It may be so, but given the switcheroo that happened with "Open" AI, he and the entire company is untrustworthy on this particular front.
When OpenAI started I don't think anyone would have thought that AI this powerful was around the corner.
This is just more openAI marketing at work. ChatGPT is a chatbot, and it’s good at language for sure, but it isn’t some huge breakthrough that people were expecting would take a century. Its literally just a fancy RNN which we’ve had for awhile
No offense but this comment just shows a lack of understanding of the development that's happened with transformers.
"Fancy RNN" is a pretty ridiculous assertion.
And no, researchers didn't expect what GPT-3/4 has been shown to do to be around the corner at all.
GPT's aren't chatbots. That's just a neat natural consequence that's happened. They're machines that reason, understand and follow instructions in plain language. And their abilities go far beyond what any expected language modelling to provide.
"Fancy RNN" is a pretty ridiculous assertion.
And no, researchers didn't expect what GPT-3/4 has been shown to do to be around the corner at all.
GPT's aren't chatbots. That's just a neat natural consequence that's happened. They're machines that reason, understand and follow instructions in plain language. And their abilities go far beyond what any expected language modelling to provide.
Nah not really, there’s just a lot of hyperbole going around right now. They are RNNs, the difference is they have a layer/layers being fed back instead of a single neuron. That’s a fancy recurrent network.
They are not machines that reason, they are approximators. It’s all just token matching based on data we’ve fed it. Further it didn’t happen over night but through successive improvements and at no point was the next improvement considered some infeasible thing.
Seems to me maybe you’ve bought into the hype here, and are confusing that with reality.
They are not machines that reason, they are approximators. It’s all just token matching based on data we’ve fed it. Further it didn’t happen over night but through successive improvements and at no point was the next improvement considered some infeasible thing.
Seems to me maybe you’ve bought into the hype here, and are confusing that with reality.
>They are not machines that reason, they are approximators.
And meaningless distinction of the year award goes to..
"It's not real [insert property]" is not an intelligent argument. By all means, divine the way to distinguish results of the two. After all, what kind of important distinction can't be distinguished by results?
>Further it didn’t happen over night but through successive improvements
The only difference between GPT-3 and GPT-2 was scale. They didn't even change the tokenizer until 4. There were no "successful improvements" to smoothen the massive gap in capabilities between the two. So to say that was expected just shows more lack of knowledge here.
And meaningless distinction of the year award goes to..
"It's not real [insert property]" is not an intelligent argument. By all means, divine the way to distinguish results of the two. After all, what kind of important distinction can't be distinguished by results?
>Further it didn’t happen over night but through successive improvements
The only difference between GPT-3 and GPT-2 was scale. They didn't even change the tokenizer until 4. There were no "successful improvements" to smoothen the massive gap in capabilities between the two. So to say that was expected just shows more lack of knowledge here.
> The only difference between GPT-3 and GPT-2 was scale. They didn't even change the tokenizer until 4. There were no "successful improvements" to smoothen the massive gap in capabilities between the two.
"Other than that, how was the play, Mrs. Lincoln?"
GPT-2 is two orders of magnitude smaller. In terms of forebrain neuron count this is the difference between a human and an elephant shrew or budgerigar. It was absolutely expected by reasonable people that 2 OOMs of scaling will provide a qualitative jump.
"Other than that, how was the play, Mrs. Lincoln?"
GPT-2 is two orders of magnitude smaller. In terms of forebrain neuron count this is the difference between a human and an elephant shrew or budgerigar. It was absolutely expected by reasonable people that 2 OOMs of scaling will provide a qualitative jump.
>GPT-2 is two orders of magnitude smaller. In terms of forebrain neuron count this is the difference between a human and an elephant shrew or budgerigar. It was absolutely expected by reasonable people that 2 OOMs of scaling will provide a qualitative jump.
When GPT-3 was released, it was by far the largest artificial neural network ever trained. And I mean by far. Now there wasn't any big jump in hardware capabilities to spur this sort of gulf. It wasn't a case of "Oh now we can train a very large model"
So Want to know why there was such a gap ? It's because most researchers assumed the models would overfit the data or display diminishing returns long before 175b.
Brain neurons are not comparable to ann parameters.
When GPT-3 was released, it was by far the largest artificial neural network ever trained. And I mean by far. Now there wasn't any big jump in hardware capabilities to spur this sort of gulf. It wasn't a case of "Oh now we can train a very large model"
So Want to know why there was such a gap ? It's because most researchers assumed the models would overfit the data or display diminishing returns long before 175b.
Brain neurons are not comparable to ann parameters.
You would expect a machine that can reason to improve in its responses as a conversation progresses, instead of hallucinating.
> They are not machines that reason, they are approximators.
They approximate a function that performs reasoning...
They approximate a function that performs reasoning...
Researchers aren't a monolith. Just because Gary Marcus (a complete fraud by the way, look up his "XProp" magitech he sold to Uber) pooh-poohed connectionist AI until ChatGPT came out doesn't mean nobody predicted gains from scaling; certainly many people deep in the know expected things to click. Certainly they expected that the basic trick of associative learning will be cracked. E.g. here's Shane Legg, December 2009 [1]:
> Conclusion: computer power is unlikely to be the issue anymore in terms of AGI being possible. The main question is whether we can find the right algorithms.
> One of the big things influencing me this year has been learning about how much we understand about how the brain works, in particular, how much we know that should be of interest to AGI designers. I won’t get into it all here, but suffice to say that just a brief outline of all this information would be a 20 page journal paper (there is currently a suggestion that I write such a paper next year with some Gatsby Unit neuroscientists, but for the time being I’ve got too many other things to attend to). At a high level what we are seeing in the brain is a fairly sensible looking AGI design. You’ve got hierarchical temporal abstraction formed for perception and action combined with more precise timing motor control, with an underlying system for reinforcement learning. The reinforcement learning system is essentially a type of temporal difference learning though unfortunately at the moment there is evidence in favour of actor-critic, Q-learning and also Sarsa type mechanisms — this picture should clear up in the next year or so. The system contains a long list of features that you might expect to see in a sophisticated reinforcement learner such as pseudo rewards for informative queues, inverse reward computations, uncertainty and environmental change modelling, dual model based and model free modes of operation, things to monitor context, it even seems to have mechanisms that reward the development of conceptual knowledge. When I ask leading experts in the field whether we will understand reinforcement learning in the human brain within ten years, the answer I get back is “yes, in fact we already have a pretty good idea how it works and our knowledge is developing rapidly.”
> I suspect that for the next 5 years, and probably longer, neuroscientists working on understanding cortex aren’t going to be of much use to AGI efforts. My guess is that sometime in the next 10 years developments in deep belief networks, temporal graphical models, liquid computation models, slow feature analysis etc. will produce sufficiently powerful hierarchical temporal generative models to essentially fill the role of cortex within an AGI.
> Right, so my prediction for the last 10 years has been for roughly human level AGI in the year 2025 (though I also predict that sceptics will deny that it’s happened when it does!) This year I’ve tried to come up with something a bit more precise. In doing so what I’ve found is that while my mode is about 2025, my expected value is actually a bit higher at 2028. This is not because I’ve become more pessimistic during the year, rather it’s because this time I’ve tried to quantify my beliefs more systematically and found that the probability I assign between 2030 and 2040 drags the expectation up. Perhaps more useful is my 90% credibility region, which from my current belief distribution comes out at 2018 to 2036.
And here's Rich Sutton's famous Bitter Lesson, a month after GPT-2 [2]:
> We have to learn the bitter lesson that building in how we think we think does not work in the long run. The bitter lesson is based on the historical observations that 1) AI researchers have often tried to build knowledge into their agents, 2) this always helps in the short term, and is personally satisfying to the researcher, but 3) in the long run it plateaus and even inhibits further progress, and 4) breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning. The eventual success is tinged with bitterness, and often incompletely digested, because it is success over a favored, human-centric approach.
> One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning.
Surprise indicates the mismatch of your mental model and reality, not the inherent weirdness of the latter. Both the surprised/alarmed people and people still in denial about the power of LLMs have to revisit their assumptions and ask if they were founded on any credible understanding to begin with.
1. http://www.vetta.org/2009/12/tick-tock-tick-tock-bing/
2. http://www.incompleteideas.net/IncIdeas/BitterLesson.html
> Conclusion: computer power is unlikely to be the issue anymore in terms of AGI being possible. The main question is whether we can find the right algorithms.
> One of the big things influencing me this year has been learning about how much we understand about how the brain works, in particular, how much we know that should be of interest to AGI designers. I won’t get into it all here, but suffice to say that just a brief outline of all this information would be a 20 page journal paper (there is currently a suggestion that I write such a paper next year with some Gatsby Unit neuroscientists, but for the time being I’ve got too many other things to attend to). At a high level what we are seeing in the brain is a fairly sensible looking AGI design. You’ve got hierarchical temporal abstraction formed for perception and action combined with more precise timing motor control, with an underlying system for reinforcement learning. The reinforcement learning system is essentially a type of temporal difference learning though unfortunately at the moment there is evidence in favour of actor-critic, Q-learning and also Sarsa type mechanisms — this picture should clear up in the next year or so. The system contains a long list of features that you might expect to see in a sophisticated reinforcement learner such as pseudo rewards for informative queues, inverse reward computations, uncertainty and environmental change modelling, dual model based and model free modes of operation, things to monitor context, it even seems to have mechanisms that reward the development of conceptual knowledge. When I ask leading experts in the field whether we will understand reinforcement learning in the human brain within ten years, the answer I get back is “yes, in fact we already have a pretty good idea how it works and our knowledge is developing rapidly.”
> I suspect that for the next 5 years, and probably longer, neuroscientists working on understanding cortex aren’t going to be of much use to AGI efforts. My guess is that sometime in the next 10 years developments in deep belief networks, temporal graphical models, liquid computation models, slow feature analysis etc. will produce sufficiently powerful hierarchical temporal generative models to essentially fill the role of cortex within an AGI.
> Right, so my prediction for the last 10 years has been for roughly human level AGI in the year 2025 (though I also predict that sceptics will deny that it’s happened when it does!) This year I’ve tried to come up with something a bit more precise. In doing so what I’ve found is that while my mode is about 2025, my expected value is actually a bit higher at 2028. This is not because I’ve become more pessimistic during the year, rather it’s because this time I’ve tried to quantify my beliefs more systematically and found that the probability I assign between 2030 and 2040 drags the expectation up. Perhaps more useful is my 90% credibility region, which from my current belief distribution comes out at 2018 to 2036.
And here's Rich Sutton's famous Bitter Lesson, a month after GPT-2 [2]:
> We have to learn the bitter lesson that building in how we think we think does not work in the long run. The bitter lesson is based on the historical observations that 1) AI researchers have often tried to build knowledge into their agents, 2) this always helps in the short term, and is personally satisfying to the researcher, but 3) in the long run it plateaus and even inhibits further progress, and 4) breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning. The eventual success is tinged with bitterness, and often incompletely digested, because it is success over a favored, human-centric approach.
> One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning.
Surprise indicates the mismatch of your mental model and reality, not the inherent weirdness of the latter. Both the surprised/alarmed people and people still in denial about the power of LLMs have to revisit their assumptions and ask if they were founded on any credible understanding to begin with.
1. http://www.vetta.org/2009/12/tick-tock-tick-tock-bing/
2. http://www.incompleteideas.net/IncIdeas/BitterLesson.html
Sure researchers aren't a monolith. Not sure why you brought up Gary Marcus though. Anyway, there's lots more quotes to indicate surprise than the opposite.
When GPT-3 was released, it was by far by the largest artificial neural network ever trained and not because of any big jump in hardware technology. That's not the usual state of affairs for technology everyone or even most expect to pan out the way it did.
When GPT-3 was released, it was by far by the largest artificial neural network ever trained and not because of any big jump in hardware technology. That's not the usual state of affairs for technology everyone or even most expect to pan out the way it did.
There’s an interview between Musk and Tucker Carlson where he says he knew Google had these powerful models internally and thought they should be out in the open, so the answer was a non-profit called Open AI, it was the whole premise of the org
I believe he disagreed with the direction the board was taking it and tried to take over, which when failed he stopped funding it, and they went to investors
I believe he disagreed with the direction the board was taking it and tried to take over, which when failed he stopped funding it, and they went to investors
No. In situations like this the best you can hope for is that his incentives aren't at odds with the rest of humanity.
Wouldn't it be an almost comical level of supervilliany for him to believe that agi is coming and to only care about his company profitting? It seems pretty plausible that agi terrifies him, that he knows it can be done by a lot of different players, and that he wants it to happen as gracefully as possible. If I put myself in his shoes, I would be terrified and asking for some guardrails too. He seems musk like where he has enough money to not care about making more money.
If you do work in AI you're not worried about AGI, period. It's fine that the average layman feels like they're in Terminator but anyone who's written and trained models knows that it's just silly. We shouldn't really expect Sam Altman to do any better than perpetuating fear and hype because of the amount of money on the line.
It's nice to think that you can have so much money you stop caring about it but it doesn't really make sense. You don't get to personally keep billions of dollars without caring about money.
It's nice to think that you can have so much money you stop caring about it but it doesn't really make sense. You don't get to personally keep billions of dollars without caring about money.
I've seen a lot of people that work in AI express concerns about the impact on society as we progress towards AGI. We don't need to reach AGI for there to be a huge impact on society whether its negative or positive. OpenAI's tool is already having an impact on jobs, it seems straight forward to extrapolate that there will be larger impacts on society as models improve. I would still assert that it would be a comical level of supervillainy for Sam to only care about his companies profits over any potential negative impacts on society. It seems highly plausible that he could not be a sociopath and have empathy and actually cares about limiting negative impacts of these tools.
> It's nice to think that you can have so much money you stop caring about it but it doesn't really make sense. You don't get to personally keep billions of dollars without caring about money.
It is definitely not every billionaire that thinks like this but there are some that seem to not care too much about material possessions like Musk or Sam. Their endeavors seem to be the things that they care about the most and they don't seem to be doing it to attain the most amount of money possible.
> It's nice to think that you can have so much money you stop caring about it but it doesn't really make sense. You don't get to personally keep billions of dollars without caring about money.
It is definitely not every billionaire that thinks like this but there are some that seem to not care too much about material possessions like Musk or Sam. Their endeavors seem to be the things that they care about the most and they don't seem to be doing it to attain the most amount of money possible.
Exacerbating ecological disaster building more and more resource and energy hungry data centers does not seem aligned with the rest of humanity
I have been running ML on my personal data and modeling worlds for the last few years. There’s no reason to keep OpenAI around given open source. My own data does not need a river water cooled data center.
OpenAIs ONLY moat is a government one so governments can keep up the free market ruse but have OpenAI in its back pocket for military and intelligence applications
I have been running ML on my personal data and modeling worlds for the last few years. There’s no reason to keep OpenAI around given open source. My own data does not need a river water cooled data center.
OpenAIs ONLY moat is a government one so governments can keep up the free market ruse but have OpenAI in its back pocket for military and intelligence applications
It's certainly possible. I can't read minds. But his statements on these issues sound more to me like "Yes, this is really dangerous stuff and we're taking a huge risk in developing it, but too bad! You can't stop us."
His point of view, that he's stated on the Lex Friedman podcast, is that AGI is potentially dangerous and inevitable, but safer to develop as soon as possible and try to make it safe by studying real-world models empirically, rather than waiting until computing power is much greater and risking a "fast takeoff".
I don't know that I agree with this strategy, but I can respect that it's a consistent viewpoint.
So what I hear from him is "Yes, this is dangerous stuff, and we're trying our hardest to make it safe, but we can't control everyone, so we need regulation ASAP because market forces are going to push everyone to race and release unsafe AIs".
I don't know that I agree with this strategy, but I can respect that it's a consistent viewpoint.
So what I hear from him is "Yes, this is dangerous stuff, and we're trying our hardest to make it safe, but we can't control everyone, so we need regulation ASAP because market forces are going to push everyone to race and release unsafe AIs".
Seems like a reasonable point of view from Sam. Better to have some kind of AGI catastrophe when some kind of Butlerian Jihad is still possible.
Yes, I know his point of view, but it doesn't really help answer the question.
> I can respect that it's a consistent viewpoint.
I don't think I agree that there's a great deal of consistency between that stated perspective and what OpenAI is actually doing. It's also, conveniently, a point of view that allows him to keep doing what he wants to do, and making bank by doing it.
Understand, I'm not saying he's a bad actor. I'm saying that it's hard to rule that out. From his WorldCoin stuff to this, there is plenty of reason to be suspicious of his motives.
> I can respect that it's a consistent viewpoint.
I don't think I agree that there's a great deal of consistency between that stated perspective and what OpenAI is actually doing. It's also, conveniently, a point of view that allows him to keep doing what he wants to do, and making bank by doing it.
Understand, I'm not saying he's a bad actor. I'm saying that it's hard to rule that out. From his WorldCoin stuff to this, there is plenty of reason to be suspicious of his motives.
Possible, yes. But coming from the guy that wants to scan everyone's irises, unlikely.
Maybe he sincerely believes the line OpenAI's PR department is pushing, but that's an even better reason not to listen to him.
Isn't it possible that Sam is actually a lizard person with a really good disguise?
possible, absolutely. just like it's possible every asshole who did it before him was due to a genuine desire for the good of humanity.
aw heck, let's go so far as to say anyone willing to testify if front of congress primarily does so because they want to protect the best interests of others.
Would be a fucking swell world to live in.
aw heck, let's go so far as to say anyone willing to testify if front of congress primarily does so because they want to protect the best interests of others.
Would be a fucking swell world to live in.
> just like it's possible every asshole who did it before him was due to a genuine desire for the good of humanity.
So.. you're saying that every person that testifies before congress has bad intentions just because they testify before congress?
Say he did want what was best for humanity, how should he have acted differently in front of congress?
To me, from every interview I've heard from him, he doesn't seem like a Zuckerberg.
So.. you're saying that every person that testifies before congress has bad intentions just because they testify before congress?
Say he did want what was best for humanity, how should he have acted differently in front of congress?
To me, from every interview I've heard from him, he doesn't seem like a Zuckerberg.
My point is that there is no room for granting leeway on either side. "But what if they really want to help the world" is the cry of founders who haven't been shown to be assholes yet.
Don't assume they're they're to help. Don't assume they're there to hurt.
They are there to cover their own asses and nothing more.
Don't assume they're they're to help. Don't assume they're there to hurt.
They are there to cover their own asses and nothing more.
> Don't assume they're they're to help. Don't assume they're there to hurt.
Why is the assumption that they are only there to cover their own asses ok to make but other assumptions are not?
We don't know what he is actually thinking. It is entirely plausible that he is not a sociopath and actually wants to minimize negative impacts on society through regulation because he thinks that is the best path forward.
Why is the assumption that they are only there to cover their own asses ok to make but other assumptions are not?
We don't know what he is actually thinking. It is entirely plausible that he is not a sociopath and actually wants to minimize negative impacts on society through regulation because he thinks that is the best path forward.
justrealist(6)
It's called "regulatory capture"[1]. As a CEO If you manage to accomplish it, you get +100 HP.
1. https://en.wikipedia.org/wiki/Regulatory_capture
1. https://en.wikipedia.org/wiki/Regulatory_capture
Outside of self driving cars (which imo can and should be regulated by the ntsa), the thing that has never killed zero people probably shouldn't be regulated by the thing that has killed... a lot more than that until there's a clear and obvious safety concern with something specific within the "AI" category of technology (would love if we de-scifi'd this and called it machine learning but I'm picking my battles here).
I don't think "it hasn't killed people yet" is a very good criteria. There are a lot of plausible dangers that existing and imaginable large-machine-learning could involve.
I can't see how one could do the regulating of AI but I don't think one can limit the worry-area to "clear and obvious" situations.
I can't see how one could do the regulating of AI but I don't think one can limit the worry-area to "clear and obvious" situations.
> plausible dangers
Name three. There are 50,000 gun related deaths in the United States per year, are any of them a bigger threat than gun violence? I struggle to find a single thing AI can do, sans being hooked up to nuclear weapons and time travelling robots with german accents, that's even imaginatively more dangerous than what this country already allows and decides doesn't require useful regulations.
I'm sorry if that's too political for HN but you know what, it actually does matter to compare real dangers (car accidents, gun violence) to conceived and made up ones (gender indifferent bathrooms, AI) when deciding how governments should prioritize policy and regulations.
Name three. There are 50,000 gun related deaths in the United States per year, are any of them a bigger threat than gun violence? I struggle to find a single thing AI can do, sans being hooked up to nuclear weapons and time travelling robots with german accents, that's even imaginatively more dangerous than what this country already allows and decides doesn't require useful regulations.
I'm sorry if that's too political for HN but you know what, it actually does matter to compare real dangers (car accidents, gun violence) to conceived and made up ones (gender indifferent bathrooms, AI) when deciding how governments should prioritize policy and regulations.
Existential threat is a really hard thing to grapple with. You can't wait for real-world evidence of danger to deal with it.
If there's a possibility of an asteroid coming that will wipe out life on earth, waiting until our telescopes can see it to start taking action may not give us enough time to react. We have to plan for the theoretical possibility.
Maybe we'll never be able to create artificial superintelligence. Maybe artificial superintelligence will be much easier to align than we thing. Maybe it'll be hard, but manageable, possibly because we have more time to work on the problem than we think.
All of those are possibilities, but the fact that if you ask 100 people about it, you'll get 100 different opinions, means that we have very little idea how things will go. Given that, I don't like the approach of wing it / hope for the best. If we end up stifling AI innovation unnecessarily, that may mean that our techno-utopia is delayed for a few years, but we get there in the end. On the other hand, if we under-react and things turn out really bad, we may not get another try. Scenarios where AGI kills a billion people are really bad, but scenarios where we create it and end up permanently disempowered or dead are worse.
I wonder sometimes if people are just really dissatisfied with their lives, that they react with such vitriol to the idea of AI progress being delayed. I know we have a lot of problems, but how many of them are technological vs. having poor human institutions and incentive structures? AI probably won't help much with that.
If there's a possibility of an asteroid coming that will wipe out life on earth, waiting until our telescopes can see it to start taking action may not give us enough time to react. We have to plan for the theoretical possibility.
Maybe we'll never be able to create artificial superintelligence. Maybe artificial superintelligence will be much easier to align than we thing. Maybe it'll be hard, but manageable, possibly because we have more time to work on the problem than we think.
All of those are possibilities, but the fact that if you ask 100 people about it, you'll get 100 different opinions, means that we have very little idea how things will go. Given that, I don't like the approach of wing it / hope for the best. If we end up stifling AI innovation unnecessarily, that may mean that our techno-utopia is delayed for a few years, but we get there in the end. On the other hand, if we under-react and things turn out really bad, we may not get another try. Scenarios where AGI kills a billion people are really bad, but scenarios where we create it and end up permanently disempowered or dead are worse.
I wonder sometimes if people are just really dissatisfied with their lives, that they react with such vitriol to the idea of AI progress being delayed. I know we have a lot of problems, but how many of them are technological vs. having poor human institutions and incentive structures? AI probably won't help much with that.
> If there's a possibility of an asteroid coming that will wipe out life on earth, waiting until our telescopes can see it to start taking action may not give us enough time to react.
How would we even know what to do until we have a concrete target?
How would we even know what to do until we have a concrete target?
Vecr gives good analogies, but for AGI, I think investing a lot into alignment research, trying to set up incentive structures to actively avoid "arms races", establishing global oversight and agreements, having a "fire alarm" or process in place that a company can trigger if they develop an AI that they think is close to becoming dangerous, or even individuals. Strong whistleblower protections may help. Tracking GPUs so we know where compute is being gathered together. Starting talks with other countries like China, right now, to build a common understanding of how to deal with a potential problem.
I think it's true that it may be very difficult to solve alignment theoretically, and solving it when we have real-world systems to look at may make it easier (but still probably not easy). But we need to be able to buy companies time so they can slow down as we approach what seems to be the "edge" of dangerous capability / consciousness / goal directedness, without feeling like they will lose out. And to be very strict about the conditions the models are trained under and what testing they undergo before release.
A lot of this is stuff that people are already suggesting, but not everyone is taking the risk seriously.
I think it's true that it may be very difficult to solve alignment theoretically, and solving it when we have real-world systems to look at may make it easier (but still probably not easy). But we need to be able to buy companies time so they can slow down as we approach what seems to be the "edge" of dangerous capability / consciousness / goal directedness, without feeling like they will lose out. And to be very strict about the conditions the models are trained under and what testing they undergo before release.
A lot of this is stuff that people are already suggesting, but not everyone is taking the risk seriously.
They can test deflection methods on other asteroids, build large spacecraft launchers and have them ready to go, do surveys to make sure the N-body predictions are as accurate as possible, etc. Some of that's being done (NEOWISE, DART) but if you want to deflect a planet killer you need a lot more.
I won't name three but the one everyone's really afraid of is the theoretical fast/hard "take-off" runaway problem. It is plausible that a smart enough AI given any instruction, without guardrails, will ultimately destroy the world in its attempt to do that thing. See the wiki on Instrumental Convergence: https://en.wikipedia.org/wiki/Instrumental_convergence and in particular, read about the "paperclip maximizer". Paperclips are just an example.
It remains to be seen if this is ultimately a real problem. If it turns out that it is, then not regulating the development of these systems in advance may be a huge mistake.
It remains to be seen if this is ultimately a real problem. If it turns out that it is, then not regulating the development of these systems in advance may be a huge mistake.
So, because we haven't been able to resolve a highly political and divisive issue (gun deaths), that means we shouldn't try to resolve other dangerous issues?
> I struggle to find a single thing AI can do
How about job loss? If AI can do what proponents say, then it's easy to see that a large amount of job loss will result. If even 20% of people can no longer earn a living, people will die.
> I struggle to find a single thing AI can do
How about job loss? If AI can do what proponents say, then it's easy to see that a large amount of job loss will result. If even 20% of people can no longer earn a living, people will die.
It's also important to note here that even if you believe that all these ML models are not even close to "AI", that doesn't stop your naive boss from buying their hype and replacing you with it. There's good reason to talk about how society should handle that even if ChatGPT is all hot air.
> How about job loss?
This is called creative destruction, and it's not even remotely a new problem created by this particular technology, it's a constant and ongoing mechanism of market capitalism by which technology streamlines or improves something that then requires less labor to accomplish.
When the 400k+ telephone switch operators lost their jobs to digitization, it didn't cause a mass die off of humans, and while I won't say it was a zero bad consequences event, they mostly found other work, work that was probably more interesting and more productive. We certainly could have regulated telephone digitization to prevent this "danger" and instead employed 4.5 million people, or about 3 percent of the labor force, to operate the phone network today as it would have existed technologically in the 1970s, but I'm happy we skipped the scare mongering and went the digitization route, which probably also was a prerequisite for the internet to be able to be used by.. users (which mostly used digital phone services via modems to connect). Would even consumer internet access exist today if they decided to regulate the "danger" of losing analog phone operator jobs?
Either way, I fail to see how allowing only licensed and government regulated companies to work on machine learning will do anything to improve anyone's job prospects moving forward, as increasingly onerous requirements created by people that like to say things like "the internet is a bunch of tubes" make it so only a few rent-seeking monopolies will be able to do anything (legally) with the technology.
I'm not 100% against government regulation, but it's early, things are relatively benign so far (sans self driving cars), let's wait until there's some actual non-sci-fi concerns that emerge and then tackle them as specific things to regulate, rather than wrapping the whole thing into a giant regulatory framework that incorrectly calls it AI before we even learn what the problems are going to be.
This is called creative destruction, and it's not even remotely a new problem created by this particular technology, it's a constant and ongoing mechanism of market capitalism by which technology streamlines or improves something that then requires less labor to accomplish.
When the 400k+ telephone switch operators lost their jobs to digitization, it didn't cause a mass die off of humans, and while I won't say it was a zero bad consequences event, they mostly found other work, work that was probably more interesting and more productive. We certainly could have regulated telephone digitization to prevent this "danger" and instead employed 4.5 million people, or about 3 percent of the labor force, to operate the phone network today as it would have existed technologically in the 1970s, but I'm happy we skipped the scare mongering and went the digitization route, which probably also was a prerequisite for the internet to be able to be used by.. users (which mostly used digital phone services via modems to connect). Would even consumer internet access exist today if they decided to regulate the "danger" of losing analog phone operator jobs?
Either way, I fail to see how allowing only licensed and government regulated companies to work on machine learning will do anything to improve anyone's job prospects moving forward, as increasingly onerous requirements created by people that like to say things like "the internet is a bunch of tubes" make it so only a few rent-seeking monopolies will be able to do anything (legally) with the technology.
I'm not 100% against government regulation, but it's early, things are relatively benign so far (sans self driving cars), let's wait until there's some actual non-sci-fi concerns that emerge and then tackle them as specific things to regulate, rather than wrapping the whole thing into a giant regulatory framework that incorrectly calls it AI before we even learn what the problems are going to be.
> it's not even remotely a new problem created by this particular technology
True. But this technology (again, if the proponents are correct) threatens to do it on a truly unprecedented scale. It was bad enough -- barely tolerable in some cases -- in history. Getting it even worse seems like something nobody would want, let alone cheerlead.
I am deeply, deeply concerned that there are people willing to gamble with the lives of innocent others (not to mention society itself) like this. It's something we need to be addressing now, before it happens.
True. But this technology (again, if the proponents are correct) threatens to do it on a truly unprecedented scale. It was bad enough -- barely tolerable in some cases -- in history. Getting it even worse seems like something nobody would want, let alone cheerlead.
I am deeply, deeply concerned that there are people willing to gamble with the lives of innocent others (not to mention society itself) like this. It's something we need to be addressing now, before it happens.
If “has killed people” was a viable argument for regulation we’d have a lot more regulation.
Which is fine; engineers are not divine beings. They can have their figurative value deflated like everyone else is these days. Engineered devices have killed a lot of people.
Which is fine; engineers are not divine beings. They can have their figurative value deflated like everyone else is these days. Engineered devices have killed a lot of people.
Is your POV (US/Federal) government should only be allowed to regulate in cases of lethality because the government itself has carried lethal behvaior?
No one seems to care about this obvious absurdity.
We are not bothering to regulate AI controlling a 3500lb death machine that goes from 0 to 60 in 2 seconds but we are worried about the dangers of a chatbot.
So typical of our society to be captured by the more compelling narrative and ignore actual reality.
We are not bothering to regulate AI controlling a 3500lb death machine that goes from 0 to 60 in 2 seconds but we are worried about the dangers of a chatbot.
So typical of our society to be captured by the more compelling narrative and ignore actual reality.
> Outside of self driving cars
This is what I find so hilariously suspicious about Elon Musk's sudden anxiety about the dangers of AI chatbots. Like, dude, you've spent the better part of two decades making cars that drive themselves and have _already_ killed people with their faulty, ironically oversold "AI." Now all of a sudden someone else's AI, which isn't in the drivers seat of vehicles capable of 100mph+ speeds, and which actually works, is an existential danger to humanity?
This is what I find so hilariously suspicious about Elon Musk's sudden anxiety about the dangers of AI chatbots. Like, dude, you've spent the better part of two decades making cars that drive themselves and have _already_ killed people with their faulty, ironically oversold "AI." Now all of a sudden someone else's AI, which isn't in the drivers seat of vehicles capable of 100mph+ speeds, and which actually works, is an existential danger to humanity?
A minor correction: it’s not unambiguously zero since there was a suicide blamed on a chatbot.
But sure, most of the concern about AI is about stuff people imagine happening in the future. So the question is whether future safety concerns are worrisome enough to want to get a head start on regulation.
Also, it’s not just deaths. Generative AI is, among other things, potentially an unlimited supply of disgusting imagery. That’s likely to be a big legal headache for lots of organizations.
But sure, most of the concern about AI is about stuff people imagine happening in the future. So the question is whether future safety concerns are worrisome enough to want to get a head start on regulation.
Also, it’s not just deaths. Generative AI is, among other things, potentially an unlimited supply of disgusting imagery. That’s likely to be a big legal headache for lots of organizations.
If “has killed someone” is a criteria for extreme regulation then fossil fuel and banking would be nationalized by now.
Sam Altman suggesting AI dev require licensing is a pretty perfect move to capitalize on AI fear while also blocking new market entrants
Why do we need to regulate ai?
Because palintir keeps trying to sell "just arrest all the black people" to the judicial system as "AI", and is also trying to sell "just let the computer blow everyone up" to the military. We don't need AGI for "AI" to make our lives worse.
Can you really not see how this will automate harm at massive parallelized scale?
Even if you're nearsighted you can see how the internet will in short order be rendered useless by fake real-time interaction and digital malice. Look at how much automated ssh poking you get on an IP address. Look at how much email you receive without a comprehensive spam filter (it's much more than shows up in your Gmail spam folder). If we combine that level of relentless volume and speed with moderate skills at hacking, fraud, influence campaigns, abuse, and so on, the internet will break.
Look at the all the plugins GPT-4 has already. Do you really think if a system with those capabilities, with evil intent, running millions of threads, wouldn't cause untold levels of harm? If not, then at which point do you push for regulation? Do you wait until anyone can build a literal terminator in their basement? Openai is already working on robotic control models, so that's not likely not far off.
All I see are mindless silicon valley tech-optimists who can't see the forest for the trees.
Even if you're nearsighted you can see how the internet will in short order be rendered useless by fake real-time interaction and digital malice. Look at how much automated ssh poking you get on an IP address. Look at how much email you receive without a comprehensive spam filter (it's much more than shows up in your Gmail spam folder). If we combine that level of relentless volume and speed with moderate skills at hacking, fraud, influence campaigns, abuse, and so on, the internet will break.
Look at the all the plugins GPT-4 has already. Do you really think if a system with those capabilities, with evil intent, running millions of threads, wouldn't cause untold levels of harm? If not, then at which point do you push for regulation? Do you wait until anyone can build a literal terminator in their basement? Openai is already working on robotic control models, so that's not likely not far off.
All I see are mindless silicon valley tech-optimists who can't see the forest for the trees.
We already get spammed and hammered by bots without ai so? Did we manage to somehow regulate away existing systems to stop this? No, so why put ai behind a bunch of regulation.
Maybe in a dystopia we have ai judges...wait he already have judges using shitty software for sentencing.
You want to further regulate what I can build in my basement?
You want to further regulate what I can build in my basement?
> Did we manage to somehow regulate away existing systems to stop this?
No, but other countries have. We just don't do regulation because we're America or something (and because the FCC has been gutted). No European country has the same level of spam phone calls that we do.
No, but other countries have. We just don't do regulation because we're America or something (and because the FCC has been gutted). No European country has the same level of spam phone calls that we do.
It creates situations no existing laws cover or are ambiguous about. Copyright, for example. Can you demand the training data to exclude your works? Can you circumvent GPL by using machine learning?
Can you make discrimination legal by flaming AI did it? Can you sue for damages when AI produces incorrect result?
The chat bots might be allowed to say bad words.
It seems silly, but that's an insightful remark.
AI systems mis-identifying people and getting them arrested - minor problem. Systems firing people for underperforming - that's just a company exercising its right to enforce "at-will employment". Chatbots designed to make it difficult or impossible to cancel a service - almost standard. Announcing a hiring freeze while jobs are replaced with AI systems - that's productivity.
But a system that will answer "Rank races by intelligence" gets people very upset.
AI systems mis-identifying people and getting them arrested - minor problem. Systems firing people for underperforming - that's just a company exercising its right to enforce "at-will employment". Chatbots designed to make it difficult or impossible to cancel a service - almost standard. Announcing a hiring freeze while jobs are replaced with AI systems - that's productivity.
But a system that will answer "Rank races by intelligence" gets people very upset.
A cyber attack that publicizes your patient records database or wipes your production configs are also "bad words" if you type them in a terminal.
Under the current copyright regime, I think we need very strict protections for creators and these huge AI models that can generate new content. Maybe make works with copyright require a separate and explicit permission for training and structure in a specific way so that it is not easy to get permission for this stuff without paying creators (so no just slurping up data from GitHub using ToS, they'd have to get explicit agreements from developers).
Ai is just the tool. The copyright issue is now with the derived work produced by ai.
So what is meant by regulate ai is amend existing laws for the age of ai?
So what is meant by regulate ai is amend existing laws for the age of ai?
For a lot of people bally-hoing about Sam Altman trying to trick Congress into letting him hold the reigns of AI regulation;
Good. What would be even better if AI researchers and engineers applied en-mass to be the in-house technical advisement staff for congressional members and executive/congressional regulatory agencies for AI.
Good. What would be even better if AI researchers and engineers applied en-mass to be the in-house technical advisement staff for congressional members and executive/congressional regulatory agencies for AI.
Really a shame if anyone is fooled by this. It's just monopolistic behavior camouflaged as virtue signaling.
I see two possible motivations:
1) he is truly scared AI can become dangerous and try to distribute the responsibility to more people 2) he anticipates that there will be some regulation anyway, and when that's the case, it is better to be involved and act friendly.
1) he is truly scared AI can become dangerous and try to distribute the responsibility to more people 2) he anticipates that there will be some regulation anyway, and when that's the case, it is better to be involved and act friendly.
Here as a post on his blog from 8 years ago: https://blog.samaltman.com/the-software-revolution
"...development of an AI that could end human life may only require a few hundred people in an office building anywhere in the world, with no equipment other than laptops.
The new existential threats won’t require the resources of nations to produce."
and
"The fact that we don’t have serious efforts underway to combat threats from synthetic biology and AI development is astonishing."
So (2) could be a factor, for sure. But (1) is consistent with what seems to have been his viewpoint for a long time. It's possible he's updated his viewpoint and is faking concern about safety, but given how recent advances have seemed to make a lot of people more concerned, I don't think that's likely.
"...development of an AI that could end human life may only require a few hundred people in an office building anywhere in the world, with no equipment other than laptops.
The new existential threats won’t require the resources of nations to produce."
and
"The fact that we don’t have serious efforts underway to combat threats from synthetic biology and AI development is astonishing."
So (2) could be a factor, for sure. But (1) is consistent with what seems to have been his viewpoint for a long time. It's possible he's updated his viewpoint and is faking concern about safety, but given how recent advances have seemed to make a lot of people more concerned, I don't think that's likely.
But his existence as a head of an AI company makes him a terrible choice for any recommendations, full stop. He could be 100% genuine but we can't trust him because we have an antagonistic system.
Yes, him being the head of an AI company means you should be skeptical. But jumping right to "let's do the exact opposite" is not sensible.
Also, sitting right next to him was Gary Marcus, who doesn't have a dog in the fight as far as I know, and he agreed with a most of the recommendations.
Also, sitting right next to him was Gary Marcus, who doesn't have a dog in the fight as far as I know, and he agreed with a most of the recommendations.
How could AI practically be regulated? Would this be the first illegal algorithm?
> Would this be the first illegal algorithm?
Not by a longshot. See cryptography before the '90s for the most recognizable example.
Not by a longshot. See cryptography before the '90s for the most recognizable example.
All of the very capable models required millions of dollars of compute training on high end low latency hardware.
You can count on your hands the number of high end chip manufacturers and GPU companies, as well as the number of companies with existing high performance clusters. Treat the production and use of datacenter GPUs the same way we treat enriched uranium. We've already started since database GPUs were added to the export control list.
You can count on your hands the number of high end chip manufacturers and GPU companies, as well as the number of companies with existing high performance clusters. Treat the production and use of datacenter GPUs the same way we treat enriched uranium. We've already started since database GPUs were added to the export control list.
This sounds like the answer I was looking for. Thank you!
and why trust the state to select/enact control policy?
True but DC is so backward and flatfooted relative to Beijing that he may be.
Personally I’d rather Sydney regulate AI
bionhoward(1)