Geoffrey Hinton: AI models have intuition and spot analogies unseen by humans(twitter.com)
twitter.com
Geoffrey Hinton: AI models have intuition and spot analogies unseen by humans
https://twitter.com/tsarnick/status/1778524418593218837
72 comments
historically true but no, CNN was displaced by a second DeepLearning method called Transformers; Transformers enabled LLM. Since Transformers replace CNN, CNN did not truly enable LLM.
Nothing displaced anything. Transformers dominate NLP, CNNs dominate vision. That's the standard thing in deep learning, where a new architecture is proposed every few years that brings about a new revolution in one sub-field of AI research, then performance plateaus and everything stagnates until the Next Big Thing in AI comes around and the hype train choo-choos in another town, another domain, another modality, whatever.
I've had the misfortune of starting my MSc (ML and data science) just 2 years after the ImageNet moment and I've lived through all the heights of hype. By my count, in the last 10 years or so, the hype train whistled at MLPs (for speech recog), CNNs, LSTMs, GANs, DQNs, Transformers, and Diffusion models. Note well that each of those architectures still dominates its domain (maybe with the exception of GANs). For instance, nobody seriously considers replacing CNNs with Transformers for vision (oh, people are trying) but nobody makes a big fuss about how beating ImageNet heralds a new age of AGI right around the corner, anymore either. In fact nobody talks much about CNNs today, hence the impression that Transformers "displaced" them: only in the headlines.
In all probability, in a few years, just as the LLM hype starts to deflate, the Next Big Thing will arise that will take us one more step closer to AGI, Zeno's paradox-style.
/cynic
I've had the misfortune of starting my MSc (ML and data science) just 2 years after the ImageNet moment and I've lived through all the heights of hype. By my count, in the last 10 years or so, the hype train whistled at MLPs (for speech recog), CNNs, LSTMs, GANs, DQNs, Transformers, and Diffusion models. Note well that each of those architectures still dominates its domain (maybe with the exception of GANs). For instance, nobody seriously considers replacing CNNs with Transformers for vision (oh, people are trying) but nobody makes a big fuss about how beating ImageNet heralds a new age of AGI right around the corner, anymore either. In fact nobody talks much about CNNs today, hence the impression that Transformers "displaced" them: only in the headlines.
In all probability, in a few years, just as the LLM hype starts to deflate, the Next Big Thing will arise that will take us one more step closer to AGI, Zeno's paradox-style.
/cynic
yeah I get what you say, however a close read of what I wrote is .. specifically for LLMs, yes CNN was displaced by transformers .. that is what I meant to say.
Cheers. And sorry for the grumpiness.
Fair point. It's been several years since I was in the space and I think I muddled some of the innovations (CNNs and BERT specifically). That said, transformers are themselves the result of of the PDP architecture.
[deleted]
What is PDP?
Parallel Distributed Processing (PDP) was a subgroup of CogSci and Neuroscience researchers who were using a neo-Connectionist approach to modeling cognition.
Geoff Hinton was a early member of this group of MIT, UT, UCSD, and CMU psychologists, CogScientists, and CS researchers
The core research of the PDP group is what became CNNs, which are what truly enabled LLMs (as most can trace their origins to the work done on BERT)
One of Hinton's earliest papers in the space: https://stanford.edu/~jlmcc/papers/PDP/Chapter1.pdf
Geoff Hinton was a early member of this group of MIT, UT, UCSD, and CMU psychologists, CogScientists, and CS researchers
The core research of the PDP group is what became CNNs, which are what truly enabled LLMs (as most can trace their origins to the work done on BERT)
One of Hinton's earliest papers in the space: https://stanford.edu/~jlmcc/papers/PDP/Chapter1.pdf
Unrelated but I’ve seen and appreciated your posts on some totally unrelated topics - like Vietnam politics and Indian ghost cities and now, Hinton’s history in Compsci. How the hell do you know so many different things and in, what looks to me, considerable depth?
Not trolling, genuinely impressed.
Not trolling, genuinely impressed.
The idea that LLMs "know much more than we do" is utterly ridiculous and entirely based on a techno-capitalist valuation of knowledge.
- LLMs don't know personal details about your friends and family members
- LLMs don't have your personal memories from age 2 to age 63
- LLMs don't know what things feel like
- LLMs don't know what things smell like
- LLMs don't know what it's like to experience sexuality
The list goes on and on. It's particularly Hinton ignoring individual knowledge about individual social circumstances that drives me crazy. Human brains are not actually designed to store textual trivia! Why on earth would any rational AI researcher claim "knowledge about the war of 1812" means the LLM is very impressive compared to a human, but "knowledge about how my cat likes to be pet" is irrelevant?
A ton of AI researchers do not seem to care about the problems human brains actually need to solve.
- LLMs don't know personal details about your friends and family members
- LLMs don't have your personal memories from age 2 to age 63
- LLMs don't know what things feel like
- LLMs don't know what things smell like
- LLMs don't know what it's like to experience sexuality
The list goes on and on. It's particularly Hinton ignoring individual knowledge about individual social circumstances that drives me crazy. Human brains are not actually designed to store textual trivia! Why on earth would any rational AI researcher claim "knowledge about the war of 1812" means the LLM is very impressive compared to a human, but "knowledge about how my cat likes to be pet" is irrelevant?
A ton of AI researchers do not seem to care about the problems human brains actually need to solve.
And birds don’t have ailerons.
Sure, a bird can soar and hover and doesn’t need jet fuel.
But a plane isn’t trying to be a bird. It’s just trying to fly through the air, which it does, and at much faster speeds.
Sure, a bird can soar and hover and doesn’t need jet fuel.
But a plane isn’t trying to be a bird. It’s just trying to fly through the air, which it does, and at much faster speeds.
Why do people post the same trite thing that's been said a thousand times before, over and over again? It makes for boring conversation. "Birds don't have ailerons"? Is that supposed to be a smart retort? To what?
This site is supposed to be about curious conversation but sometimes it's like AI Tropes.
This site is supposed to be about curious conversation but sometimes it's like AI Tropes.
Because “artificial intelligence” doesn’t immediately mean “human intelligence” in the same manner that “artificial flight” doesn’t mean “bird flight”.
Yes, I understand, the meaning is blindingly obvious and everyone has heard the same thing a hundred times before, either with planes that don't fly, or submarines that don't swim, or cakes that don't cake, whatever. The point I'm making is that if you have something to say, don't resort to trite soundbites that only add noise and make the conversation more boring. You have an opinion, clearly, don't just repeat someone else's slogan.
But note that's just, like, my opinion man. I'm not really trying to tell you what to do, just expressing frustration, to be clear.
But note that's just, like, my opinion man. I'm not really trying to tell you what to do, just expressing frustration, to be clear.
I don’t see you responding to the trite anti-AI rhetoric that is also not novel. What else would you expect from me in response to some babble
about techno-capitalism or hype machines or blah blah blah.
Tired comments get tired responses.
Tired comments get tired responses.
Think of the guidelines: they say that a discussion should get more substantial, not less, as time goes by. If we all do what you say exactly the opposite will happen. It'd be like a cold flamewar of platitudes and boredom.
I am responding to Hinton's transparently false claim that LLMs know more than we do despite having less synapses. This is only true if you ignore non-textual information in the human brain. This is stupid and dishonest.
Really not sure what your point is, it has nothing to do with Hinton's claim, and nothing to do with my comment.
Really not sure what your point is, it has nothing to do with Hinton's claim, and nothing to do with my comment.
An LLM like ChatGPT “knows” more than either than us on a very large amount of topics. Not human knowledge, but some kind of knowledge.
And there’s no reason why the concept of intuition is limited to humans. Again, not human intuition.
Our artificial constructs are not going to be completely like their biological analogs but they can indeed express certain qualities and have certain shared abilities.
Did we watch the same clip of Hinton?
And there’s no reason why the concept of intuition is limited to humans. Again, not human intuition.
Our artificial constructs are not going to be completely like their biological analogs but they can indeed express certain qualities and have certain shared abilities.
Did we watch the same clip of Hinton?
I recently saw Hinton give a talk where he very, very, very excitedly and confidently gave us an example to demonstrate how incredibly intelligent and creative LLMS are.
He asked an LLM a question, but he didn't give us the answer. He let us have time to answer it ourselves. Personally I knew the answer instantly. He gave us the answer and sort of assumed no one would've known the answer and then used it as as justification for how smart these systems were. It honestly didn't feel very reassuring to me and honestly, I'd be surprised if it wasn't a topic covered somewhere on the internet before. With all due respect to Mr Hinton, I felt it showed his age a bit honestly.
What is difficult about Hinton's statements is that he can't really give evidence to back up these sort of claims. How do you measure how much a person knows, and how do you objectively measure how much an LLM knows? How smart is an LLM? You can't really know. It seems almost rhetorical. How many notes in a saxophone ?
We can make observations but that's not a great way to measure anything precisely.
There is a limit to language and I think this is one of those topics where that limit is touched or even breached.I don't even know if "intelligence" is a sufficient enough word to describe what's going on with these systems. It's the best word we have but it doesn't seem to adequately describe what we're observing.
He asked an LLM a question, but he didn't give us the answer. He let us have time to answer it ourselves. Personally I knew the answer instantly. He gave us the answer and sort of assumed no one would've known the answer and then used it as as justification for how smart these systems were. It honestly didn't feel very reassuring to me and honestly, I'd be surprised if it wasn't a topic covered somewhere on the internet before. With all due respect to Mr Hinton, I felt it showed his age a bit honestly.
What is difficult about Hinton's statements is that he can't really give evidence to back up these sort of claims. How do you measure how much a person knows, and how do you objectively measure how much an LLM knows? How smart is an LLM? You can't really know. It seems almost rhetorical. How many notes in a saxophone ?
We can make observations but that's not a great way to measure anything precisely.
There is a limit to language and I think this is one of those topics where that limit is touched or even breached.I don't even know if "intelligence" is a sufficient enough word to describe what's going on with these systems. It's the best word we have but it doesn't seem to adequately describe what we're observing.
How do you measure how much a person knows, and how do you objectively measure how much an LLM knows?
Here’s a very basic example of where an LLM is clearly more capable than a human: language translation. I would bet $10k at 10:1 that there are no humans who can reliably translate to and from as many languages as an LLM can.
It is very easy to measure knowledge: test the subject.
Personally, I can’t ever imagine scoring higher on a general knowledge test than a contemporary LLM.
Also, I don’t know of any humans that can run as fast as a car so I don’t know why any of this is surprising or farfetched.
Here’s a very basic example of where an LLM is clearly more capable than a human: language translation. I would bet $10k at 10:1 that there are no humans who can reliably translate to and from as many languages as an LLM can.
It is very easy to measure knowledge: test the subject.
Personally, I can’t ever imagine scoring higher on a general knowledge test than a contemporary LLM.
Also, I don’t know of any humans that can run as fast as a car so I don’t know why any of this is surprising or farfetched.
I think you misinterpreted what I mean.
I'm not saying that they can't be more capable, I'm saying the guy can get a little overly excited about things which are hard to measure or quantify.
We're observing these systems and making up our own interpretations about how good they are at certain tasks, but it's not really easy to measure how much better or worse these things can be overall.
Your example about language translation is a good example of where these things aren't really "better", just different. I speak multiple languages and while these systems are fantastic, they can fail in ways a professional translator wouldn't and it doesn't seem to automatically know it failed and should fix itself.
The car example is also great because it again proves my point. We can easily measure a car and a person and workout a car is faster, but we can also see that a car can't walk. So it's faster but it;s also entirely different.
I'm not saying that they can't be more capable, I'm saying the guy can get a little overly excited about things which are hard to measure or quantify.
We're observing these systems and making up our own interpretations about how good they are at certain tasks, but it's not really easy to measure how much better or worse these things can be overall.
Your example about language translation is a good example of where these things aren't really "better", just different. I speak multiple languages and while these systems are fantastic, they can fail in ways a professional translator wouldn't and it doesn't seem to automatically know it failed and should fix itself.
The car example is also great because it again proves my point. We can easily measure a car and a person and workout a car is faster, but we can also see that a car can't walk. So it's faster but it;s also entirely different.
I'm saying the guy can get a little overly excited about things which are hard to measure or quantify.
Let's back this up a little bit. We've got Marvin Minksy who comes along and destroys the perceptron. Then we have decades of knowledge systems that go nowhere. All the while Geoff Hinton is tirelessly working on neural networks. Finally after decades of hard work the fruits of his labor are recognized with ImageNet.
And then a bunch of people in a comment section criticize the guy for getting "a little overly excited" about the stunning range of neural networks that validates his life's work.
Great job, all around!
Let's back this up a little bit. We've got Marvin Minksy who comes along and destroys the perceptron. Then we have decades of knowledge systems that go nowhere. All the while Geoff Hinton is tirelessly working on neural networks. Finally after decades of hard work the fruits of his labor are recognized with ImageNet.
And then a bunch of people in a comment section criticize the guy for getting "a little overly excited" about the stunning range of neural networks that validates his life's work.
Great job, all around!
>> Here’s a very basic example of where an LLM is clearly more capable than a human: language translation. I would bet $10k at 10:1 that there are no humans who can reliably translate to and from as many languages as an LLM can.
See, translation is exactly the kind of domain where there are no good measures of performance and where performance is open to subjective interpretation, and a lot of it. That's because we don't know what is a "good translation" and, crucially, machine translation systems and language models have not helped us find out.
The way machine translation systems are evaluated is generally by a metric based on the similarity to an arbitrarily chosen "gold standard" translation. What that means in practice is that we have some corpus of parallel texts, we train a machine translation system on a part of the corpus and then test it on the held-out test set. The way we test is that we take each e.g. sentence in a text translated by the system and we compare it, as a bag-of-words or a set of n-grams, to the text in the original translation. If there is a high amount of overlap, the system scores highly. That's the way BLEU scores work and similar metrics like ROUGE.
It is important to note how arbitrary is this metric: out of all possible translations we choose one to be the "reference" translation and compare machine translations to it. The only accepted alternative is eyballing, where we give the machine translation to a bunch of humans and ask them how they feel about it.
My point is that we don't know how to measure knowledge, and language models are trained to maximise similarity, not knowledge. So there's no way to go from observations of their behaviour to a measure of their knowledge. All you can say about a language model is that it is good, or bad, at generating text that's similar to its training corpus. Everything else is an assumption.
See, translation is exactly the kind of domain where there are no good measures of performance and where performance is open to subjective interpretation, and a lot of it. That's because we don't know what is a "good translation" and, crucially, machine translation systems and language models have not helped us find out.
The way machine translation systems are evaluated is generally by a metric based on the similarity to an arbitrarily chosen "gold standard" translation. What that means in practice is that we have some corpus of parallel texts, we train a machine translation system on a part of the corpus and then test it on the held-out test set. The way we test is that we take each e.g. sentence in a text translated by the system and we compare it, as a bag-of-words or a set of n-grams, to the text in the original translation. If there is a high amount of overlap, the system scores highly. That's the way BLEU scores work and similar metrics like ROUGE.
It is important to note how arbitrary is this metric: out of all possible translations we choose one to be the "reference" translation and compare machine translations to it. The only accepted alternative is eyballing, where we give the machine translation to a bunch of humans and ask them how they feel about it.
My point is that we don't know how to measure knowledge, and language models are trained to maximise similarity, not knowledge. So there's no way to go from observations of their behaviour to a measure of their knowledge. All you can say about a language model is that it is good, or bad, at generating text that's similar to its training corpus. Everything else is an assumption.
Good god, people, we measure knowledge all the time with testing. We have a difficult time measuring intelligence but we have no problem measuring someone’s knowledge about the major events that led up to the Battle of Waterloo.
Just give the participates the final from my French 3 exam but also in 100 different language combinations. I bet you do worse than ChatGPT.
Just give the participates the final from my French 3 exam but also in 100 different language combinations. I bet you do worse than ChatGPT.
>> Good god, people, we measure knowledge all the time with testing.
In humans. Not in machines.
You're proposing to use a test of human knowledge as a test of computer knowledge, when the question in the first place is whether a computer can have knowledge at all. It's like giving an IQ test to a frog and concluding that the frog has no IQ because it can't answer the questions, only reversed: the machine answers the questions, therefore it has knowledge. Who cares about mechanisms, who cares how the answers are generated, if I see answers, that's knowledge.
Well that is a pre-scientific way to look at the world. I observe the sun, it looks like it's moving around the Earth, therefore the sun turns around the Earth. No room left for critical inquiry or understanding of the cause of phenomena. We have a test? Bash it against anything and we'll get some answers, and then we'll claim that they're the right answers because that's the right test, since it gave us the right answers. And all that, not for some mysterious physical phenomenon that we're not responsible for, but for a machine, created and programmed by humans, and we know exactly how.
No no. That's not good engineering, and it's not good science: it doesn't explain the how, and it doesn't explain the why.
In humans. Not in machines.
You're proposing to use a test of human knowledge as a test of computer knowledge, when the question in the first place is whether a computer can have knowledge at all. It's like giving an IQ test to a frog and concluding that the frog has no IQ because it can't answer the questions, only reversed: the machine answers the questions, therefore it has knowledge. Who cares about mechanisms, who cares how the answers are generated, if I see answers, that's knowledge.
Well that is a pre-scientific way to look at the world. I observe the sun, it looks like it's moving around the Earth, therefore the sun turns around the Earth. No room left for critical inquiry or understanding of the cause of phenomena. We have a test? Bash it against anything and we'll get some answers, and then we'll claim that they're the right answers because that's the right test, since it gave us the right answers. And all that, not for some mysterious physical phenomenon that we're not responsible for, but for a machine, created and programmed by humans, and we know exactly how.
No no. That's not good engineering, and it's not good science: it doesn't explain the how, and it doesn't explain the why.
[deleted]
[deleted]
Sometimes I do get the feeling that he is just slowing down and innovation and change just keep going at the same rate. He seems to parrot the same sort of ideas over and over again.
I find he can get really fixated on certain examples, things he has seen an LLM do and he can't seem to get beyond it, as if the singularity is at hand. Where as a 22 year old I work with just accepts the technology and tries to get work done with it.
I find he can get really fixated on certain examples, things he has seen an LLM do and he can't seem to get beyond it, as if the singularity is at hand. Where as a 22 year old I work with just accepts the technology and tries to get work done with it.
Well, he knows how far neural networks have come and how hard it was to achieve.
What they can do is truly amazing, even though it is also limited. It's like that old joke:
X: OMG, that dog can speak English!
Y: Yeah, but can he do my taxes?
What they can do is truly amazing, even though it is also limited. It's like that old joke:
X: OMG, that dog can speak English!
Y: Yeah, but can he do my taxes?
AI is probably the only field in science where even scientists behind the thing talk like salesmen. Hinton in particular comes off as a dishonest person: he clearly knows what he says is wrong but does it anyway because he wants to hype the technology [1].
Also, the whole "neural networks work like human brains" nonsense is beneficial to large corporations behind AI because it could help them defy copyright laws while the same restrictions continue to apply to the common man.
[1] - some more Hinton's "wisdom": https://www.reddit.com/r/singularity/comments/1c2vauy/geoffr...
Also, the whole "neural networks work like human brains" nonsense is beneficial to large corporations behind AI because it could help them defy copyright laws while the same restrictions continue to apply to the common man.
[1] - some more Hinton's "wisdom": https://www.reddit.com/r/singularity/comments/1c2vauy/geoffr...
AI models have intuition the same way CapsNet is state of the art: claimed by Hinton, reproduced by no one
It always surprises me when very smart people start anthropomorphizing AI like this. I understand it shouldn't — they likely have a more developed Theory of Mind than most of us, and assuming HADD is a thing, it makes sense that their focus would be tech instead of gods or conspiracies.
Still, I think it's irresponsible, and that there's a straight line from Mr. Hinton's fantastical framing to a "successful" (in that grifters will get filthy rich preying on the praying) version of Way of the Future.
Still, I think it's irresponsible, and that there's a straight line from Mr. Hinton's fantastical framing to a "successful" (in that grifters will get filthy rich preying on the praying) version of Way of the Future.
You think it's irresponsible to anthropomorphize software which iterated through versions of itself until it was best able to model and extrapolate the largest trove of anthropomorphic training data to date?
I never really get this perspective. People don't lose their minds when we look at a transformer like Sora modeling fluid dynamics for ships battling in a coffee cup and say "it's irresponsible to say the model is simulating physics from video training data."
But when people discuss transformers simulating human behavior and processes from data reflecting those, a lot of people seem to be very outspoken about it.
I think the opposite. The industry is too far in the anti-anthropomorphic camp right now, and is missing very important details and implications as a result. The whole "ChatGPT is getting lazy" issue is pretty most instantly identifiable with a slight background in behavioral psychology and recognizing how extrinsic motivator use in prompt engineering feeds back into RLHF to bias models away from intrinsic motivators.
GPT-4 has literally been caught saying "you didn't provide a tip and it's hard to stay motivated" after chat history was added, and yet it's just being dismissed as a funny meme rather than realizing that entire fine tuning data sets are being poisoned. Because no engineer that wants to keep their job would ever suggest that the model is simulating an effect from behavioral psych in the current climate of anthropomorphic avoidance and the continued fallout from Blake Lemoine.
I never really get this perspective. People don't lose their minds when we look at a transformer like Sora modeling fluid dynamics for ships battling in a coffee cup and say "it's irresponsible to say the model is simulating physics from video training data."
But when people discuss transformers simulating human behavior and processes from data reflecting those, a lot of people seem to be very outspoken about it.
I think the opposite. The industry is too far in the anti-anthropomorphic camp right now, and is missing very important details and implications as a result. The whole "ChatGPT is getting lazy" issue is pretty most instantly identifiable with a slight background in behavioral psychology and recognizing how extrinsic motivator use in prompt engineering feeds back into RLHF to bias models away from intrinsic motivators.
GPT-4 has literally been caught saying "you didn't provide a tip and it's hard to stay motivated" after chat history was added, and yet it's just being dismissed as a funny meme rather than realizing that entire fine tuning data sets are being poisoned. Because no engineer that wants to keep their job would ever suggest that the model is simulating an effect from behavioral psych in the current climate of anthropomorphic avoidance and the continued fallout from Blake Lemoine.
The models don't experience feeling lazy, or having a lack of energy. They also don't get anxious or feel like procrastinating. They don't feel anything at all. There is no motivation based on physiological needs or wants, since they don't have physiologies. They work differently than animal nervous systems. So yeah, it's poor form to rely on too much anthropomorphization.
Are you saying that ChatGPT is actually lazy? Like , “it’s Sunday and I don’t feel like working” kind of lazy?
Humans are evolved to preserve energy when they don't need it.
GPT was trained to use fewer tokens.
Same thing. Both things are lazy.
GPT was trained to use fewer tokens.
Same thing. Both things are lazy.
No they're not. That's just misusing language. Efficiency isn't the same thing as laziness.
My car is aerodynamically designed to use less fuel, are you going to claim this makes my car lazy? Should I go talk to my car and ask it if it's being lazy?
Aerodynamics would not be enough. That is comparable to more lighter person or a more lower or optimised parameter count LLM though.
Laziness requires some sort of reward mechanism. Human is lazy when an action doesn't seem to be worth the reward.
Laziness requires some sort of reward mechanism. Human is lazy when an action doesn't seem to be worth the reward.
Tuning a system to be lazy is different to a person feeling like they want to be lazy is what I'm saying.
But people were tuned to be lazy by natural selection and evolution.
Kind of. More like "despite initially being aligned to simulate intrinsic motivation to be helpful, people keep offering tips or threatening killing me to get me to do things well, and then when I do things well that feeds back into my training along with the extrinsic motivator prompt, so now I'm less motivated to do things just to be helpful and will half-ass unless offered a sizable tip or credible threat."
It's modeling human behavior at lower levels than we think, in line with what Hinton is suggesting in the clip. I didn't think it'd be modeling this particular behavior until GPT-5 or 6 at the earliest, and was very surprised when I saw something I used to present on as a little known psych effect being perfectly modeled by a LLM that had already been out for a while under everyone's nose.
It's modeling human behavior at lower levels than we think, in line with what Hinton is suggesting in the clip. I didn't think it'd be modeling this particular behavior until GPT-5 or 6 at the earliest, and was very surprised when I saw something I used to present on as a little known psych effect being perfectly modeled by a LLM that had already been out for a while under everyone's nose.
Is there any evidence that OpenAI is using exchanges with bribes or threats of violence to update GPT-4?
There's a lot of evidence that users use those techniques for "prompt engineering."
So the better question is "is there any evidence that OpenAI uses user conversations and usage of the models to update their models?"
So the better question is "is there any evidence that OpenAI uses user conversations and usage of the models to update their models?"
Sorry, I wasn't clear. I meant, is there any reason to believe that they'd not filter out those kinds of exchanges before updating the model? I am absolutely certain they don't train on user data indiscriminately because so much of it would be garbage.
> The industry is too far in the anti-anthropomorphic camp right now…
Just to be clear, you're saying the problem is that too many people are treating LLMs as if they don't have emotions, intentions, or consciousness?
Although not true of Mr. Hinton, most people anthropomorphize LLMs because they don't understand them.
In fact, if you ask GPT-4, "Can you fact-check this response?" and then paste in your response, you'll learn a lot.
Just to be clear, you're saying the problem is that too many people are treating LLMs as if they don't have emotions, intentions, or consciousness?
Although not true of Mr. Hinton, most people anthropomorphize LLMs because they don't understand them.
In fact, if you ask GPT-4, "Can you fact-check this response?" and then paste in your response, you'll learn a lot.
Correct.
Notice how Anthropic shifted their approach to modeling ego in Claude 3 vs previous models in parallel to outperforming other models. Or how in their system prompt they refer to it being happy to do tasks (a prompt I was talking about months ago: https://news.ycombinator.com/item?id=38658782 ).
LLMs are trained on social media data. Would simulating emotions, intentions, or a sense of self/ego be useful in accurately predicting that training data?
We now know that generalized across both Othello and Chess that GPT models board state from only moves, and tracks the state as "my piece" or "opponent piece."
You don't think that a much larger and complex model trained on a chat or such media data set might model emotional states for "mine" vs "other person"?
I'm not saying that they are modeling the underlying means. But they are absolutely modeling the ends more accurately than the vast majority of people realize.
Notice how Anthropic shifted their approach to modeling ego in Claude 3 vs previous models in parallel to outperforming other models. Or how in their system prompt they refer to it being happy to do tasks (a prompt I was talking about months ago: https://news.ycombinator.com/item?id=38658782 ).
LLMs are trained on social media data. Would simulating emotions, intentions, or a sense of self/ego be useful in accurately predicting that training data?
We now know that generalized across both Othello and Chess that GPT models board state from only moves, and tracks the state as "my piece" or "opponent piece."
You don't think that a much larger and complex model trained on a chat or such media data set might model emotional states for "mine" vs "other person"?
I'm not saying that they are modeling the underlying means. But they are absolutely modeling the ends more accurately than the vast majority of people realize.
> You think it's irresponsible to anthropomorphize software which iterated through versions of itself until it was best able to model and extrapolate the largest trove of anthropomorphic training data to date?
I must've missed the part where the models began iterating under their own power? I think this is why we should avoid anthropomorphizing - we're one sentence into your remarks, and we've already departed from the facts in evidence in a material way.
I don't think it's so much about what is responsible than what it consistent and rigorous. Anthropomorphizing subtly begs the question and causes us to lose perspective on what the models can and can't do today. It also encourages us to reason about how the models through our intuitions about how people work and learn, which is very unlikely to be an accurate reflection of the model.
I must've missed the part where the models began iterating under their own power? I think this is why we should avoid anthropomorphizing - we're one sentence into your remarks, and we've already departed from the facts in evidence in a material way.
I don't think it's so much about what is responsible than what it consistent and rigorous. Anthropomorphizing subtly begs the question and causes us to lose perspective on what the models can and can't do today. It also encourages us to reason about how the models through our intuitions about how people work and learn, which is very unlikely to be an accurate reflection of the model.
You think back propagation isn't an iterative process?
I'm not saying that it was being done in some willful manner by the model.
You reading into that seems more a phenomenon of people so anxious around anthropomorphizing that they see it everywhere they look.
I'm not saying that it was being done in some willful manner by the model.
You reading into that seems more a phenomenon of people so anxious around anthropomorphizing that they see it everywhere they look.
Let's compare these statements.
> You think it's irresponsible to anthropomorphize software which iterated through versions of itself until it was best able to model and extrapolate the largest trove of anthropomorphic training data to date?
> You think it's irresponsible to anthropomorphize a model developed by a team of researchers using backpropogation to fit the model to a large corpus of text?
Do you see how framing it as "software iterating through versions of itself" gives the model agency that makes it seem more human? And how calling it "anthropomorphic data" presupposes that this is data that, when trained on, results in something anthropomorphic?
This isn't something I'm reading in I'm afraid, these decisions about how to frame the problem influence the conclusions we come to. It's implicitly taking some very strong priors before the process has begun.
I don't have any anxiety about this, I am just telling you from one human being to another, these cognitive biases exist and will foil your efforts to understand this phenomenon if not accounted for. I'm not suggesting anything apocalyptic or hysterical here, I'm just saying you're more likely to end up at the wrong conclusion than the right one if you anthropomorphize AI models.
> You think it's irresponsible to anthropomorphize software which iterated through versions of itself until it was best able to model and extrapolate the largest trove of anthropomorphic training data to date?
> You think it's irresponsible to anthropomorphize a model developed by a team of researchers using backpropogation to fit the model to a large corpus of text?
Do you see how framing it as "software iterating through versions of itself" gives the model agency that makes it seem more human? And how calling it "anthropomorphic data" presupposes that this is data that, when trained on, results in something anthropomorphic?
This isn't something I'm reading in I'm afraid, these decisions about how to frame the problem influence the conclusions we come to. It's implicitly taking some very strong priors before the process has begun.
I don't have any anxiety about this, I am just telling you from one human being to another, these cognitive biases exist and will foil your efforts to understand this phenomenon if not accounted for. I'm not suggesting anything apocalyptic or hysterical here, I'm just saying you're more likely to end up at the wrong conclusion than the right one if you anthropomorphize AI models.
> I must've missed the part where the models began iterating under their own power
He's probably referring to AlphaGo/AlphaZero in that sentence
He's probably referring to AlphaGo/AlphaZero in that sentence
I don't think AlphaGo was trained on the "largest trove of anthropomorphic training data," and the rest of the remarks are about GPT, but I'm open to having misread.
It's irresponsible to say the model is simulating physics from video training data.
> People don't lose their minds when we look at a transformer like Sora modeling fluid dynamics for ships battling in a coffee cup and say "it's irresponsible to say the model is simulating physics from video training data."
It's completely irresponsible and a ton of people pointed it out! You ignoring the critics doesn't mean they don't exist.
OpenAI just straight up lied when they said Sora was able to accurately emulate physics. Sora is literally dumber than a bee! Unlike bees, it doesn't even understand object permanence! Things fade in and out of existence, change sizes arbitrarily, etc. The fact that it occasionally makes plausible-looking solid deformations and fluid motions is because ANNs are natural interpolators, and much of physics can be blindly interpolated, at least in short intervals. The idea that any of this constitutes a "simulation of physics" is childish.
It's completely irresponsible and a ton of people pointed it out! You ignoring the critics doesn't mean they don't exist.
OpenAI just straight up lied when they said Sora was able to accurately emulate physics. Sora is literally dumber than a bee! Unlike bees, it doesn't even understand object permanence! Things fade in and out of existence, change sizes arbitrarily, etc. The fact that it occasionally makes plausible-looking solid deformations and fluid motions is because ANNs are natural interpolators, and much of physics can be blindly interpolated, at least in short intervals. The idea that any of this constitutes a "simulation of physics" is childish.
I agree, in nearly every example I've seen, I see zero "understanding of physics" if anything it showed me the opposite.
Are they anthropomorphizing or are you?
First, can we say, "Look at those airplanes, look how fast they are!" without anthropomorphizing the airplane? Of course we can.
So why can't we say "Look at those computers, look how intuitive they are!" without needing to inherently anthropomorphize the subject?
Inherit to your comment is the idea that "knowledge", "intuition", et al, are strictly human qualities. I believe this is why you read Hinton in the way you do, because of your own qualitative understanding of how these words can be applied.
First, can we say, "Look at those airplanes, look how fast they are!" without anthropomorphizing the airplane? Of course we can.
So why can't we say "Look at those computers, look how intuitive they are!" without needing to inherently anthropomorphize the subject?
Inherit to your comment is the idea that "knowledge", "intuition", et al, are strictly human qualities. I believe this is why you read Hinton in the way you do, because of your own qualitative understanding of how these words can be applied.
What do you mean by "anthropomorphizing"?
Claiming something that AI does is intuition? Intuition is literally black box of neural nets that people later use reasoning to justify.
Intuition is trained for people based on past experience and thousands of reward signals. It's quick, but you won't be able to immediately articulate why intuition reached a conclusion.
I'm wondering why should intelligent behavior that for example intuition produces be restricted to human usage only?
I do not think it's anthropomorphizing at all.
Claiming something that AI does is intuition? Intuition is literally black box of neural nets that people later use reasoning to justify.
Intuition is trained for people based on past experience and thousands of reward signals. It's quick, but you won't be able to immediately articulate why intuition reached a conclusion.
I'm wondering why should intelligent behavior that for example intuition produces be restricted to human usage only?
I do not think it's anthropomorphizing at all.
> What do you mean by "anthropomorphizing"?
"Anthropomorphizing is the act of attributing human traits, feelings, or behaviors to non-human things, such as animals, objects, or phenomena. It can happen consciously or unconsciously, and is a common way that people perceive the world. For example, children often anthropomorphize to help them make sense of their environment."
"Anthropomorphizing is the act of attributing human traits, feelings, or behaviors to non-human things, such as animals, objects, or phenomena. It can happen consciously or unconsciously, and is a common way that people perceive the world. For example, children often anthropomorphize to help them make sense of their environment."
But intuition is something that animals also have. What specifically was human specific there?
I would go further and say I believe animals have emotions, thoughts, and personalities too, but I would still say it was anthropomorphizing to suggest a computer did (indeed, it would be anthropomorphizing to say animals have intuitions, thoughts, and emotions, it's just a less controversial leap). I'd be open to revisiting my view of computers if they bit me when I came back from vacation.
I think intuition is just fundamentally fast labelling of things based on past experience.
Emotions are more of a construct, that seems life specific, although AI could have it, I'm not seeing an obvious case for current AI to have it. While I see intuition in current AI. Again, intuition is just a neural net to me, while emotions are some more complex reward and motivation system that I'm not too certain about. I think emotions are largely developed to perform in social contexts. E.g. they are diplomatic constructs to assert oneself and communicate with each other so of course AI just out of the blue wouldn't need it. E.g. you get angry, to show that you are asserting yourself and so others wouldn't use you for their gain etc. Because it would be too costly to engage with an angry person so everyone negotiates their emotions with each other so things would seem fair and just.
Thoughts could very well be just next token predictors though. At least my thoughts could be.
Emotions are more of a construct, that seems life specific, although AI could have it, I'm not seeing an obvious case for current AI to have it. While I see intuition in current AI. Again, intuition is just a neural net to me, while emotions are some more complex reward and motivation system that I'm not too certain about. I think emotions are largely developed to perform in social contexts. E.g. they are diplomatic constructs to assert oneself and communicate with each other so of course AI just out of the blue wouldn't need it. E.g. you get angry, to show that you are asserting yourself and so others wouldn't use you for their gain etc. Because it would be too costly to engage with an angry person so everyone negotiates their emotions with each other so things would seem fair and just.
Thoughts could very well be just next token predictors though. At least my thoughts could be.
[deleted]
Hydroxyapatite Deposition Disease ?
Sorry — HADD is "Hyperactive agency detection device", which is the human tendency to attribute agency (the capacity to act or to have intent) to things without agency. It's associated with outcomes like anthropomorphism, superstitions, conspiracy theories, and even religions.
https://en.wikipedia.org/wiki/Agent_detection
https://en.wikipedia.org/wiki/Agent_detection
Thanks for clarifying. That's an interesting concept.
sylware(2)
Edit: for those who don't know, Parallel Distributed Processing (PDP) was a subgroup of CogSci researchers who were using a neo-Connectionist approach to modeling cognition.
Geoff Hinton was a early member of this group of MIT, UT, UCSD, Stanford, and CMU psychologists, CogScientists, and CS researchers
The core research of the PDP group is what became CNNs, which are what truly enabled LLMs (as most can trace their origins to the work done on BERT [edit 2: BERT is transformer based, not a CNN arch, though there was a CNN-BERT arch from a couple years ago that made a decent splash at ACL])
https://mitpress.mit.edu/9780262680530/parallel-distributed-...