Why Mastering Language Is So Difficult for AI(undark.org)
undark.org
Why Mastering Language Is So Difficult for AI
https://undark.org/2022/10/07/interview-why-mastering-language-is-so-difficult-for-ai/
53 comments
Reading things like this on hacker news and twitter reminds me a lot like particle physicists unwilling to grapple with lack of evidence for SUSY. Every time someone points it out, these guys merely recite the holy texts (describe the model in gross detail with more magic words as if the criticism is because the critic don't understand it well enough), as if that addresses the critique at all. They then eventually gesture at the energy horizon, suggesting that "with more s[0] you'll eventually see my particles."
It's feels a bit similar here. There are clear limitations to AI by just NNs, namely they like Stable Diffusion must be trained on so much data, far more than any single artist will ever see, and they still fall short sometimes. With NLP it seems a little more obvious because the limitations seem to lag actual speech understood by humans still, even with the massive dataset again beyond what any single human will ever digest. There are clear issues here, merely reciting to critics what your model of intelligence is doesn't address the shortcomings these systems have which any literate person can perceive. And gesturing at the data horizon (you need even more data??) is not convincing either.
Apparently this cat (the interviewed author) has some baggage, but the things he's talking about (symbols) have already been studied and used by philosophers and linguists long before AI or computers were a thing. It helps to build upon what other researchers have already done, I don't know why NN researchers don't seem to map what they are doing to existing research on intelligence in other areas, namely cognitive science. Do NN researchers have any contact with linguists or cognitive scientists generally?
Again, being a little ignorant of his baggage, I do think a hybrid approach sounds like it would be promising. Symbolic AI failed in the 70s, NN is getting a bit of the way there now and is pretty useful but misses a few things, why go all in on just one type of approach?
[0] s is basically the center of mass energy, what particle physicists mean when they say the "energy" of an interaction (collision)
It's feels a bit similar here. There are clear limitations to AI by just NNs, namely they like Stable Diffusion must be trained on so much data, far more than any single artist will ever see, and they still fall short sometimes. With NLP it seems a little more obvious because the limitations seem to lag actual speech understood by humans still, even with the massive dataset again beyond what any single human will ever digest. There are clear issues here, merely reciting to critics what your model of intelligence is doesn't address the shortcomings these systems have which any literate person can perceive. And gesturing at the data horizon (you need even more data??) is not convincing either.
Apparently this cat (the interviewed author) has some baggage, but the things he's talking about (symbols) have already been studied and used by philosophers and linguists long before AI or computers were a thing. It helps to build upon what other researchers have already done, I don't know why NN researchers don't seem to map what they are doing to existing research on intelligence in other areas, namely cognitive science. Do NN researchers have any contact with linguists or cognitive scientists generally?
Again, being a little ignorant of his baggage, I do think a hybrid approach sounds like it would be promising. Symbolic AI failed in the 70s, NN is getting a bit of the way there now and is pretty useful but misses a few things, why go all in on just one type of approach?
[0] s is basically the center of mass energy, what particle physicists mean when they say the "energy" of an interaction (collision)
Yes, and the history of philosophy is replete with examples where the learned opinion on a subject stymied progress. Respect for one's intellectual forebearers is good, but being constrained by them has proven to be detrimental in areas where the learned opinion is largely speculative. The progress made by deep learning is so unlikely anything that came before, trying to shoehorn old methods and ontologies into the new paradigm would only harm progress.
How do you define "progress" in the history of philosophy? By some criteria we haven't made any progress in over 2000 years. There is no clear goal or success metric.
In my view, intelligibility is the basis on which we should judge a philosophical theory. Then we can rank theories by which are more intelligible or render more phenomena more intelligible. A theory that makes some phenomena transparent, i.e. conceptually reducible to other phenomena, is more intelligible than a competing theory. Progress in philosophy is the process by which opaque phenomena are rendered transparent. Dogma hinders this kind of progress.
It’s worth pointing out that you are not making a fair comparison when you compare the amount of data that a NN must parse to that of an individual when learning a specific new concept. A human brain is not just trained on their own lived experiences, but also the experiences of all their ancestors via the process of evolution, in addition to years and decades of general training and fine tuning.
You specifically mention Stable Diffusion, but SD has an analogue to the learning of an individual, textual inversion, which can adapt the NN to express new concepts with as few as a handful of examples.
You specifically mention Stable Diffusion, but SD has an analogue to the learning of an individual, textual inversion, which can adapt the NN to express new concepts with as few as a handful of examples.
"Every time I fire a linguist, the performance of my speech recognizer goes up"
You say far more than any artist but I m not so sure. When we spend 20 years looking at stuff before doing one passably interesting work of art (bar absolute outlying geniuses), it's not a small time nor a small training cost. It may feel easy for us but our brain never stop ingesting information for decades before being original enough.
It is impressive to me that the giant comment you replied to is factually wrong in one of its central premises about the amount of data consumed, and you are the one who gets negatives.
Our vision alone uncompressed is about 13TB per hour (!) (assuming 8k per eye resolution, "full color", and about 20hz sample rate). By the time a kid knows to play with a color book, they saw an equivalent of 4 PB of imagery.
For comparison Dall-E 2 only seen 175TB. And GPT-3 only a few TB.
Our vision alone uncompressed is about 13TB per hour (!) (assuming 8k per eye resolution, "full color", and about 20hz sample rate). By the time a kid knows to play with a color book, they saw an equivalent of 4 PB of imagery.
For comparison Dall-E 2 only seen 175TB. And GPT-3 only a few TB.
This NN breakthrough in language is only about 2 years old now [0].
People are working on your symbolic logic hybrid approach, but not much interesting has happened yet. This is a bit expected, since the short amount of time, and the fast scaling these language models have done in those last two years. It's been hard to outscale them with any other approach, let alone by the particular one of symbolic logic.
I think the main discripancy with particle physics, is that every iteration of bigger language models are also significantly better, no diminishing returns in sight.
Better for concrete applications as well. There is therefore no need to claim these models are scaled for scientific reasons in the search of AI, the accountants are happy as it is.
[0] https://arxiv.org/abs/2005.14165
People are working on your symbolic logic hybrid approach, but not much interesting has happened yet. This is a bit expected, since the short amount of time, and the fast scaling these language models have done in those last two years. It's been hard to outscale them with any other approach, let alone by the particular one of symbolic logic.
I think the main discripancy with particle physics, is that every iteration of bigger language models are also significantly better, no diminishing returns in sight.
Better for concrete applications as well. There is therefore no need to claim these models are scaled for scientific reasons in the search of AI, the accountants are happy as it is.
[0] https://arxiv.org/abs/2005.14165
You might be right that these "multimodal" transformers, by integrating additional data from non-text sources, would be more capable than GPT-3. But I don't think that invalidates Gary Marcus's point.
The word "model" is another of those words that Minsky called "suitcase words"--- they can be used to mean many things. I don't think Marcus is saying that that LLMs have "no model", just that they don't have a model of the type that a symbolic system could have. He gives many examples of deductions that a symbolic AI system can easily do, which GPT-3 is simply incapable of.
The word "model" is another of those words that Minsky called "suitcase words"--- they can be used to mean many things. I don't think Marcus is saying that that LLMs have "no model", just that they don't have a model of the type that a symbolic system could have. He gives many examples of deductions that a symbolic AI system can easily do, which GPT-3 is simply incapable of.
To hint at what I fundamentally mean by model I'd be interested to see a symbolic model of vision. E.g. take 5125123 numbers and give them names and then follow some rules to arrive at "cat" or "dog". Image recognition, I think, is demonstrably not symbolic. Likewise most transformations from the real world to model state are not symbolic. Within model state, symbolism may have uses but Church-Turing claims that it isn't necessary.
It seems clear to me that if the CLIP-like part of Imagen or Stable Diffusion can take an image made of pixels and yield "cat" and similarly take "dog" and produce a 3D neural radiance field that we can light just like any other 3D model and recognize as a dog then there must be an accurate and useful model of both how vision works and what dogs look like and the English relationship between those two things inside the machine.
I also wish Gary Marcus was replying to Google's Minerva paper instead of GPT-3. The ability to answer multi-step symbolic problems is basically here. https://ai.googleblog.com/2022/06/minerva-solving-quantitati...
I'd also note that decades of attempts at automated theorem proving with symbolic systems haven't yielded similarly impressive results. Now we have deep learning models helping with proof search.
It seems clear to me that if the CLIP-like part of Imagen or Stable Diffusion can take an image made of pixels and yield "cat" and similarly take "dog" and produce a 3D neural radiance field that we can light just like any other 3D model and recognize as a dog then there must be an accurate and useful model of both how vision works and what dogs look like and the English relationship between those two things inside the machine.
I also wish Gary Marcus was replying to Google's Minerva paper instead of GPT-3. The ability to answer multi-step symbolic problems is basically here. https://ai.googleblog.com/2022/06/minerva-solving-quantitati...
I'd also note that decades of attempts at automated theorem proving with symbolic systems haven't yielded similarly impressive results. Now we have deep learning models helping with proof search.
Well put on what language is. Language encodes the delta of worlds. There are way more unsaid than what is said explicitly.
Are you saying that GPT-3 doesn't grasp Object Permanence?
Replace GPT-3 with Bob, language model with person, and deep learning with learning, and you arrive at the conclusion that we're not able to determine whether people actually understand the world around them. Any utterance is indistinguishable from a sufficiently advanced computation model which simply produces the next token of text. Which is to say that the essential discriminating characteristic isn't the fact that GPT-3 runs on a computer, it's that it doesn't have a basis of reality rooted in the present.
The article correctly recognizes this, but then fails to actually make any meaningful contributions. Say the model was in fact conditioned on the basis of a present reality, would that change the interviewee's mind? The fact that it's difficult to say simply hints that the interviewer didn't really do that great of a job drilling down to the interesting questions.
The article correctly recognizes this, but then fails to actually make any meaningful contributions. Say the model was in fact conditioned on the basis of a present reality, would that change the interviewee's mind? The fact that it's difficult to say simply hints that the interviewer didn't really do that great of a job drilling down to the interesting questions.
I recall reading some advice to parents not to freak out if they discover their child lying at an early age. The sentiment was that lying is built on a mental model of another person, and therefore requires some substantial development of emotional intelligence to even make the attempt.
There's some romanticism in common between magicians, con artists, and fraudsters about how unclearly 'normal people' perceive the world so perhaps there's some truth to what you say. And the perpetual problem with artists and visionaries is that they see the world before everyone else does. For the one group they see things as they are now while everyone else seems to see things as they were 2, 5 years ago. For the other they see what the world could be, by exercising the tools near at hand.
One wonders then how an AI that actually understand things would be seen. As a trickster? A misunderstood intellectual? A madman?
There's some romanticism in common between magicians, con artists, and fraudsters about how unclearly 'normal people' perceive the world so perhaps there's some truth to what you say. And the perpetual problem with artists and visionaries is that they see the world before everyone else does. For the one group they see things as they are now while everyone else seems to see things as they were 2, 5 years ago. For the other they see what the world could be, by exercising the tools near at hand.
One wonders then how an AI that actually understand things would be seen. As a trickster? A misunderstood intellectual? A madman?
Except - as pointed out near the start of the interview - Bob knows that Biden is the current president. GPT-3 not only doesn't know, it doesn't know that it doesn't know. Despite having been trained on the entire internet, which talks quite a bit about who was president when.
Intelligence has been concisely defined as "the ability to profit from one's experiences." GPT-3 seems unable to profit from its ample experience being told who the president is.
People get so incredibly gullible when an exciting fantasy like sentient AI is in front of them. ELIZA was a laughably simple program and still fooled people.
Intelligence has been concisely defined as "the ability to profit from one's experiences." GPT-3 seems unable to profit from its ample experience being told who the president is.
People get so incredibly gullible when an exciting fantasy like sentient AI is in front of them. ELIZA was a laughably simple program and still fooled people.
> Replace GPT-3 with Bob, language model with person, and deep learning with learning, and you arrive at the conclusion that we're not able to determine whether people actually understand the world around them.
Sure we can, there are diseases which greatly hampers your ability to understand the world but doesn't hamper your verbal abilities at all, Williams syndrome for example. They are great at generating nonsense stories that doesn't make sense. GPT-3 is like that but much worse.
https://en.wikipedia.org/wiki/Williams_syndrome
Sure we can, there are diseases which greatly hampers your ability to understand the world but doesn't hamper your verbal abilities at all, Williams syndrome for example. They are great at generating nonsense stories that doesn't make sense. GPT-3 is like that but much worse.
https://en.wikipedia.org/wiki/Williams_syndrome
For a particularly striking example of the opposite problem, see people with Wernicke's Aphasia [1]. Here's an example [2] of a man who had a stroke that left him with the disorder. He can speak fluently but his speech is meaningless.
Byron can understand what's happening in the world around him and can communicate it through body language (just see the warmth in his eyes and his smile) but when he speaks everything comes out in a stream of nonsense.
If we tried to apply a language-model-centric view of intelligence we would rate Byron is unintelligent. That would clearly be a mistake, as anyone can plainly see that he is intelligent.
[1] https://en.wikipedia.org/wiki/Receptive_aphasia
[2] https://www.youtube.com/watch?v=3oef68YabD0
Byron can understand what's happening in the world around him and can communicate it through body language (just see the warmth in his eyes and his smile) but when he speaks everything comes out in a stream of nonsense.
If we tried to apply a language-model-centric view of intelligence we would rate Byron is unintelligent. That would clearly be a mistake, as anyone can plainly see that he is intelligent.
[1] https://en.wikipedia.org/wiki/Receptive_aphasia
[2] https://www.youtube.com/watch?v=3oef68YabD0
I would point put out that the symbolic structures that enhance han intelligence were implanted during that patient's early life and while the external connection to those structures may have been severed, that doesn't mean that the complicated semantic structures don't still exist.
The contrapositive would be to show a clearly intelligent human with no communication ability that can still model the human world accurately. Past such examples don't tend to show this, though the co morbidity with no early human contact and severe developmental nutritional deficiency makes it hard to be sure.
The contrapositive would be to show a clearly intelligent human with no communication ability that can still model the human world accurately. Past such examples don't tend to show this, though the co morbidity with no early human contact and severe developmental nutritional deficiency makes it hard to be sure.
[deleted]
Right, so there is a characteristic other than computational substrate that allows us to differentiate between systems that have this ability and those that don't.
Say we apply the diagnostic criteria of William's syndrome to language models and deduce that they meet the criteria for diagnosis, then the diagnostic itself ie: the test results are the grounding for the conclusion not the computational substrate. In that way the discussion surrounding the computation substrate is a red herring or at the very least a unclear and roundabout way of grouping of the characteristic we're actually interesting in.
Say we apply the diagnostic criteria of William's syndrome to language models and deduce that they meet the criteria for diagnosis, then the diagnostic itself ie: the test results are the grounding for the conclusion not the computational substrate. In that way the discussion surrounding the computation substrate is a red herring or at the very least a unclear and roundabout way of grouping of the characteristic we're actually interesting in.
I think implying that GPT-3 and a human learn a language in the same way is an error. AI "learning" is very rigid; while human learning is often a complicated mishmash of behaviour, social feedback, reasoning, and imagination.
> conditioned on the basis of a present reality
If we grant this hypothetical, we might find what it "says" fundamentally alien to us --- we might not even know it is saying anything at all.
> conditioned on the basis of a present reality
If we grant this hypothetical, we might find what it "says" fundamentally alien to us --- we might not even know it is saying anything at all.
Neural networks are modeled after the neurons in human brains, so by definition, the way NNs learn approximates the way humans learn.
Neural networks were modeled after a model of the neurons in human brains, a model which we know by now is inaccurate and misses a lot of details.
Airplane wings and tails bear about as much resemblance to bird wings and tails as neural networks do to neurons.
The way airplanes fly doesn't even vaguely resemble how birds fly. Same with neural networks and brains.
The way airplanes fly doesn't even vaguely resemble how birds fly. Same with neural networks and brains.
So, that's pretty wrong.
Spiking neural networks are possibly closer to being biologically analogies, but they kind of suck to train and don't have good performance.
It takes a lot of computer "neurons" to simulate a single human neuron, and we can't simulate it perfectly in any case.
Spiking neural networks are possibly closer to being biologically analogies, but they kind of suck to train and don't have good performance.
It takes a lot of computer "neurons" to simulate a single human neuron, and we can't simulate it perfectly in any case.
Well, yeah, that’s why it’s a model. Our weather models can’t predict the weather perfectly either.
How does the brain do backprop?
> Replace GPT-3 with Bob, language model with person, and deep learning with learning, and you arrive at the conclusion that we're not able to determine whether people actually understand the world around them.
Wouldn't the fact that people's facts do not match up prove that they do not?
Wouldn't the fact that people's facts do not match up prove that they do not?
[taunt]Most things that I hear about natural language processing boil down to a bunch of codemonkeying assumers trying to oversimplify linguistic phenomena that they are completely ignorant about, and yet trying to sell it as if it was the next best thing after sliced bread.[/taunt]
Anyway. There are three things that NLP is notoriously bad at:
1. Using world knowledge to interpret utterances in a given context. The article itself provides a neat example of that with United-Statian presidents.
2. Contextualisation that goes beyond the text that the utterance is found in.
3. Assessing the pragmatic purpose of an utterance.
But of course, "linguistics is useless for NLP!", the codemonkeys say.
Anyway. There are three things that NLP is notoriously bad at:
1. Using world knowledge to interpret utterances in a given context. The article itself provides a neat example of that with United-Statian presidents.
2. Contextualisation that goes beyond the text that the utterance is found in.
3. Assessing the pragmatic purpose of an utterance.
But of course, "linguistics is useless for NLP!", the codemonkeys say.
No, it unironically IS useless. You're welcome to bark up the symbolic tree along with Chomsky, Gary Marcus, and others who are about to fade into obscurity.
Or you can embrace the bitter lesson, which is that symbolic techniques are considered harmful, like the future winners of the AI race have: http://www.incompleteideas.net/IncIdeas/BitterLesson.html
Or you can embrace the bitter lesson, which is that symbolic techniques are considered harmful, like the future winners of the AI race have: http://www.incompleteideas.net/IncIdeas/BitterLesson.html
Chomsky has been saying that the way to do what we call AI is with statistics since about approximately before you were born. People wanted him to use his grammar ideas to research AI in the 50’s but he declined because he was interested in linguistics, not stats.
Not sure why you criticize a man for having different research interests than you in a completely different field.
Not sure why you criticize a man for having different research interests than you in a completely different field.
In the kindest interpretation of what he said, he might be referring to people trying to use Chomsky's ideas for AI language processing (plenty do), and implicitly restricting that "nobody will remember him" when it comes to this specific subject. (X-Bar will haunt us forever. /s)
But that's just me hoping, that there's anything worthwhile in the comment that you're replying to.
But that's just me hoping, that there's anything worthwhile in the comment that you're replying to.
I could evoke Brandolini's Law on this one, but for the sake of other posters, I won't.
>No, it unironically IS useless.
There's an example of why Linguistics isn't useless in _the very comment that you're replying to_: pragmatics. That is, modelling the _purpose_ of an utterance is essential to process it.
Without that pragmatic modelling, I believe that NLP will keep failing to deliver what it should, and it'll continue being just a cute toy to keep codemonkeys busy with their blackboxes. Granted, I may be wrong (I'm not stupid to claim certainty over the future), but I do not think that I am.
>You're welcome to bark up the symbolic tree along with Chomsky, Gary Marcus, and others who are about to fade into obscurity.
Damn, could you lend me that crystal ball that allows you to claim certainty over the future?*
It's perfectly possible that they fade into obscurity. It's also perfectly possible that they get hailed as heroes of language processing. Or anything in between. We, in 2022, are in no position to know it. The same applies to any other linguist that is interested on natural language processing.
Specifically on Chomsky. He often spouts some stupid reductionist shit and, paradoxically, unnecessary complexity to keep that reductionism intact. Even then, his approach looks saner than "NEEDZ MOAR NODES X-D".
>Or you can embrace the bitter lesson
"If I say so then you should accept it" is not a valid argument, sorry.
>which is that symbolic techniques are considered harmful, like the future winners of the AI race have
Another instance of "I KNOW THE FUTURE! CHRUST MEEEE" through your comment...
Checking the link,
>The ultimate reason for this is Moore's law
It's already common knowledge that Moore's Law is dead, and brute force only goes so far.
*That specific sentence is a great example highlighting the role of pragmatics. Humans can easily get what I did there; NLP should too, but it doesn't.
>No, it unironically IS useless.
There's an example of why Linguistics isn't useless in _the very comment that you're replying to_: pragmatics. That is, modelling the _purpose_ of an utterance is essential to process it.
Without that pragmatic modelling, I believe that NLP will keep failing to deliver what it should, and it'll continue being just a cute toy to keep codemonkeys busy with their blackboxes. Granted, I may be wrong (I'm not stupid to claim certainty over the future), but I do not think that I am.
>You're welcome to bark up the symbolic tree along with Chomsky, Gary Marcus, and others who are about to fade into obscurity.
Damn, could you lend me that crystal ball that allows you to claim certainty over the future?*
It's perfectly possible that they fade into obscurity. It's also perfectly possible that they get hailed as heroes of language processing. Or anything in between. We, in 2022, are in no position to know it. The same applies to any other linguist that is interested on natural language processing.
Specifically on Chomsky. He often spouts some stupid reductionist shit and, paradoxically, unnecessary complexity to keep that reductionism intact. Even then, his approach looks saner than "NEEDZ MOAR NODES X-D".
>Or you can embrace the bitter lesson
"If I say so then you should accept it" is not a valid argument, sorry.
>which is that symbolic techniques are considered harmful, like the future winners of the AI race have
Another instance of "I KNOW THE FUTURE! CHRUST MEEEE" through your comment...
Checking the link,
>The ultimate reason for this is Moore's law
It's already common knowledge that Moore's Law is dead, and brute force only goes so far.
*That specific sentence is a great example highlighting the role of pragmatics. Humans can easily get what I did there; NLP should too, but it doesn't.
Saved you a click: it's an interview with Gary Marcus.
Saved you another click if you also have never heard of this person: https://en.wikipedia.org/wiki/Gary_Marcus
While language seemed difficult for AI for a while, it now seems to be falling quite rqpidly to recent techniques.
(It reminds me a little of the early years of aviation. In 1901 the US Navy's chief engineer called manned flight a "vain fantasy", & in 1903 the New York Times suggested flying machines would be possible in "one million to 10 million years". But it was accomplished later in 1903, with rapid improvement after that. Things that seem impossible now can become humdrum in a few decades.)
What are the current best results from Marcus, or others using his preferred approaches?
(It reminds me a little of the early years of aviation. In 1901 the US Navy's chief engineer called manned flight a "vain fantasy", & in 1903 the New York Times suggested flying machines would be possible in "one million to 10 million years". But it was accomplished later in 1903, with rapid improvement after that. Things that seem impossible now can become humdrum in a few decades.)
What are the current best results from Marcus, or others using his preferred approaches?
Mastering language is difficult for humans. I don't think most people realize how often they just guess at meanings, and how often they are wrong. Especially in spoken language.
Constantly asking for clarification is exhausting and gets you labeled as pedantic and "difficult". So most people don't do it. I'd guess that a fairly high percentage of human communication results in a misunderstanding. This is largely masked by human inaction or ability to disguise their lack of understanding. The AIs have no ego and no incentive to hide their confusion.
Constantly asking for clarification is exhausting and gets you labeled as pedantic and "difficult". So most people don't do it. I'd guess that a fairly high percentage of human communication results in a misunderstanding. This is largely masked by human inaction or ability to disguise their lack of understanding. The AIs have no ego and no incentive to hide their confusion.
Thankfully someone is pushing against the hype, good for him. Thank god.
It really frustrates me when people, particularly people on here specifically respond to criticism of NNs and the like by saying nonsense like "but YOUR brain is a neural net!" as if all your brain is but mimickry after all and there's nothing else. There is this need to reduce all the mind's complexity down to a model that you seem to understand, just because you understand it, which seems like such a lazy, reductionist way to think. Reductionism in general has value but it tends to narrow the mind.
I feel like Gary Marcus is right, you don't just need the mimickry even if it plays a role, you do need "symbolic" understanding somewhere, and that makes it easier to determine whether Trump is president vs. Biden at the moment, as they talk about in the article. You do need concepts (symbols) deep down somewhere, it seems so bizarre to think you don't because no one even in their own lives lives that way or thinks that way. I mean literally right now, me reading and writing comments on hacker news is all about ideas and concepts, symbols in the philosophical sense. I feel like even if it started as mimickry when I was an infant, eventually the mimickry takes a life of its own, I figure out concepts themselves, and I no longer even work in mere mimickry, I work with symbols.
It reminds me of Baudrillard's concept (unintended pun, I think) of simulacra, it's not that simulacra are just representations of the real and are thus fake, like money represents some amount of food or gold or something and so money is fake. Money, as we all know, is in fact very real! Our daily lives are shaped around it, they shape our reality, things like loans, credit, bank accounts, stock markets eventually become things we care about. Money is therefore hardly fake. Eventually the simulacra develop a reality of their own, in fact, they become hyperreal.
Ideas are the same, and any real AI will need them.
It really frustrates me when people, particularly people on here specifically respond to criticism of NNs and the like by saying nonsense like "but YOUR brain is a neural net!" as if all your brain is but mimickry after all and there's nothing else. There is this need to reduce all the mind's complexity down to a model that you seem to understand, just because you understand it, which seems like such a lazy, reductionist way to think. Reductionism in general has value but it tends to narrow the mind.
I feel like Gary Marcus is right, you don't just need the mimickry even if it plays a role, you do need "symbolic" understanding somewhere, and that makes it easier to determine whether Trump is president vs. Biden at the moment, as they talk about in the article. You do need concepts (symbols) deep down somewhere, it seems so bizarre to think you don't because no one even in their own lives lives that way or thinks that way. I mean literally right now, me reading and writing comments on hacker news is all about ideas and concepts, symbols in the philosophical sense. I feel like even if it started as mimickry when I was an infant, eventually the mimickry takes a life of its own, I figure out concepts themselves, and I no longer even work in mere mimickry, I work with symbols.
It reminds me of Baudrillard's concept (unintended pun, I think) of simulacra, it's not that simulacra are just representations of the real and are thus fake, like money represents some amount of food or gold or something and so money is fake. Money, as we all know, is in fact very real! Our daily lives are shaped around it, they shape our reality, things like loans, credit, bank accounts, stock markets eventually become things we care about. Money is therefore hardly fake. Eventually the simulacra develop a reality of their own, in fact, they become hyperreal.
Ideas are the same, and any real AI will need them.
Oh, he loves to push against the hype alright. To his credit, he’s very smart - I think he sold an AI startup to Uber several years ago. And I agree with a lot of his criticisms and general AI philosophy.
Still, it’s impossible for me to shake the feeling that he just likes to hear himself talk and doesn’t want to see progress in AI.
I think I trust him more than Sam Altman though, not that anyone’s being asked to make that judgement.
Still, it’s impossible for me to shake the feeling that he just likes to hear himself talk and doesn’t want to see progress in AI.
I think I trust him more than Sam Altman though, not that anyone’s being asked to make that judgement.
> Still, it’s impossible for me to shake the feeling that [Gary Marcus] just likes to hear himself talk and doesn’t want to see progress in AI.
He says he wants to see progress in AI. He wants people to get their heads out of the neural network cul-de-sac and use other tools, too. His quote:
"Imagine a world in which iron makers shouted “iron,” and carbon lovers shouted “carbon,” and nobody ever thought to combine the two [to make steel]; that’s much of what the history of modern artificial intelligence is like." [1]
An adjustment will happen when the limitations of these purely statistical methods play out and burn a few self-driving car investors. Unfortunately, it will also probably taint the entire field of AI and start another AI winter for a decade or so.
[1] https://nautil.us/deep-learning-is-hitting-a-wall-238440/
He says he wants to see progress in AI. He wants people to get their heads out of the neural network cul-de-sac and use other tools, too. His quote:
"Imagine a world in which iron makers shouted “iron,” and carbon lovers shouted “carbon,” and nobody ever thought to combine the two [to make steel]; that’s much of what the history of modern artificial intelligence is like." [1]
An adjustment will happen when the limitations of these purely statistical methods play out and burn a few self-driving car investors. Unfortunately, it will also probably taint the entire field of AI and start another AI winter for a decade or so.
[1] https://nautil.us/deep-learning-is-hitting-a-wall-238440/
Thanks, that's a worthy addition (and good example of what I mean about agreeing with his philosophy). I was probably a bit too uncharitable in saying he doesn't want to see progress.
Oh yes and it can be annoying. Especially bad it is for any other Language than English. Every evening me and my children have problems like this: "Alexa spiele Paw Patrol Folge 67 auf Spotify" "OK Paw patrol Folge 122 auf Spotify wird abgespielt..." "ALEXA STOP!!1"
I have no accent or such, plain standard German,same for my wife and Kids - Alexa drives us crazy with wrong understood episodes and sometimes titles.
I have no accent or such, plain standard German,same for my wife and Kids - Alexa drives us crazy with wrong understood episodes and sometimes titles.
Perhaps it's just me, but if I ask myself whether or not the comments on this post, or the article itself, was written by GPT-3, I'm unable to decide one way or the other.
DreamFusion can learn 3d shapes without a 3d dataset.
Next step is to build a model to learn 3d motion from videos.
This should get us closer to mastering language.
[deleted]
It's nice to see some coverage of AI that is realistic about the issues.
Ask yourself that can you master something without understanding it?
TL;DR "A parrot’s not a bad metaphor, because we don’t think parrots actually understand what they’re talking about. And GPT-3 certainly does not understand what it’s talking about."
BTW a parrot knows a big deal about the world. Not much about human language but still more about language than GPT-3. Of course a 737 doesn't know how to fly but it still flies people around the world. Landing on a branch, not much.
BTW a parrot knows a big deal about the world. Not much about human language but still more about language than GPT-3. Of course a 737 doesn't know how to fly but it still flies people around the world. Landing on a branch, not much.
Grandmother mentioned her aunt had a parrot. One time she and her aunt are hanging out. There was a lineman up on the pole in front of the house and the bird was curiously watching him. After a bit a young lady walks by and the bird chuckles and gives out a loud wolf whistle. And the lady then begins to cuss out the lineman up on the pole. The bird giggled and then giggled off and on for the rest of the afternoon.
GPT-3 is closer to a talking toaster than a Parrot.
GPT-3 is closer to a talking toaster than a Parrot.
Talkie's the name, toasting's the game. Anyone like any toast?
The central thesis of his argument, that large language models are capable only of large-scale mimicry, is flat-out wrong.
> I think that people are led to believe that this system actually understands human language, which it certainly does not. What it really is, is an autocomplete system that predicts next words and sentences. Just like with your phone, where you type in something and it continues. It doesn’t really understand the world around it. And a lot of people are confused by that.
> They’re confused by that because what these systems are ultimately doing is mimicry. They’re mimicking vast databases of text. And I think the average person doesn’t understand the difference between mimicking 100 words, 1,000 words, a billion words, a trillion words — when you start approaching a trillion words, almost anything you can think of is already talked about there. And so when you’re mimicking something, you can do that to a high degree, but it’s still kind of like being a parrot, or a plagiarist, or something like that. A parrot’s not a bad metaphor, because we don’t think parrots actually understand what they’re talking about. And GPT-3 certainly does not understand what it’s talking about.
I think this is an easy accusation to disprove, and we can do so with a battery of simple tests!
To test whether a language-model is only constrained by mimicry, we can just give it a task that it has never seen before. One of my favorite tasks is the construction of novel analogies.
Here's a kind of "template" that I use often for prompting novel analogies...
PROMPT: You might not realize at first, but “________” and “________” actually have a lot in common. For example,
You can fill-in-the-blanks with various different phrases and see how well the LLM is able to construct a coherent analogy. I have an LLM running in a beta environment (trained on the literary corpus at prosecraft.io), so I'll ask it to run a few of these tests for me.
For example, here's what we get when we ask for an analogy between “writing poetry” and “driving a race car”...
PROMPT + COMPLETION: You might not realize at first, but “writing poetry” and “driving a race car” actually have a lot in common. For example, your first time in a race car, you’re scared. But after a few races, you’re confident. Or in poetry, the first time you write a poem, you’re scared and unsure. But after you’ve read and improved the poem many times, you’re confident. And you are a better writer because of it.
I'm pretty sure that the training corpus doesn't include any metaphors about writing poetry and driving race cars, so the model is relying on a hierarchy of connected abstractions in order to navigate a novel pathway between two disparate concepts. In this case, the central connective-tissue of the analogy is the fact that writing poetry and driving race cars can be scary for beginners, but with more practice, the practitioner increases their skill and confidence.
It's truly astonishing that a system comprised primarily of matrix multiplications on word vectors is capable of achieving this level of abstraction on such a complex cognitive task, but there ya go.
That's what the "deep" in "deep learning" refers to... The LLM training process decomposes language understanding into a hierarchy of layered composite concepts, and the inferencing process can use those different layers of meaning to compose and manipulate novel concepts.
With the "transformer" architecture, and the "self-attention" mechanisms, an LLM is capable of learning a vocabulary of symbols, a hierarchy of relations between those symbols, and a set of operators for transforming those symbols and relations.
We can try it again with “playing chess” and “rock climbing”:
PROMPT + COMPLETION: You might not realize at first, but “playing chess” and “rock climbing” actually have a lot in common. For example, at first glance, chess is a fairly abstract and rigid game. But as you play and gain skill, you realize that certain situations require a certain decision from you. For example, you realize that certain sequences of moves are either bad or good depending on your opponent’s next move. And as your skill increases, you find the moves you’re used to thinking are bad now can be good, and vice versa. The same is true with rock climbing. And with each successive attempt, you realize how much you’re actually improvising in terms of where you place your feet on the wall, how you orient your body, what holds you, where you lean.
In this example, the AI clearly shows that it has an underlying real-world model for the meaning of both chess-playing and rock-climbing, and it was able to find a semantic parallel between those concepts: that they both involve improvising a sequence of strategic moves toward a goal.
Pretty cool, huh?!
It's true that there are still a lot of missing capabilities in these models. For example, a temporal model would certainly help LLMs answer questions about history and current events more accurately (Who is the current president of the US? Who was the previous president?)
Likewise, since most LLMs operate at the "token" level, they actually don't understand how most words are spelled or pronounced, and they're not good at answering questions about rhythm, rhyme, alliteration, etc. For example...
PROMPT: The word “whale” rhymes with the word
COMPLETION: “blue”
Huh? Obviously, the model doesn't have enough information about the pronunciation of words to solve this problem. But if we trained a model jointly on language-as-text and language-as-speech, then there's no reason the model couldn't integrate those sensory data into a unified model. And again, it would be a multi-layered hierarchical model. Not just a lookup-table to be used for mimicry.
If you've been following along with CLIP (VQGAN, CLIP-guided diffusion, Stable Diffusion, etc), which is a joint-sensory model trained on text + imagery, it's easy to imagine future projects that might train joint-sensory models on other modalities (voice, music, locomotion, time, physics, mathematics, etc).
There's no reason those models can't be trained too!
But it makes sense to start with text and imagery, and then bootstrap the other models from there.
> I think that people are led to believe that this system actually understands human language, which it certainly does not. What it really is, is an autocomplete system that predicts next words and sentences. Just like with your phone, where you type in something and it continues. It doesn’t really understand the world around it. And a lot of people are confused by that.
> They’re confused by that because what these systems are ultimately doing is mimicry. They’re mimicking vast databases of text. And I think the average person doesn’t understand the difference between mimicking 100 words, 1,000 words, a billion words, a trillion words — when you start approaching a trillion words, almost anything you can think of is already talked about there. And so when you’re mimicking something, you can do that to a high degree, but it’s still kind of like being a parrot, or a plagiarist, or something like that. A parrot’s not a bad metaphor, because we don’t think parrots actually understand what they’re talking about. And GPT-3 certainly does not understand what it’s talking about.
I think this is an easy accusation to disprove, and we can do so with a battery of simple tests!
To test whether a language-model is only constrained by mimicry, we can just give it a task that it has never seen before. One of my favorite tasks is the construction of novel analogies.
Here's a kind of "template" that I use often for prompting novel analogies...
PROMPT: You might not realize at first, but “________” and “________” actually have a lot in common. For example,
You can fill-in-the-blanks with various different phrases and see how well the LLM is able to construct a coherent analogy. I have an LLM running in a beta environment (trained on the literary corpus at prosecraft.io), so I'll ask it to run a few of these tests for me.
For example, here's what we get when we ask for an analogy between “writing poetry” and “driving a race car”...
PROMPT + COMPLETION: You might not realize at first, but “writing poetry” and “driving a race car” actually have a lot in common. For example, your first time in a race car, you’re scared. But after a few races, you’re confident. Or in poetry, the first time you write a poem, you’re scared and unsure. But after you’ve read and improved the poem many times, you’re confident. And you are a better writer because of it.
I'm pretty sure that the training corpus doesn't include any metaphors about writing poetry and driving race cars, so the model is relying on a hierarchy of connected abstractions in order to navigate a novel pathway between two disparate concepts. In this case, the central connective-tissue of the analogy is the fact that writing poetry and driving race cars can be scary for beginners, but with more practice, the practitioner increases their skill and confidence.
It's truly astonishing that a system comprised primarily of matrix multiplications on word vectors is capable of achieving this level of abstraction on such a complex cognitive task, but there ya go.
That's what the "deep" in "deep learning" refers to... The LLM training process decomposes language understanding into a hierarchy of layered composite concepts, and the inferencing process can use those different layers of meaning to compose and manipulate novel concepts.
With the "transformer" architecture, and the "self-attention" mechanisms, an LLM is capable of learning a vocabulary of symbols, a hierarchy of relations between those symbols, and a set of operators for transforming those symbols and relations.
We can try it again with “playing chess” and “rock climbing”:
PROMPT + COMPLETION: You might not realize at first, but “playing chess” and “rock climbing” actually have a lot in common. For example, at first glance, chess is a fairly abstract and rigid game. But as you play and gain skill, you realize that certain situations require a certain decision from you. For example, you realize that certain sequences of moves are either bad or good depending on your opponent’s next move. And as your skill increases, you find the moves you’re used to thinking are bad now can be good, and vice versa. The same is true with rock climbing. And with each successive attempt, you realize how much you’re actually improvising in terms of where you place your feet on the wall, how you orient your body, what holds you, where you lean.
In this example, the AI clearly shows that it has an underlying real-world model for the meaning of both chess-playing and rock-climbing, and it was able to find a semantic parallel between those concepts: that they both involve improvising a sequence of strategic moves toward a goal.
Pretty cool, huh?!
It's true that there are still a lot of missing capabilities in these models. For example, a temporal model would certainly help LLMs answer questions about history and current events more accurately (Who is the current president of the US? Who was the previous president?)
Likewise, since most LLMs operate at the "token" level, they actually don't understand how most words are spelled or pronounced, and they're not good at answering questions about rhythm, rhyme, alliteration, etc. For example...
PROMPT: The word “whale” rhymes with the word
COMPLETION: “blue”
Huh? Obviously, the model doesn't have enough information about the pronunciation of words to solve this problem. But if we trained a model jointly on language-as-text and language-as-speech, then there's no reason the model couldn't integrate those sensory data into a unified model. And again, it would be a multi-layered hierarchical model. Not just a lookup-table to be used for mimicry.
If you've been following along with CLIP (VQGAN, CLIP-guided diffusion, Stable Diffusion, etc), which is a joint-sensory model trained on text + imagery, it's easy to imagine future projects that might train joint-sensory models on other modalities (voice, music, locomotion, time, physics, mathematics, etc).
There's no reason those models can't be trained too!
But it makes sense to start with text and imagery, and then bootstrap the other models from there.
People who don't believe the LLMs have a world model are wrong because they are mistaking a physically weak world model for no world model. GPT-3 doesn't understand physics well enough to embed models of the referents of language into a unified model that has accurate gravity and motion dynamics, so it maintains a much more dreamlike model where objects exist in scenes and have relationships to each other but those relationships are governed by literary relationships instead of physical ones and so contradictions and superpositions and causality violations are allowed in the model. As multimodal transformers like Gato get trained on more combined language and sensory input their world models will become much more physically and causally accurate which will be reflected in their accuracy on NLP tasks.