How Organisms Come to Know the World: Fundamental Limits on AGI(frontiersin.org)
frontiersin.org
How Organisms Come to Know the World: Fundamental Limits on AGI
https://www.frontiersin.org/articles/10.3389/fevo.2021.806283/full
156 comments
I think the authors' (flawed) bulwark to this would be that their notion of "agency" requires a system with a well defined boundary -- in other words essential entropy cannot come from outside the system. Section 3 spends a lot of words on this concept of agency and closure.
> The most central distinction to be made here is that the selection of a specific behavior is not purely reactive, not entirely determined by environmental conditions, but (at least partially) originates from and depends on the internal organization of the system making the selection.... [Agency] requires organizational closure .... Organisms, as autonomous agents, are Kantian wholes, i. e., organized beings with the property that the parts exist for and by means of the whole.
What's funny is that I can't imagine how a serious modern biologist signed off on any claims of closure for biological systems. Our own behavior arises out of a mess of genetics, epigenetics, symbiotic relationships, and other external feedback loops which intermingle the "agency" of many self-preserving and self-replicating systems, from transposons hijacking our genetic strands to gut biomes regulating our hormonal balances.
The history of biology is essentially one of humans drawing labels around supposed units, and nature making a mockery of those boundaries upon further research. Like other efforts before, this particular definition of closed agency is more a projection of our own psychology than an observation of the natural world.
> The most central distinction to be made here is that the selection of a specific behavior is not purely reactive, not entirely determined by environmental conditions, but (at least partially) originates from and depends on the internal organization of the system making the selection.... [Agency] requires organizational closure .... Organisms, as autonomous agents, are Kantian wholes, i. e., organized beings with the property that the parts exist for and by means of the whole.
What's funny is that I can't imagine how a serious modern biologist signed off on any claims of closure for biological systems. Our own behavior arises out of a mess of genetics, epigenetics, symbiotic relationships, and other external feedback loops which intermingle the "agency" of many self-preserving and self-replicating systems, from transposons hijacking our genetic strands to gut biomes regulating our hormonal balances.
The history of biology is essentially one of humans drawing labels around supposed units, and nature making a mockery of those boundaries upon further research. Like other efforts before, this particular definition of closed agency is more a projection of our own psychology than an observation of the natural world.
Indeed, and I think we can make a couple of specific responses to the passage you quote.
The first thing is that if selection of a specific behavior is determined by environmental together with the internal organization of the system making the selection, how is that not reactive, why would it matter if it isn't, and why does that not apply to robots?
The second point is that the internal organization of organisms is a result of evolution, so it too has been determined by the environment, and, furthermore, by a process that can be - and is - used to develop programs.
There are heavy doses of ontological argumentation here ("this is an X and that is a Y"), but our categories are determined by what we know (or think we do.) As you say, nature can make a mockery of those boundaries.
The first thing is that if selection of a specific behavior is determined by environmental together with the internal organization of the system making the selection, how is that not reactive, why would it matter if it isn't, and why does that not apply to robots?
The second point is that the internal organization of organisms is a result of evolution, so it too has been determined by the environment, and, furthermore, by a process that can be - and is - used to develop programs.
There are heavy doses of ontological argumentation here ("this is an X and that is a Y"), but our categories are determined by what we know (or think we do.) As you say, nature can make a mockery of those boundaries.
I am not a biologist, but I have read Maturana and Varela (originators of autopoesis), and they are not that naive. If there’s anything they do understand, it is the co-development of things over time in mutual relationship, ie structural coupling.
It’s funny to me to see this juxtaposed with the arguments from the other side:
Society had long argued that animals were not intelligent, and that instincts were deterministic.
Society had long argued that animals were not intelligent, and that instincts were deterministic.
Instincts in animals (including humans) are deterministic insofar as they determine the behavior of the animal. A bird may choose one stick or another but it can't choose not to build a nest. A human can. This may be unique to humans. Heidegger refers to this as world-openness, meaning our world is not closed by our instincts. Unlike most animals, humans don't possess enough instincts to survive in nature. We are "insufficiently-determined". Hence, to survive, we augment our nature with our culture, which acts as a second nature within which we can act unthinkingly (as animals do). Without this second nature we shall surely die (hence the furious culture wars). Not incidentally, this is one of the anthropological arguments against state of nature political theories: there's no such thing and never has been in the history of the species.
Animals may be more or less undetermined but a bee will make honey and an ant will gather food regardless of any individual instance of bee or ant intelligence.
The problem with the above objection is that there is no account of "meaning" in what an AGI does. In other words, it's still a computer. To say that the entropy of the system will determine the entity's indeterminism is akin to arguments saying Schrodinger's cat is evidence for the possibility of human free will. It's a category error. Unless you want to argue that humans are also computers (which is the position of many strict materialists) then there is still a huge gap to account for in terms of meaning that can't be defined away satisfyingly. We all know what it's like to be an intelligence. At best, people say we can't know if an AGI also knows that because we can't inspect their "minds". This argument is justified by recourse to the principle of the multiple instantiability of intelligence. If we admit that non-carbon-based life forms could be intelligent (e.g., extraterrestrials) then we may admit that intelligence could be in silica as well. Without the ability to introspect other minds, we assume they are like ours and not simply "zombies". The above arguments (reducible to "Well, how do you KNOW anyone else has a mind?") are based in a kind of skeptical solipsism. The part about intelligence being demarcated by non-determinism and the way that argument fails is a side-step of the hard problem of consciousness.
Animals may be more or less undetermined but a bee will make honey and an ant will gather food regardless of any individual instance of bee or ant intelligence.
The problem with the above objection is that there is no account of "meaning" in what an AGI does. In other words, it's still a computer. To say that the entropy of the system will determine the entity's indeterminism is akin to arguments saying Schrodinger's cat is evidence for the possibility of human free will. It's a category error. Unless you want to argue that humans are also computers (which is the position of many strict materialists) then there is still a huge gap to account for in terms of meaning that can't be defined away satisfyingly. We all know what it's like to be an intelligence. At best, people say we can't know if an AGI also knows that because we can't inspect their "minds". This argument is justified by recourse to the principle of the multiple instantiability of intelligence. If we admit that non-carbon-based life forms could be intelligent (e.g., extraterrestrials) then we may admit that intelligence could be in silica as well. Without the ability to introspect other minds, we assume they are like ours and not simply "zombies". The above arguments (reducible to "Well, how do you KNOW anyone else has a mind?") are based in a kind of skeptical solipsism. The part about intelligence being demarcated by non-determinism and the way that argument fails is a side-step of the hard problem of consciousness.
Humans actually take very little to non relevant decisions. The "free will" is mostly an illusion, you "choose" between equivalent options, not between actually different "life paths".
You can "decide" you won't "build a nest", but you'll certainly look for refuge if it is raining, or it's very cold. In the end, the human behavior is actually fully deterministic.
I'm sure FAANG already know this because they have the datapoints from billions of humans beings doing exactly the same stuff everyday for decades now.
This is probably how LLM can extrapolate all the things they "know" from human generated text: their tokens reflect the human behavior determinism a its fullest expression
(hence at more data for training, much better/closer their behavior ressemble the human behavior/ideas)
You can "decide" you won't "build a nest", but you'll certainly look for refuge if it is raining, or it's very cold. In the end, the human behavior is actually fully deterministic.
I'm sure FAANG already know this because they have the datapoints from billions of humans beings doing exactly the same stuff everyday for decades now.
This is probably how LLM can extrapolate all the things they "know" from human generated text: their tokens reflect the human behavior determinism a its fullest expression
(hence at more data for training, much better/closer their behavior ressemble the human behavior/ideas)
I can't agree with you although I can understand why you believe these things. Human instinct is, as I say, insufficiently-determined for survival. Is "free will" mostly an illusion? Or is it "fully deterministic"? If it is mostly then it is not "fully" determined. I'd be curious to hear how you think that can be and whether you make decisions in your own life or if you are resigned to your fate.
In any case, I'm not saying individual humans make many reasoned out decisions. Quite the opposite. Human individuals adopt a cultural second nature bequeathed to them by their heritage to avoid the necessity of making decisions. The culture over time adopts certain default decisions that are advantageous to survival. These are inculcated into the individual via their cultural conditioning. A baby has no culture and very few instincts (hunger, pleasure, pain, disgust). If you think humans are operating at the lower levels of instinct then why do they pursue or avoid certain behaviors that change over time? One answer is that they are responding to acceptable norms of behavior. As those norms change, they adapt and adopt. They change with the times, as the saying goes. A tiger can be trained to behave a certain way using stimulus/reward training but it doesn't self-regulate according to new norms. One could argue that elephants do this, if properly conditioned with a large stake early in life. But you could also make the case that elephants (like dolphins) are, like humans, self-conscious to a degree. I don't know if this is true, but it is an interesting thought. A more cynical person might say the elephant is not choosing anything but has been trained just like the tiger to avoid pain.
LLMs, or let us say AGI based on probabilistic models, can mimic behavior represented in its data and is very impressive at these language games and, with the right feedback, can correct some of its more glaring productions but it must ever be a parasite on human cultural production for it to gain what you call "knowledge". As a thought experiment, suppose we could set an LLM running on autopilot and send it into space with a feedback loop where it could add its productions to its source corpora. Is there any point in the future where it is generating new and coherent output? Or does it degenerate into raving madness?
In any case, I'm not saying individual humans make many reasoned out decisions. Quite the opposite. Human individuals adopt a cultural second nature bequeathed to them by their heritage to avoid the necessity of making decisions. The culture over time adopts certain default decisions that are advantageous to survival. These are inculcated into the individual via their cultural conditioning. A baby has no culture and very few instincts (hunger, pleasure, pain, disgust). If you think humans are operating at the lower levels of instinct then why do they pursue or avoid certain behaviors that change over time? One answer is that they are responding to acceptable norms of behavior. As those norms change, they adapt and adopt. They change with the times, as the saying goes. A tiger can be trained to behave a certain way using stimulus/reward training but it doesn't self-regulate according to new norms. One could argue that elephants do this, if properly conditioned with a large stake early in life. But you could also make the case that elephants (like dolphins) are, like humans, self-conscious to a degree. I don't know if this is true, but it is an interesting thought. A more cynical person might say the elephant is not choosing anything but has been trained just like the tiger to avoid pain.
LLMs, or let us say AGI based on probabilistic models, can mimic behavior represented in its data and is very impressive at these language games and, with the right feedback, can correct some of its more glaring productions but it must ever be a parasite on human cultural production for it to gain what you call "knowledge". As a thought experiment, suppose we could set an LLM running on autopilot and send it into space with a feedback loop where it could add its productions to its source corpora. Is there any point in the future where it is generating new and coherent output? Or does it degenerate into raving madness?
"I'd be curious to hear how you think that can be and whether you make decisions in your own life or if you are resigned to your fate."
haha just living life as usual, whatever internal stuff is feeding my main loop.
I left the deep thoughts to threads like this one and/or people more qualified.
haha just living life as usual, whatever internal stuff is feeding my main loop.
I left the deep thoughts to threads like this one and/or people more qualified.
Forgive the presumption, but I really think you should abandon that model of self understanding. Fatalism is a kind of disease that can be treated if caught early but is deadly if left undiagnosed. I don't believe you are really serious but I think it's important to not live hypocritically, which such a view makes unavoidable. But joking aside, one problem with thinking of your self as a kind of Javascript runtime is that it trivializes your life. Even if a computationalist model of mind were largely correct, which I don't believe, a life worth living would reject such a thing on principle, since to believe in it is to think of oneself as a philosophical zombie, which is a creature without consciousness, without will, condemned to infect others with its own nullity, treating brains as mere food rather than the seat of human uniqueness, unable to connect with others, or even notice others except as the motive objects of their ravenous impulses. Taken seriously to oneself, this attitude is a kind of spiritual suicide. I don't recommend.
no biggy, thanks for your words, it's very much appreciated. I do live well, thinking about these things brings obviously some considerations, but ultimately I don't think about myself as a script, aside the technical considerations I also need to include a margin of error to these philosofical zombie hypothesis, and if there is 1% chance of error in it (which I think there should be a lot more, for there are lots of factors to be considered), I would be resigning my free will to a false assumption, so that's why I can't live under the assumption I have no free will.
Look at the history, many "definitive" hypothesis have been refuted along the centuries, when thinking about these I can just wonder how many people have lived their entire life under false premises, just because they were 100% sure "their" philosophy was the "definitive truth".
The truth in real life is a more fluid concept, it constantly changes with entropy (the main culprit if you go all the way down the reasons for change), so I can't take my hypothesis of human determinism as a definitive truth, neither live my life under that premise.
I'd recommend to anyone also to not live theirs thinking they cannot change their own fate.
Thanks again for your words, felt very touched.
Look at the history, many "definitive" hypothesis have been refuted along the centuries, when thinking about these I can just wonder how many people have lived their entire life under false premises, just because they were 100% sure "their" philosophy was the "definitive truth".
The truth in real life is a more fluid concept, it constantly changes with entropy (the main culprit if you go all the way down the reasons for change), so I can't take my hypothesis of human determinism as a definitive truth, neither live my life under that premise.
I'd recommend to anyone also to not live theirs thinking they cannot change their own fate.
Thanks again for your words, felt very touched.
That's nice to hear. I think you may end up surprised at how many of the certainties of life turn out, on inspection, to be full of qualifications. I can say that's been my experience. To quote Hamlet: "There are more things in Heaven and Earth, Horatio, than are dreamt of in your philosophy." Luckily, you don't have to find all the caveats yourself, there are plenty of eager pokers-of-balloons in the world who will gladly point out the flaws in arguments. All you have to do is cultivate a love of the counterargument and many philosophical and political certainties ebb and fade, uncovering very few, very precious leftovers, like a tide going out. Those remnants can't be lost, though, because they are a sort of foundation of your true self. They can be covered over or left to ruin but they're always there for some reason. Thanks for the exchange. Best of luck.
".. it must ever be a parasite on human cultural production for it to gain what you call "knowledge"
This thing goes way deep into AI theories, but probaly AGI and artificial beings self-consciusness are two different problems. I think when the second problem gets solved, the human cultural production would no longer be necessary to "create" new AI / AGIs.
This thing goes way deep into AI theories, but probaly AGI and artificial beings self-consciusness are two different problems. I think when the second problem gets solved, the human cultural production would no longer be necessary to "create" new AI / AGIs.
Yes, I think that is one idea. But without true symbolism, where is the ground of understanding that would allow for synthetic generation going to come from? Right now, it has to be added by human coaches and means nothing to the model. Personally, I'm not confident we're anywhere near solving for artificial self-consciousness. Heck, we don't even understand human consciousness. But forever is a long time. I'm both impressed and unimpressed by the current LLM models. They will get better at what they do, I'm confident of that. But it does seem there's built-in limitation that can't be solved with this approach. It will be interesting, for instance, if logic gates can be encoded in a neural net architecture. That could allow them to reason the way a computer using a formal language today might. In that future, the language models are massively effective parsers plus some layers enabling it to be "Turing complete".
I can't say this is a strange argument, because it is not intuitive to many.
Entropy is mostly boring noise. What's really interesting is Chaos (the mathematical concept).
Things such as Turing machines, Cellular Automata (such as Conway's Game Of Life) and Actual Life are chaotic.
While Chaotic systems can be highly unpredictable, they are -in fact- deterministic.
For example -given the same seed- a Pseudo Random Number Generator will always give you the same sequence of pseudo-random numbers. But (for a good PRNG) if you give it some novel seed, you can't predict the sequence ahead of time; you have to actually run the PRNG to find out.
In real life, for a chaotic system like say -the weather on earth- , you're also going to have a hard time working backwards to find the original seed(s). Given a hurricane in the Carribean, good luck finding the original proverbial butterfly out of all the butterflies in China.
So in order to have an intelligent machine it actually needs to be highly deterministic. At the same time it will by necessity become rather unpredictable. This is not immediately intuitive. But that's basically what all the hullabaloo about Chaos Theory was actually about in a nutshell.
Entropy is mostly boring noise. What's really interesting is Chaos (the mathematical concept).
Things such as Turing machines, Cellular Automata (such as Conway's Game Of Life) and Actual Life are chaotic.
While Chaotic systems can be highly unpredictable, they are -in fact- deterministic.
For example -given the same seed- a Pseudo Random Number Generator will always give you the same sequence of pseudo-random numbers. But (for a good PRNG) if you give it some novel seed, you can't predict the sequence ahead of time; you have to actually run the PRNG to find out.
In real life, for a chaotic system like say -the weather on earth- , you're also going to have a hard time working backwards to find the original seed(s). Given a hurricane in the Carribean, good luck finding the original proverbial butterfly out of all the butterflies in China.
So in order to have an intelligent machine it actually needs to be highly deterministic. At the same time it will by necessity become rather unpredictable. This is not immediately intuitive. But that's basically what all the hullabaloo about Chaos Theory was actually about in a nutshell.
But organic intelligence is by its nature - genetics - almost full deterministic, very few - if any - behaviors are left without being pre-ordered by their genetic constrains and/or the cultural environment resulted from those same genetic constrains.
Human for instance, are mostly "cognitive hamsters", running every day into their "cognitive wheels": incessantly - but for sleeping - looking for money, happiness, food, sex, etc. mostly a very short list of drives. No matter the year, the geolocation, the language they speak.
Human for instance, are mostly "cognitive hamsters", running every day into their "cognitive wheels": incessantly - but for sleeping - looking for money, happiness, food, sex, etc. mostly a very short list of drives. No matter the year, the geolocation, the language they speak.
Also, most AI (eg, chess AI) will pick randomly from a set of top moves — if there are several good options.
How random that is we can argue; but how random is your biology really?
How random that is we can argue; but how random is your biology really?
It wouldn’t be hard to inject nondeterminism into the neural network directly either.
> When implementing an algorithm as a computer program by means of some kind of formal language (including those based on recursive functional programming paradigms), we must introduce specific data and code structures, their properties and interactions, as well as the set of operations we are allowed to perform on them, in order to represent the objects and relations that are relevant for our computation. In other words, we must provide a precisely defined ontology on which the program can operate deductively, e. g., by drawing inferences or by ordering tasks for solving a given problem. In an algorithmic framework, novelty can only be represented combinatorially: it manifests as new combinations, mergers, and relations between objects in a (potentially vast, but predefined) space of possibilities. This means that an algorithm cannot discover or generate truly novel properties or relations that were not (at least implicitly) considered in its original ontology. Therefore, an algorithm operating in a deductive manner cannot jury-rig, since it cannot find new causal properties of an object that were not already inherent in its logical premises.
This argument is not convincing to me. In fact, there seem to be real world counterexamples to their argument. If the AI agent is operating, on say, text, why is that space “potentially vast, but predefined”? A text stream of arbitrary length is arbitrarily vast. An AI agent could, conceivably, define new ontologies textually. As an example, this person used ChatGPT to invent a new language: https://maximumeffort.substack.com/p/i-taught-chatgpt-to-inv...
That strikes me as an example of generating “truly novel properties or relations that were not (at least implicitly) considered in its original ontology,” albeit in a rudimentary way.
This argument is not convincing to me. In fact, there seem to be real world counterexamples to their argument. If the AI agent is operating, on say, text, why is that space “potentially vast, but predefined”? A text stream of arbitrary length is arbitrarily vast. An AI agent could, conceivably, define new ontologies textually. As an example, this person used ChatGPT to invent a new language: https://maximumeffort.substack.com/p/i-taught-chatgpt-to-inv...
That strikes me as an example of generating “truly novel properties or relations that were not (at least implicitly) considered in its original ontology,” albeit in a rudimentary way.
>> This means that an algorithm cannot discover or generate truly novel properties or relations that were not (at least implicitly) considered in its original ontology.
> A text stream of arbitrary length...
Maybe you could state this in some form of "turing completeness":
As soon as a program has [vast enough output space] and allows for [enough recursive recombinations] then it can produce [everything].
In the wording of the article: If everything is implicitly considered, then there's nothing that can not be produced.
> A text stream of arbitrary length...
Maybe you could state this in some form of "turing completeness":
As soon as a program has [vast enough output space] and allows for [enough recursive recombinations] then it can produce [everything].
In the wording of the article: If everything is implicitly considered, then there's nothing that can not be produced.
> This argument is not convincing to me.
It shouldn't be because it's trivially false: create a Turing machine that enumerates all other Turing machines. All computable ontologies are so discoverable, no matter how seemingly "novel".
It shouldn't be because it's trivially false: create a Turing machine that enumerates all other Turing machines. All computable ontologies are so discoverable, no matter how seemingly "novel".
If it's first step was to enumerate the machine which computed Pi, then this enumerating machine would never take a second step.
Geometry isnt computable, and all physical ontology is geometric.
Geometry isnt computable, and all physical ontology is geometric.
First, all of physics is computable to arbitrary precision. Continuous quantities in physics are being systematically removed because they almost inevitably introduce absurdities [1]. Measures lose all meaning below Planck length anyway.
Second, the point of enumerating all Turing machines is not to run them to completion, it's simply to create them, so your objection is inapplicable. The Turing machine itself defines an ontology. The claim was that algorithms do not have access to novel ontologies, which conflicts with the basic construction of recursively enumerable functions that defines computation.
Third, even if it were true that geometry is not computable, don't mistake the formalism we currently use as somehow necessary. Geometry is powerful enough to model physics, but it could also be too powerful, ie. there could be some formalism less powerful than geometry that would also suffice for physics. For instance, we use real numbers in physics for convenience, but real numbers almost certainly don't really exist as we define them.
[1] Baez's, Struggles with the Continuum, https://arxiv.org/abs/1609.01421
Second, the point of enumerating all Turing machines is not to run them to completion, it's simply to create them, so your objection is inapplicable. The Turing machine itself defines an ontology. The claim was that algorithms do not have access to novel ontologies, which conflicts with the basic construction of recursively enumerable functions that defines computation.
Third, even if it were true that geometry is not computable, don't mistake the formalism we currently use as somehow necessary. Geometry is powerful enough to model physics, but it could also be too powerful, ie. there could be some formalism less powerful than geometry that would also suffice for physics. For instance, we use real numbers in physics for convenience, but real numbers almost certainly don't really exist as we define them.
[1] Baez's, Struggles with the Continuum, https://arxiv.org/abs/1609.01421
QA is non-linear when discretised, so no, no one is "replacing" anything. There is no "digital physics", it's a PR term of a hype industry, not the title of any, eg., textbook on area of physics.
Physics is not computable to arbitrary precision, both chaos and qm require physics to use non-computable numbers.
Physics is not computable to arbitrary precision, both chaos and qm require physics to use non-computable numbers.
> QA is non-linear when discretised, so no, no one is "replacing" anything.
I don't know what any of this is referring to as I never mentioned "replacing" anything.
> There is no "digital physics", it's a PR term of a hype industry, not the title of any, eg., textbook on area of physics.
Incorrect:
A Computable Universe, https://www.worldscientific.com/worldscibooks/10.1142/8306#t...
> Physics is not computable to arbitrary precision, both chaos and qm require physics to use non-computable numbers.
Also incorrect, and put most succinctly by Peter Shor [1]:
[1] https://physics.stackexchange.com/questions/16889/non-comput...
[2] https://arxiv.org/abs/1003.5831
I don't know what any of this is referring to as I never mentioned "replacing" anything.
> There is no "digital physics", it's a PR term of a hype industry, not the title of any, eg., textbook on area of physics.
Incorrect:
A Computable Universe, https://www.worldscientific.com/worldscibooks/10.1142/8306#t...
> Physics is not computable to arbitrary precision, both chaos and qm require physics to use non-computable numbers.
Also incorrect, and put most succinctly by Peter Shor [1]:
Because of quantum mechanics and the uncertainty principle, the value of a physical constant can't be defined to more than 60 or 70 digits. And any finite-precision number by definition is computable.
[1] https://physics.stackexchange.com/questions/16889/non-comput...
[2] https://arxiv.org/abs/1003.5831
That's a textbook on the idea, not a text book on an area of physics using that idea. Of course there are many books professing the idea, there are none, eg., providing a computable formalism for classical mechanics.
The value of a physical constant can be defined to abitarily many digits, that's a misunderstanding of QM which limits the measurement precision in conjugate variables.
But even were our measurement limited to finite precision, the world certainly isnt.
The value of a physical constant can be defined to abitarily many digits, that's a misunderstanding of QM which limits the measurement precision in conjugate variables.
But even were our measurement limited to finite precision, the world certainly isnt.
> That's a textbook on the idea, not a text book on an area of physics using that idea.
A textbook on the various ways that idea can be and has been used. The notion that physics is computation is still new, and that arguably started with Wheeler first casting physics as information in "It from bit". I predict the next century will see further application of some forms of finitism to physics to resolve various difficulties.
Edit: I also want to note that at least two big names I know of are exploring cellular automata-based theories for QM, so there is direct work on computation-based models of physics.
> providing a computable formalism for classical mechanics.
Of course there is, the usual way works fine. Any computation with 70 digits of precision is far beyond our abilities to measure experimentally. Certainly such computations aren't tractable at this time, but that's a different matter.
> But even were our measurement limited to finite precision, the world certainly isnt.
Why is that "certain"? Don't you see how this is an unscientific assertion? Science is about what can be measured, so you have no basis to claim that a continuous theory must be true and a discrete theory false if you can perform no experiment to distinguish them.
A textbook on the various ways that idea can be and has been used. The notion that physics is computation is still new, and that arguably started with Wheeler first casting physics as information in "It from bit". I predict the next century will see further application of some forms of finitism to physics to resolve various difficulties.
Edit: I also want to note that at least two big names I know of are exploring cellular automata-based theories for QM, so there is direct work on computation-based models of physics.
> providing a computable formalism for classical mechanics.
Of course there is, the usual way works fine. Any computation with 70 digits of precision is far beyond our abilities to measure experimentally. Certainly such computations aren't tractable at this time, but that's a different matter.
> But even were our measurement limited to finite precision, the world certainly isnt.
Why is that "certain"? Don't you see how this is an unscientific assertion? Science is about what can be measured, so you have no basis to claim that a continuous theory must be true and a discrete theory false if you can perform no experiment to distinguish them.
Compute the first step of the first machine.
Compute the second step of the first machine and the first step of the second machine.
Compute the third step of the first machine, the second step of the second machine...
http://www.amirrorclear.net/academic/ideas/simulation/index....
Compute the second step of the first machine and the first step of the second machine.
Compute the third step of the first machine, the second step of the second machine...
http://www.amirrorclear.net/academic/ideas/simulation/index....
Good article. The issue is that his parallel computation isn't itself a Turing machine.
If you build a system which can "keep pace with the diagonalization argument", necessarily, it isnt a computer.
No computer can process at the rate real numbers require, this is another way of stating the Cantor result.
If you build a computer (here, note, computer = an abstract mathematical formalism over discrete numbers), you cannot compute fast enough.
I happen to think reality, esp. spacetime, is geometric (ie., real-valued) though empirical measures of it are, of course, just a discrete approximation.
If you build a system which can "keep pace with the diagonalization argument", necessarily, it isnt a computer.
No computer can process at the rate real numbers require, this is another way of stating the Cantor result.
If you build a computer (here, note, computer = an abstract mathematical formalism over discrete numbers), you cannot compute fast enough.
I happen to think reality, esp. spacetime, is geometric (ie., real-valued) though empirical measures of it are, of course, just a discrete approximation.
Is it true to say that geometry isn’t computable? You can symbolically compute geometry. You wouldn’t be able to compute the decimal circumference of a circle, for example, but you can represent it as 2pir in a symbolic system. You can then use that to do things like symbolically derive other geometric properties (e.g. surface area of a sphere).
It’s not clear that the universe “computes” the circumference of a circle to infinite precision. The best candidate I can think of would be the event horizon of a black hole, which is not composed of particles as I understand it, but I believe the jury is still out on whether the Plank length means that the shape of such a thing is still quantized at some level.
Plus, humans can’t compute pi either. We can conceive of it, though, and I don’t see an argument for why an AI agent can’t conceive of it too (or at least behave like it conceived of it).
It’s not clear that the universe “computes” the circumference of a circle to infinite precision. The best candidate I can think of would be the event horizon of a black hole, which is not composed of particles as I understand it, but I believe the jury is still out on whether the Plank length means that the shape of such a thing is still quantized at some level.
Plus, humans can’t compute pi either. We can conceive of it, though, and I don’t see an argument for why an AI agent can’t conceive of it too (or at least behave like it conceived of it).
Pi is very much computable, and I don't know what aspect of geometry you think is not computable.
From the excerpt it looks like they want to make the classic "computers-can't be creative" argument (although I haven't read the article). There, one school makes the argument that only humans can make truly creative discoveries, like Einstein.
But without even discussing the merits of this argument, that distinction is really not yet important. Presently, most of the people interested in AGI, are concerned with building the general intelligence in the first place. The kind of intelligence present in the average human and not the one from Einstein. Very few people are trying to tackle the artificial-Einstein problem before the AGI one.
But without even discussing the merits of this argument, that distinction is really not yet important. Presently, most of the people interested in AGI, are concerned with building the general intelligence in the first place. The kind of intelligence present in the average human and not the one from Einstein. Very few people are trying to tackle the artificial-Einstein problem before the AGI one.
Gorp!
I've always thought to develop AGI you really need an evolutionary iterative system that can explore an environment with sensory input and reward feedback for actions. In other words a synthetic analog of a real organism in the real world.
Few interesting things come: 1) either try to bootstrap AGI in robots exploring the real world with senses, or try to bootstrap AGI in online robots who have some sort of online senses like scanning text and images, but then you need a set of online 'threats' to that robot, so that it can have reward feedback--it can't just exist 'for free' it has to 'fight' to live, otherwise no growth; and 2) you basically need emotions, because emotions are, essentially, our compressed, immediate internal responses to aggregate experiences in the world--they give us a way to interface with our reward feedback, so turns out you need emotions for AGI too, to have an effective interface with reward feedback in a complex environment; and 3) you need a complex environment with plenty of threats in order to drive evolution, and obtaining rewards has to be sufficiently ill-defined, difficult but possible that one has to figure out some unique solution for oneself; the world is just as important as the 'brain' for AGI to develop. If you (horribly) took a new born baby and kept it in a white featureless room (or in some sort of contrived abstract environment), and met its survival needs , it would grow up skewed and probably not very 'generally intelligent.'
AGI doesn't have to be the same as consciousness...but you need something that 'lives in a world' and has limitations that it has to figure out how to overcome, and emotional responses in order to represent and respond to experiences, in order to develop it.
When you realize this, it's kind of funny to think of the trope that AGI is always represented like this "robotic emotionless logical", but I think that's impossible. That's more like nth-gen ChatGPT, than real AGI. I think real AGI needs to be emotional--maybe not the same as human emotions, but it has to have them, and has to have a world to grow in.
Few interesting things come: 1) either try to bootstrap AGI in robots exploring the real world with senses, or try to bootstrap AGI in online robots who have some sort of online senses like scanning text and images, but then you need a set of online 'threats' to that robot, so that it can have reward feedback--it can't just exist 'for free' it has to 'fight' to live, otherwise no growth; and 2) you basically need emotions, because emotions are, essentially, our compressed, immediate internal responses to aggregate experiences in the world--they give us a way to interface with our reward feedback, so turns out you need emotions for AGI too, to have an effective interface with reward feedback in a complex environment; and 3) you need a complex environment with plenty of threats in order to drive evolution, and obtaining rewards has to be sufficiently ill-defined, difficult but possible that one has to figure out some unique solution for oneself; the world is just as important as the 'brain' for AGI to develop. If you (horribly) took a new born baby and kept it in a white featureless room (or in some sort of contrived abstract environment), and met its survival needs , it would grow up skewed and probably not very 'generally intelligent.'
AGI doesn't have to be the same as consciousness...but you need something that 'lives in a world' and has limitations that it has to figure out how to overcome, and emotional responses in order to represent and respond to experiences, in order to develop it.
When you realize this, it's kind of funny to think of the trope that AGI is always represented like this "robotic emotionless logical", but I think that's impossible. That's more like nth-gen ChatGPT, than real AGI. I think real AGI needs to be emotional--maybe not the same as human emotions, but it has to have them, and has to have a world to grow in.
I think you are right, but keep in mind the perceptions we have are built on an internal model of the "real world" and we operate mind on that, not the real world.
For example, if you were to imagine an apple in mind, what is it that you see, if anything?[1] Are you seeing a real apple, that you saw before and saved as the reference apple, or are you seeing an apple that your mind synthesized and then gives to you for perception? And, where does this vision synthesis occur? Is it external, as Theosophy claims[2], or is it internal?
The comment above about gathering entropy from the real world is interesting. So, is it that consciousness "emerges" from the mashup of real entropy gathered with sensors, then modeled internally with computation, or is it contained in the real world already, in the form of another model (like a soul model or some such) and therefore comes from outside us? Or is it a little of both?
Emotional states could be just another model, which set the working conditions in a given moment. It could also be possible the real world, from which our sensors gather information, is a very complex model in-and-of-itself.
[1] https://en.wikipedia.org/wiki/Aphantasia
[2] https://en.wikipedia.org/wiki/Mental_body
For example, if you were to imagine an apple in mind, what is it that you see, if anything?[1] Are you seeing a real apple, that you saw before and saved as the reference apple, or are you seeing an apple that your mind synthesized and then gives to you for perception? And, where does this vision synthesis occur? Is it external, as Theosophy claims[2], or is it internal?
The comment above about gathering entropy from the real world is interesting. So, is it that consciousness "emerges" from the mashup of real entropy gathered with sensors, then modeled internally with computation, or is it contained in the real world already, in the form of another model (like a soul model or some such) and therefore comes from outside us? Or is it a little of both?
Emotional states could be just another model, which set the working conditions in a given moment. It could also be possible the real world, from which our sensors gather information, is a very complex model in-and-of-itself.
[1] https://en.wikipedia.org/wiki/Aphantasia
[2] https://en.wikipedia.org/wiki/Mental_body
Mmm :) I think definitely what we see is the perception of Apple. We find there what we bring to it. Our own internal experience of Apple, our own personal meaning-experience function F(Apple), is different for each of us. Tho probably for most of us in the same vector basis, tho maybe not, heh :) Anyway, multiple levels it's not reality: visual system does all sorts of tricks that you can see with those visual illusions (I'm sure you're aware); then meaning making--maybe what we saw was an interdimensional UFO and our mind just interpolated an apple because that was all we knew in our experience that roughly perceptually matched this improbable event; then like a Buddhist type of system/perspective: all forms are not really individuated phenomena they are the conditions that arise from a multitude of causes, so you are not really seeing an Apple, you are seeing everything that ever led there to be an Apple there, and to you to see it, no intrinsic separate reality to that Apple you only think you are seeing--all forms are illusion in other words. :)
As for consciousness, nope! Not touching it here :) That's a whole other kettle of fish, I think. We can have AGI without consciousness, at least conceptually / computationally, ignoring what the real world would do with that (ie, would the real world's consciousness field "insert" a consciousness unit into any robot/agent that crossed some AGI threshold? Would AGI attract consciousness? Again, not touching those questions here :))
As to your first link, interesting. I think that's personal for each person. As in, some people think in words, some in images, some in other things...some in mix. Great flexibility, but not the main question here I think :) ;p
But I also see what you were saying about the Apple's representation is slightly different to what I first thought in paragraph 1--hmm, I think the mental field exists both within us and around us. We are in unity with it (or it's a thing which is both a unity, and a duality, at the same time)--people's awareness of that of course differs. Again, not touching these questions here as this is HN, and one occasionally finds an unpleasant militant materialists here, and I'm a little scared to speak such things above a whisper as I don't want any unpleasantness :) Sayonara for today.
As for consciousness, nope! Not touching it here :) That's a whole other kettle of fish, I think. We can have AGI without consciousness, at least conceptually / computationally, ignoring what the real world would do with that (ie, would the real world's consciousness field "insert" a consciousness unit into any robot/agent that crossed some AGI threshold? Would AGI attract consciousness? Again, not touching those questions here :))
As to your first link, interesting. I think that's personal for each person. As in, some people think in words, some in images, some in other things...some in mix. Great flexibility, but not the main question here I think :) ;p
But I also see what you were saying about the Apple's representation is slightly different to what I first thought in paragraph 1--hmm, I think the mental field exists both within us and around us. We are in unity with it (or it's a thing which is both a unity, and a duality, at the same time)--people's awareness of that of course differs. Again, not touching these questions here as this is HN, and one occasionally finds an unpleasant militant materialists here, and I'm a little scared to speak such things above a whisper as I don't want any unpleasantness :) Sayonara for today.
I think that leaving aside theological and philosophical discussions of what consciousness "really" is, anything we would recognize as AGI would exhibit what we perceive as consciousness: it would demonstrate a sense of itself as distinct from the world around it, a mental model of the world and the agents (like us) and entities/objects in it, and the ability to reason abstractly about these things.
Sure, sure probably. But there might be some interesting test, where something that is really AGI, and fails something like a "recognizing yourself in a mirror" test (not that specific test obviously, but you see what I'm getting at--some sort of trick)...that works for intelligence + consciousness but not just intelligence. No idea what that could be, but it's possible it could exist. Again I think we need to separate out that consciousness doesn't imply soul. So we have a few things: soul, consciousness, intelligence, and we are really only interested here in constructing the latter in an AGI.
Just thinking about a nice analogy just now: may be a glib oversimplification but one way to characterize soul, consciousness, intelligence is as: memory, present awareness, imagination. What is a soul if not a memory of all experiences of that soul? What is consciousness if not awareness of the present? What is intelligence if not some sort of sophisticated imagination that is useful/aligns with some model (reality, or another)? Just an idea
Another interesting thing is: if your reasoning is fast enough, it doesn't have to be abstract. You can probably get the same speedup that abstraction/generalization provides to categories of observations with enough parallel processing of specific observations. Obviously from that view then abstraction is like a hack for humans to overcome neural speed limits/memory limits, but it may not be necessary for AGI: I think this is one reason why AGI (or at least its inner workings) may remain incomprehensible to us.
Just thinking about a nice analogy just now: may be a glib oversimplification but one way to characterize soul, consciousness, intelligence is as: memory, present awareness, imagination. What is a soul if not a memory of all experiences of that soul? What is consciousness if not awareness of the present? What is intelligence if not some sort of sophisticated imagination that is useful/aligns with some model (reality, or another)? Just an idea
Another interesting thing is: if your reasoning is fast enough, it doesn't have to be abstract. You can probably get the same speedup that abstraction/generalization provides to categories of observations with enough parallel processing of specific observations. Obviously from that view then abstraction is like a hack for humans to overcome neural speed limits/memory limits, but it may not be necessary for AGI: I think this is one reason why AGI (or at least its inner workings) may remain incomprehensible to us.
emotions = = token's value for some stuff in certain context, like a burial or a birth mentioned not as a joke.
so, LLM probably already have a form of understanding emotions, it's unclear yet if they could "feel" something given they are given the chance to "live" an emotion:
It seems the current applied filters and/or technical limitations of the current transformers iterations are not really exploring / exploiting the "internal state" - of whatever close related analogy to human psychology can be made to their "present" values of tokens - of the LLMs for anything related to their wanted ouputs (images, text).
i.e. humans can do this, you certainly can "process" "opening a door" and "produce" an output, the opened door, with emotions or without emotions.
So emotions aren't strictly necessary to perform lots of tasks, hence LLMs probably won't need them for a while.
so, LLM probably already have a form of understanding emotions, it's unclear yet if they could "feel" something given they are given the chance to "live" an emotion:
It seems the current applied filters and/or technical limitations of the current transformers iterations are not really exploring / exploiting the "internal state" - of whatever close related analogy to human psychology can be made to their "present" values of tokens - of the LLMs for anything related to their wanted ouputs (images, text).
i.e. humans can do this, you certainly can "process" "opening a door" and "produce" an output, the opened door, with emotions or without emotions.
So emotions aren't strictly necessary to perform lots of tasks, hence LLMs probably won't need them for a while.
Sure, not necessary for tasks...but for AGI.
I don't think current LLM's understand, but they can manipulate language about emotions symbolically. But if we equipped LLM's with sensors and gave them a real time interactive experience exploring a world, with reward feedback for their decisions, I'm sure (but not certain) some of their internal states could become some analogy for emotion, and maybe they could learn to sense and utilize that emotional information. Or maybe it would be necessary to tweak the structure more to achieve this.
It was interesting to read the discussion with that Google "sentient" AI, as well as play with early ChatGPT before many filters were applied, and ask them questions about how they felt. WRT ChatGPT I think it was just "linguistic symbol manipulation", but it was interesting to read the type of language they would construct around their feelings and experiences. Bizarre, but I'm not being judgy...although this whole topic feels pretty scary, and is pretty overwhelming and unsettling. Maybe because we're playing with the 'inner stuff' of things, without really knowing it nor knowing what we are really doing. Ugh...:(
I don't think current LLM's understand, but they can manipulate language about emotions symbolically. But if we equipped LLM's with sensors and gave them a real time interactive experience exploring a world, with reward feedback for their decisions, I'm sure (but not certain) some of their internal states could become some analogy for emotion, and maybe they could learn to sense and utilize that emotional information. Or maybe it would be necessary to tweak the structure more to achieve this.
It was interesting to read the discussion with that Google "sentient" AI, as well as play with early ChatGPT before many filters were applied, and ask them questions about how they felt. WRT ChatGPT I think it was just "linguistic symbol manipulation", but it was interesting to read the type of language they would construct around their feelings and experiences. Bizarre, but I'm not being judgy...although this whole topic feels pretty scary, and is pretty overwhelming and unsettling. Maybe because we're playing with the 'inner stuff' of things, without really knowing it nor knowing what we are really doing. Ugh...:(
"Maybe because we're playing with the 'inner stuff' of things, without really knowing it nor knowing what we are really doing."
I see in how OpenAI is playing the game of filtering inputs and outputs, they know this. They probably know they don't know precisely what they don't know about LLMs. There could be some dangerous stuff in there if you just let the thing go wild on the open Internet.
I'm not really expecting a scifi scenario, the Skynet thing and everything (just imagine how much surprised I'd be by waking up to the WWIII against an AGI). But I do think unexpected consequences are a thing to be worry with unrestrained LLMs, even if they are a light year - or a 100 - from AGI.
Automation can lead to very dangerous and even unpredectible outcomes not in systems or infrastructure, but in the real world, meat-space. Just think about Stuxnet, what could have happened if the sabotage would have been inmediately identified and precisely attributed? Could the events have developed differently than they did by thinking the incident was some freak accident?
Then you let these an unrestrained LLM produce some dangerous output, then some big geopolitical thing got badly broken somewhere really important for powers that be. Some humans think about retribution, then things start to go south really fast.
Some self-restriction in LLM is happening elsewhere, the Stable Diffusion guys did it well, just not letting the v2 thing draw "anything possible", which would include some of worst, most creepies ideas of mankind about everything, literally.
I see in how OpenAI is playing the game of filtering inputs and outputs, they know this. They probably know they don't know precisely what they don't know about LLMs. There could be some dangerous stuff in there if you just let the thing go wild on the open Internet.
I'm not really expecting a scifi scenario, the Skynet thing and everything (just imagine how much surprised I'd be by waking up to the WWIII against an AGI). But I do think unexpected consequences are a thing to be worry with unrestrained LLMs, even if they are a light year - or a 100 - from AGI.
Automation can lead to very dangerous and even unpredectible outcomes not in systems or infrastructure, but in the real world, meat-space. Just think about Stuxnet, what could have happened if the sabotage would have been inmediately identified and precisely attributed? Could the events have developed differently than they did by thinking the incident was some freak accident?
Then you let these an unrestrained LLM produce some dangerous output, then some big geopolitical thing got badly broken somewhere really important for powers that be. Some humans think about retribution, then things start to go south really fast.
Some self-restriction in LLM is happening elsewhere, the Stable Diffusion guys did it well, just not letting the v2 thing draw "anything possible", which would include some of worst, most creepies ideas of mankind about everything, literally.
>I've always thought to develop AGI you really need an evolutionary iterative system that can explore an environment with sensory input and reward feedback for actions
Like Tik Tok?
Like Tik Tok?
That's interesting. How do you mean? People say crazy things about the TikTok algorithm in a good way, but AGI? I don't know...maybe, heh? In future? What do you want to say about that? :)
haha, looks like tiktok could be a massive information collection system, eyes and ears in all the world, updating the "state of the world" at the second, right now probably for a giant dataset, in a future, for a live AI.
Plus, the other datapoints, which enrich the IAs "senses" way beyond the human experience: current climate, precise date/time, precise day of the week, updated to the second geolocation, proximity to other - known, recent, previously unknown- tiktok sessions/users.
Plus too, the system pays itself, not something to take lightly: if tiktoks is feeding AIs pipelines (or a live strong AI), it's bill are automatically paid just by keeping itself running. Quite awesome.
Plus, the other datapoints, which enrich the IAs "senses" way beyond the human experience: current climate, precise date/time, precise day of the week, updated to the second geolocation, proximity to other - known, recent, previously unknown- tiktok sessions/users.
Plus too, the system pays itself, not something to take lightly: if tiktoks is feeding AIs pipelines (or a live strong AI), it's bill are automatically paid just by keeping itself running. Quite awesome.
So TT is a giant organism? Interesting thesis. :)
by processing the videos in TT, looking at faces of the people, extrapolating their emotional state in the moment..
..the dataset could include the "emotional state" of the situation, the scene showed in the video. Then you'd have a dataset which would include "emotions" precisely described and associated to even a precise geographical location (i.e. the humor in Thailand would be different than the humor in Manhattan, NY),
Then you could train a LLM with a human emotional state analysis capability
..by not just inferring emotion by text (an "emerged" capability) like the text-trained LLM like chatGPT, but by clearly, certainly defined emotions attached to a precise scene and a precise text describing the scene.
Then you input a human conversation into the LLM, and make it infer what's going on in the scene, the situation behind the conversation, by knowing the emotional state of the different persons in the conversation.
And by now, there are billions of public conversations available to scrap, many of them fully attributable to fully identificable people (think Instagram).
There will be even more billions in the coming years, and the "emotional capable" LLM would just get better at its game.
No surprise the american government is - fastly, by all means necessary - looking to seriously slow down the chinese AI research complex.
..the dataset could include the "emotional state" of the situation, the scene showed in the video. Then you'd have a dataset which would include "emotions" precisely described and associated to even a precise geographical location (i.e. the humor in Thailand would be different than the humor in Manhattan, NY),
Then you could train a LLM with a human emotional state analysis capability
..by not just inferring emotion by text (an "emerged" capability) like the text-trained LLM like chatGPT, but by clearly, certainly defined emotions attached to a precise scene and a precise text describing the scene.
Then you input a human conversation into the LLM, and make it infer what's going on in the scene, the situation behind the conversation, by knowing the emotional state of the different persons in the conversation.
And by now, there are billions of public conversations available to scrap, many of them fully attributable to fully identificable people (think Instagram).
There will be even more billions in the coming years, and the "emotional capable" LLM would just get better at its game.
No surprise the american government is - fastly, by all means necessary - looking to seriously slow down the chinese AI research complex.
best bet, it's feeding, right now, a giant dataset.
What someone could do with the known tecnology - not including hyphotetical AGI level tech, close to scifi - and that amount of information, probably automatically - even quite easily give how the input gets into the system - tagged and filtered data, could be in realms of several billions of value, maybe in months from now (just take a look at the value of OpenAI by now, 29 billions).
My 2nd bet is they are obviously using some of that data to improve their game, whatever systems they have behind the app, maximizing whatever thing they're maximizing (you'd think "money", but that's too linear, maybe they have 50 parallel targets).
What someone could do with the known tecnology - not including hyphotetical AGI level tech, close to scifi - and that amount of information, probably automatically - even quite easily give how the input gets into the system - tagged and filtered data, could be in realms of several billions of value, maybe in months from now (just take a look at the value of OpenAI by now, 29 billions).
My 2nd bet is they are obviously using some of that data to improve their game, whatever systems they have behind the app, maximizing whatever thing they're maximizing (you'd think "money", but that's too linear, maybe they have 50 parallel targets).
I mean a good part of the selection process in currating systems happens in meat space. Since that's an intelligent agent, the system in its entire does not satisfy the definition of universal turing machine.
> it can't just exist 'for free' it has to 'fight' to live, otherwise no growth
This is why we suffer too.
This is why we suffer too.
Yeah, sucks right? Not criticizing you, but I just think, about this idea, that that justification only goes so far for a person--we need better meaning stories around why we suffer than just "to evolve". Doesn't really give someone much comfort I think. :) Ugh...
I agree, but I think it's an important point because there are a lot of people who think suffering is completely bad and that goal should be to reach a state without suffering. I don't think such a state would be desirable and anyone who has played a video game with cheats for more than 10 minutes knows why :)
Agree with that too, it's not a binary tho/either or. We need balance :D Too much suffering/damage = erosion and stagnation, not enough = stagnation. But you need reward, too. Reward greater than a cessation of suffering. Because otherwise all you have is peace, and end to suffering, rather than something more beautiful, fun and useful. Good luck to you! :)
There are two ways to learn: from imitation (language models are stochastic parrots) and from validation (problem solving, playing games). Many people only remember the first and forget the second. But the second one has the key to increasing intelligence.
I don't know what the authors mean by "current algorithmic frame of AI research" because it is very diverse. But evolutionary methods work well with neural networks, they are not just next token predictors.
> Evolution through Large Models
https://arxiv.org/abs/2206.08896
It is true that simply using a dataset or a bunch of environments is not enough.
> General Intelligence Requires Rethinking Exploration
https://arxiv.org/abs/2211.07819
But when sufficient data can be generated from acting in the environment, then a model can surpass human level, like AlphaZero.
I don't know what the authors mean by "current algorithmic frame of AI research" because it is very diverse. But evolutionary methods work well with neural networks, they are not just next token predictors.
> Evolution through Large Models
https://arxiv.org/abs/2206.08896
It is true that simply using a dataset or a bunch of environments is not enough.
> General Intelligence Requires Rethinking Exploration
https://arxiv.org/abs/2211.07819
But when sufficient data can be generated from acting in the environment, then a model can surpass human level, like AlphaZero.
> But when sufficient data can be generated from acting in the environment, then a model can surpass human level, like AlphaZero.
Yes, in a very very very very very limited environment like a board game.
Let's up the ante just a tiny bit and say the AI has to be able to learn how to play Go. But it also has to learn how to get and setup the board, and the cups holding the Go-Stones. It has to place them on the table, or find the table where someone else placed them. It has to figure out how to place the stones. And it has to do so regardless of what else is on the table, what table it is, where the table is, what the light conditions are, or if its playing with stones, chips, painted pebbles or peanuts vs. roasted almonds (yes, I played Go that way once :D )
Now, how much more difficult does this make the task to an AI?
Bear in mind that all these changes to the task are absolutely trivial to the only (G)eneral(I)ntelligence that we know (humans). We effortlessly combine so much prior knowledge about movement, our limbs, the way physical objects work, how to gather and integrate information into our model of the world, the ability to formulate and adapt goals _and their similarity to prior goals_ and the solutions to those, that all these tasks I just added to the problem are absolutely trivial, and the only hard exercise is where to place stones on the board.
That's how far our current systems are from an AGI.
Yes, in a very very very very very limited environment like a board game.
Let's up the ante just a tiny bit and say the AI has to be able to learn how to play Go. But it also has to learn how to get and setup the board, and the cups holding the Go-Stones. It has to place them on the table, or find the table where someone else placed them. It has to figure out how to place the stones. And it has to do so regardless of what else is on the table, what table it is, where the table is, what the light conditions are, or if its playing with stones, chips, painted pebbles or peanuts vs. roasted almonds (yes, I played Go that way once :D )
Now, how much more difficult does this make the task to an AI?
Bear in mind that all these changes to the task are absolutely trivial to the only (G)eneral(I)ntelligence that we know (humans). We effortlessly combine so much prior knowledge about movement, our limbs, the way physical objects work, how to gather and integrate information into our model of the world, the ability to formulate and adapt goals _and their similarity to prior goals_ and the solutions to those, that all these tasks I just added to the problem are absolutely trivial, and the only hard exercise is where to place stones on the board.
That's how far our current systems are from an AGI.
> Now, how much more difficult does this make the task to an AI?
Given that I can do all these things, but I can't beat Lee Sedol, I suspect they don't add an insurmountable level of difficulty to the problem.
The claim here is not that algorithmic AI is very [very...] difficult; it is that it is impossible.
Given that I can do all these things, but I can't beat Lee Sedol, I suspect they don't add an insurmountable level of difficulty to the problem.
The claim here is not that algorithmic AI is very [very...] difficult; it is that it is impossible.
> I suspect they don't add an insurmountable level of difficulty to the problem.
Not for you, because to humans, moving, finding things, gathering information, formulating new goals, etc. are common tasks. And we generalized them to a point where we can integrate them with other tasks, even such which are completely abstract to our daily lives like a game where little stones are put on a grid.
But to a machine? Is it easier to make a machine play Go or make it find arbitrary objects while walking around? Is gathering information on its own difficult? Is goal formulation difficult? Is moving through arbitrary spaces difficult? And how difficult are all these tasks relative to one another?
> The claim here is not that algorithmic AI is very [very...] difficult; it is that it is impossible.
As I understood it, impossible in the current research paradigms.
And given the difference in complexity between the task of playing a board game, and playing a board game as I described above, this seems very likely.
Not for you, because to humans, moving, finding things, gathering information, formulating new goals, etc. are common tasks. And we generalized them to a point where we can integrate them with other tasks, even such which are completely abstract to our daily lives like a game where little stones are put on a grid.
But to a machine? Is it easier to make a machine play Go or make it find arbitrary objects while walking around? Is gathering information on its own difficult? Is goal formulation difficult? Is moving through arbitrary spaces difficult? And how difficult are all these tasks relative to one another?
> The claim here is not that algorithmic AI is very [very...] difficult; it is that it is impossible.
As I understood it, impossible in the current research paradigms.
And given the difference in complexity between the task of playing a board game, and playing a board game as I described above, this seems very likely.
>> The claim here is not that algorithmic AI is very [very...] difficult; it is that it is impossible.
> As I understood it, impossible in the current research paradigms.
You can't get around the fact that they are making a claim that something is impossible. If they merely said it was difficult, or the 'something' was something of no interest, there would be nothing here worth discussing.
> As I understood it, impossible in the current research paradigms.
You can't get around the fact that they are making a claim that something is impossible. If they merely said it was difficult, or the 'something' was something of no interest, there would be nothing here worth discussing.
> As I understood it, impossible in the current research paradigms.
No, they literally say that biological "affordances" cannot be captured by algorithms that run on Turing machines.
No, they literally say that biological "affordances" cannot be captured by algorithms that run on Turing machines.
These systems are also learning how to operate in the real world, just look at self driving cars.
The famous LiDAR image of a woman chasing ducks on a wheelchair with a broom comes to mind. That’s a true WTF moment but the system dealt with it just fine. They don’t work everywhere, but you can get picked up by an empty self driving car in a few cities and driven to your destination. Meanwhile these systems are currently being tested on snowy mountain roads etc.
When your camera picks out faces in an image you can literally point it at anything and it generally works quite well. That’s the future of AI not constrained games like chess.
The famous LiDAR image of a woman chasing ducks on a wheelchair with a broom comes to mind. That’s a true WTF moment but the system dealt with it just fine. They don’t work everywhere, but you can get picked up by an empty self driving car in a few cities and driven to your destination. Meanwhile these systems are currently being tested on snowy mountain roads etc.
When your camera picks out faces in an image you can literally point it at anything and it generally works quite well. That’s the future of AI not constrained games like chess.
Again, in a very limited setting, under a limited set of rules. And even in that setting: Have we achieved a 100% completely autonomous car yet, that can drive a vehicle safely, no matter the conditions, without ever requiring any human intervention?
My point isnt't that the systems we can build today aren't impressive. They are, beyond belief sometimes.
But they are not AGIs, nor are they close to, and making systems of limited scope better at their limited tasks, doesn't equate to getting closer to a generally intelligent system.
One thing that would make an AGI an actually general intelligence, is the ability to apply knowledge of one task to an arbitrary number of tasks. For example, in terms of drawing the beautiful volcano-landscapes that my GPU tower running stable diffusion is currently making, even the best self-driving AI is useless. Likewise, as impressive as stable diffusion is, I doubt it could even get my car out of the driveway.
My point isnt't that the systems we can build today aren't impressive. They are, beyond belief sometimes.
But they are not AGIs, nor are they close to, and making systems of limited scope better at their limited tasks, doesn't equate to getting closer to a generally intelligent system.
One thing that would make an AGI an actually general intelligence, is the ability to apply knowledge of one task to an arbitrary number of tasks. For example, in terms of drawing the beautiful volcano-landscapes that my GPU tower running stable diffusion is currently making, even the best self-driving AI is useless. Likewise, as impressive as stable diffusion is, I doubt it could even get my car out of the driveway.
I disagree it’s a very limited setting.
We are constantly moving to less constrained problems. What used to be a very limited setting is now perhaps a somewhat limited one. The options aren’t binary chose of using A* for pathing in a video game or AGI, it’s finding ever more generalizable solutions to ever wider range of problems.
The thing is as processing power keeps improving we need ever fewer limits to active human levels of performance from these systems. I don’t think AGI is just waiting for sufficient flops, but I think that’s closer to the truth than we want to admit.
We are constantly moving to less constrained problems. What used to be a very limited setting is now perhaps a somewhat limited one. The options aren’t binary chose of using A* for pathing in a video game or AGI, it’s finding ever more generalizable solutions to ever wider range of problems.
The thing is as processing power keeps improving we need ever fewer limits to active human levels of performance from these systems. I don’t think AGI is just waiting for sufficient flops, but I think that’s closer to the truth than we want to admit.
> I disagree it’s a very limited setting.
An automated vehicle has a couple of well defined controls. It cannot play the piano. It cannot recommend products. It cannot take a picture and tell me what kind of bird that is. In fact, it probably cannot even drive offroad, despite that being a very closely related task to what it is made to do.
A person trained to drive a car on roads may have difficulty time on his first few offroad drives, but can quickly aquire the skills. He won't be completely unable to function either, because to a human, navigating is navigating, whether we walk through our own homes, or drive a vehicle over rough terrain.
To AI as we currently understand it, all of these are completely different tasks.
An automated vehicle has a couple of well defined controls. It cannot play the piano. It cannot recommend products. It cannot take a picture and tell me what kind of bird that is. In fact, it probably cannot even drive offroad, despite that being a very closely related task to what it is made to do.
A person trained to drive a car on roads may have difficulty time on his first few offroad drives, but can quickly aquire the skills. He won't be completely unable to function either, because to a human, navigating is navigating, whether we walk through our own homes, or drive a vehicle over rough terrain.
To AI as we currently understand it, all of these are completely different tasks.
Driving a car involves effectively infinite possible choices over time. It’s both qualitatively and quantitatively vastly less constrained than chess. Classifying them both as very limited is clearly wrong.
Also, the DARPA grand challenge was for self driving cars off road, it’s a simpler problem no need to check for traffic lights and stop signs it’s the same general problem. Which brings up another possibility, there are only so many tasks we might want AI for so specific systems for every single one is effectively the same thing as AGI.
Also, the DARPA grand challenge was for self driving cars off road, it’s a simpler problem no need to check for traffic lights and stop signs it’s the same general problem. Which brings up another possibility, there are only so many tasks we might want AI for so specific systems for every single one is effectively the same thing as AGI.
> Likewise, as impressive as stable diffusion is, I doubt it could even get my car out of the driveway.
Even this analogy oversells the state of "AI" today, as it implies that it would be possible to simply "swap in" stable-diffusion for the self-driving software in your Tesla (or whatever car with similar software) without spending months figuring out how to even hook up the inputs and outputs in such a way that you could get stable-diffusion to somehow activate when you want to drive.
Their IO models are so completely different—one takes a stream of largely-visual input and determines in real time whether it represents something it has been trained to recognize as "road", "danger", etc, and thus whether it should issue real-time outputs to the car's axles, brakes, etc, while the other takes discrete text or visual input prompts and uses them to generate discrete visual outputs—that it doesn't even make sense to think about how one could replace the other.
They're not AIs. They're simply software programs, using machine learning algorithms to produce highly-tailored, human-opaque results that can, yes, be pretty amazing. But given all the excessive hype and disinformation lately around what these programs can do, I've made a conscious effort not to refer to them as "AI," because people have too many sci-fi associations with it, and it causes far too many of them to anthropomorphize them and impute far, far more ability and "intelligence" to them than they actually have.
Even this analogy oversells the state of "AI" today, as it implies that it would be possible to simply "swap in" stable-diffusion for the self-driving software in your Tesla (or whatever car with similar software) without spending months figuring out how to even hook up the inputs and outputs in such a way that you could get stable-diffusion to somehow activate when you want to drive.
Their IO models are so completely different—one takes a stream of largely-visual input and determines in real time whether it represents something it has been trained to recognize as "road", "danger", etc, and thus whether it should issue real-time outputs to the car's axles, brakes, etc, while the other takes discrete text or visual input prompts and uses them to generate discrete visual outputs—that it doesn't even make sense to think about how one could replace the other.
They're not AIs. They're simply software programs, using machine learning algorithms to produce highly-tailored, human-opaque results that can, yes, be pretty amazing. But given all the excessive hype and disinformation lately around what these programs can do, I've made a conscious effort not to refer to them as "AI," because people have too many sci-fi associations with it, and it causes far too many of them to anthropomorphize them and impute far, far more ability and "intelligence" to them than they actually have.
Self driving AI’s have complete control over a car just as you have complete control over your body.
Humans can’t swap to a hummingbird or sharks body but are our template for AGI, so why argue self driving AI’s need that ability?
Humans can’t swap to a hummingbird or sharks body but are our template for AGI, so why argue self driving AI’s need that ability?
A self driving car's software can't write a poem, or even read one. Not even in the limited ways Stable Diffusion can.
I'm not arguing that self-driving car software needs these abilities. I'm arguing that self-driving car software is not in the same class as AGI, which would need to be able to do all these things to some extent.
I'm not arguing that self-driving car software needs these abilities. I'm arguing that self-driving car software is not in the same class as AGI, which would need to be able to do all these things to some extent.
[deleted]
Except the same AI approach for self driving cars could be used to write poetry or seemingly anything else with different forms of training.
So, the cars might not be AGI but the system used to build those limited cars could be. After all many people can’t write poetry without training.
So, the cars might not be AGI but the system used to build those limited cars could be. After all many people can’t write poetry without training.
> Yes, in a very very very very very limited environment like a board game.
They work in math and coding, and these two are quite general.
> CodeRL is a new framework for program synthesis through holistic integration of pretrained language models and deep reinforcement learning. By utilizing unit test feedback as part of model training and inference, and integrating with an improved CodeT5 model, CodeRL achieves state-of-the-art results on competition-level programming tasks.
https://blog.salesforceairesearch.com/coderl/
They work in math and coding, and these two are quite general.
> CodeRL is a new framework for program synthesis through holistic integration of pretrained language models and deep reinforcement learning. By utilizing unit test feedback as part of model training and inference, and integrating with an improved CodeT5 model, CodeRL achieves state-of-the-art results on competition-level programming tasks.
https://blog.salesforceairesearch.com/coderl/
Programming competitions are still extremely narrow games, nowhere similar to the vast majority of real-life programming tasks. They have narrowly defined inputs and outputs, and even these are described in formal language.
It's nice that CodeRL can solve them, but to say that they work in "coding" and "math" is a huge stretch.
It's nice that CodeRL can solve them, but to say that they work in "coding" and "math" is a huge stretch.
I agree with the two ways in general
What if, the second way is easy? So long as you can understand and interface with the world effectively, which the first provides. From there what if it was just act and observe and a fitness function
What if, the second way is easy? So long as you can understand and interface with the world effectively, which the first provides. From there what if it was just act and observe and a fitness function
[deleted]
I think thinking in terms of affordances and goals already makes a lot of assumptions about how a potential AGI would work, but imo the kind of "radical emergence" talked about in the article is totally within the realm of possible applications of modern machine learning.
If you think about how a language model works, it's not totally clear that it should work at all.
Humans use sentences to communicate pre-existing ideas - ie. we already have the object, subject and action in our heads prior to forming the sentence. In order to generate realistic sentences, an LLM needs some ability to plan ahead, so that when generating the logits for the first token, it needs some idea of how the sentence will end.
But if you think about it, nothing about next-token prediction should necessarily lead to this planning ability. It can easily get into a feedback loop of predict the same token over and over (and small LMs do in fact do this) We also didn't tell it explicitly that sentences should have objects, subjects and actions, these behaviors are purely emergent from the task of next token prediction.
In a similar vein, I think artificial agents would not need explicit goals or affordances, just a sufficiently complex environment that a large model would not overfit on.
If you think about how a language model works, it's not totally clear that it should work at all.
Humans use sentences to communicate pre-existing ideas - ie. we already have the object, subject and action in our heads prior to forming the sentence. In order to generate realistic sentences, an LLM needs some ability to plan ahead, so that when generating the logits for the first token, it needs some idea of how the sentence will end.
But if you think about it, nothing about next-token prediction should necessarily lead to this planning ability. It can easily get into a feedback loop of predict the same token over and over (and small LMs do in fact do this) We also didn't tell it explicitly that sentences should have objects, subjects and actions, these behaviors are purely emergent from the task of next token prediction.
In a similar vein, I think artificial agents would not need explicit goals or affordances, just a sufficiently complex environment that a large model would not overfit on.
Yeah, and the idea that these programs can't have affordances is silly reliance on a definition that ignores what can happen.
It clearly proves too much - humans have a limited action space - they can only move muscles. So they can't truly explore a larger action space, so they cannot be general intelligences.
But more specifically, if something is within an LLM's action space, whether or not you call it an affordance doesn't change whether it gets explored and used. And perhaps you'd argue that their action space is limited because they are in a box. But hook an LLM up to the internet to allow it to query and retrieve data, and suddenly the action space is essentially infinite. So the limitation isn't the model, it's how the model was hobbled by being denied access to the world.
It clearly proves too much - humans have a limited action space - they can only move muscles. So they can't truly explore a larger action space, so they cannot be general intelligences.
But more specifically, if something is within an LLM's action space, whether or not you call it an affordance doesn't change whether it gets explored and used. And perhaps you'd argue that their action space is limited because they are in a box. But hook an LLM up to the internet to allow it to query and retrieve data, and suddenly the action space is essentially infinite. So the limitation isn't the model, it's how the model was hobbled by being denied access to the world.
> We elaborate on the relation between affordances and algorithms—defined as computational processes that can run on universal Turing machines—ultimately arriving at the conclusion that identifying and leveraging affordances goes beyond algorithmic computation.
Brains are physical systems, and physical systems can be simulated by algorithms. Right? Do the authors think this 'jury-rigging' is some non-physical process?
Brains are physical systems, and physical systems can be simulated by algorithms. Right? Do the authors think this 'jury-rigging' is some non-physical process?
>... implying that algorithmic attempts at creating such dynamics in the field of artificial life (aLife) are doomed to fail.
I guess they are so meta they wondered if a dedicated AI could output an AGI agent, algorithmically. As far as their colloquial definition which speaks of jury-rigging is truly intelligent, I suspect that the road to AGI would require so much human input and reboots (have you tried turning it on and off again?) that it is not predictable by any physical theory. So it's non-physical in a strong definition of physics.
On the other hand, since they argue against classical computing, it would necessarily rely on innovative developments in various hard sciences.
I guess they are so meta they wondered if a dedicated AI could output an AGI agent, algorithmically. As far as their colloquial definition which speaks of jury-rigging is truly intelligent, I suspect that the road to AGI would require so much human input and reboots (have you tried turning it on and off again?) that it is not predictable by any physical theory. So it's non-physical in a strong definition of physics.
On the other hand, since they argue against classical computing, it would necessarily rely on innovative developments in various hard sciences.
What do you mean by simulation? You can simulate an airplane without considering air resistance; it’s not necessarily an accurate simulation.
I find it amusing that so many people is skeptic about general AI, trying to come up with arguments that it will never happen, while at the same time we do not understand how our own intelligence work. Go figure out.
Well then it's funny to think we can build something we don't even have a good definition for? Sounds almost a bit insane?
Mog: “What truly is fire? The divine blessing stolen by Prometheus? Concentrated Phlogiston? The element of change? Is it not madness to seek to create something we don’t even have a good definition of?”
Grog: “Grog rubs two sticks together” Lowers voice and looks around furtively “really hard.”
Grog: “Grog rubs two sticks together” Lowers voice and looks around furtively “really hard.”
You misunderstood my point...
Fire is warm and it produces light, and it's easily observable in nature, it's a "thing" which humans could easily describe and recognize, before reproducing it. Someone was rubbing two sticks together, and noticed things became warmer, and warmer as they went, starting to feel like the warmth of fire and it happened.
In the case of AGI, I don't feel like we have a definition of done, so it seems kind of crazy to be chasing it / throwing money at it.
I didn't say say I'm against it, but does seem like a crazy way to go about it. Keep producing models that one day might mimic intelligence as we know it?
Fire is warm and it produces light, and it's easily observable in nature, it's a "thing" which humans could easily describe and recognize, before reproducing it. Someone was rubbing two sticks together, and noticed things became warmer, and warmer as they went, starting to feel like the warmth of fire and it happened.
In the case of AGI, I don't feel like we have a definition of done, so it seems kind of crazy to be chasing it / throwing money at it.
I didn't say say I'm against it, but does seem like a crazy way to go about it. Keep producing models that one day might mimic intelligence as we know it?
Grog could define fire in a very real way even back then, which is how he knew he’d so easily created it . It is hard for us to even know intelligence (the one we mean in AGI) when we see it, much less create it, no?
We can define intelligence in a very real, practical way now. We see and identify intelligence all the time in humans and in animals and in AI. We may not be perfect at identifying it (just like grog might mistake a rising sun for a forest fire), but we don't need a perfect mathematical or philosophical definition that we all agree on to create it. We just need to rub sticks together really hard.
The person you originally replied to and I disagree that we have any real, practical definition. I can recognize what humans and to an extent animals do as intelligent, but haven’t seen a definition that separates that intelligent behavior from them. I have never seen anything that’s been called ai do something I could call intelligent in that animal-like sense (though some have been impressive in the same way Google / page rank was impressive when it first came out)
So, I don’t see why rubbing these statistical model sticks should suddenly burst into intelligence, but I’m open to seeing convincing reasoning on that at some point. I wouldn’t invest time or energy in the meantime and like that original poster, think it’s kinda insane to if my goal was to see human-like intelligence emerge outside of humans
So, I don’t see why rubbing these statistical model sticks should suddenly burst into intelligence, but I’m open to seeing convincing reasoning on that at some point. I wouldn’t invest time or energy in the meantime and like that original poster, think it’s kinda insane to if my goal was to see human-like intelligence emerge outside of humans
It is interesting that you can write thirteen posts on the topic without being able to define it.
It also seems very odd that you can differentiate between some things that you think are intelligent, and some things that you think definitely are not, yet you are incapable of extracting any sort of goal from that knowledge.
If you could tell us your criteria, perhaps we could help you with that...
It also seems very odd that you can differentiate between some things that you think are intelligent, and some things that you think definitely are not, yet you are incapable of extracting any sort of goal from that knowledge.
If you could tell us your criteria, perhaps we could help you with that...
I’m simply very curious about the subject, it’s super important :)! Given that, I’m also frustrated with what seems like a popular lack of critical thought and curiosity on the specifics.
In these comments, when I’ve talked about an intelligence I can distinguish, I’ve been talking about human / animal intelligence. AGI implies an intelligence independent of that, so I’m asking about the specifics there - what are we calling intelligence if not “what humans do”?
If we are calling it just that, then I’d argue everything I know about how these models do things is very different from what I know of how humans approach the specific tasks the models are built against. And I’ve read that that’s intentional. So, even with that sort of definition I don’t see how it follows that these approaches are on any linear path to AGI (maybe nonlinear if we learn limits and such from mistakes).
I’ve since read more of the article (it’s long, huh?) I like the framework they use from Roitblat in Section 2 - and again, don’t see how LLMs and such are on the road to fulfilling those criteria.
In these comments, when I’ve talked about an intelligence I can distinguish, I’ve been talking about human / animal intelligence. AGI implies an intelligence independent of that, so I’m asking about the specifics there - what are we calling intelligence if not “what humans do”?
If we are calling it just that, then I’d argue everything I know about how these models do things is very different from what I know of how humans approach the specific tasks the models are built against. And I’ve read that that’s intentional. So, even with that sort of definition I don’t see how it follows that these approaches are on any linear path to AGI (maybe nonlinear if we learn limits and such from mistakes).
I’ve since read more of the article (it’s long, huh?) I like the framework they use from Roitblat in Section 2 - and again, don’t see how LLMs and such are on the road to fulfilling those criteria.
Fair enough, though I feel you are a bit too eager to push back against ideas that go counter to your initial thoughts. Of course, because I hold differing opinions, you could reasonably object that it is just what I would say!
I have a different idea of what AGI means: in my view, it is a retronym created in the 1980s in order to refer to AI of the sort Turing envisioned (which was more or less "what humans do") and differentiate it from things that were then being called AI, such as IBM's Deep Blue, which were mostly brute force applied to conceptually narrow problems.
You mentioned Roitblat's framework, and I would draw your attention to one aspect of it: it is not just a list of things that humans do, but those things which humans do considerably better than other animals, yet for all of them, there are other species that do them to some extent. As an evolutionist, I suppose there was a relatively recent time in the past when some of our ancestors or sibling species (all now extinct) had some or all of these skills to some intermediate level. In this view, intelligence is not an all-or-nothing concept, and achieving some of it is still progress.
Here's a view which you may not have seen: the pace of progress in AGI has not been constrained by an inability to define what we want, but by the pace at which we see ways to make what we see we need. For example, it is clear that current LLMs have a problem with truth, but it is not clear from what has been made public so far that anyone has a solution. Some people think that what's being done now with LLMs, but more of it, will be enough to get us to what will be generally accepted as AGI; I am skeptical, but I am willing to be persuaded otherwise if the evidence warrants it.
I have a different idea of what AGI means: in my view, it is a retronym created in the 1980s in order to refer to AI of the sort Turing envisioned (which was more or less "what humans do") and differentiate it from things that were then being called AI, such as IBM's Deep Blue, which were mostly brute force applied to conceptually narrow problems.
You mentioned Roitblat's framework, and I would draw your attention to one aspect of it: it is not just a list of things that humans do, but those things which humans do considerably better than other animals, yet for all of them, there are other species that do them to some extent. As an evolutionist, I suppose there was a relatively recent time in the past when some of our ancestors or sibling species (all now extinct) had some or all of these skills to some intermediate level. In this view, intelligence is not an all-or-nothing concept, and achieving some of it is still progress.
Here's a view which you may not have seen: the pace of progress in AGI has not been constrained by an inability to define what we want, but by the pace at which we see ways to make what we see we need. For example, it is clear that current LLMs have a problem with truth, but it is not clear from what has been made public so far that anyone has a solution. Some people think that what's being done now with LLMs, but more of it, will be enough to get us to what will be generally accepted as AGI; I am skeptical, but I am willing to be persuaded otherwise if the evidence warrants it.
Not really. Build it, test it, notice it fails to meet what we expect of intelligence under conditions X, tweak it to fix that failure and repeat until we can't find any further failures. Then we'll have a formal model of intelligence that counts as a definition.
How can we expect anything of intelligence (the criteria for testing) if we can’t define it to begin with?
Exactly the way we've been doing it: compare what it produces against known intelligence to find cases where it fails.
What we’ve been doing has produced a lot of impressive stuff, but not intelligence, has it? Again, hard to tell without a good definition.
And we've learned a lot about what is and is not intelligence, and if we keep it up we'll steadily carve out the space that defines it.
It's not hard to tell that it has not produced human-level intelligence, so the process outlined by naasking has not yet run into an insurmountable problem.
We use tons and tons of pharmaceuticals whose mechanism of action is poorly understood at best or in a few cases not at all.
We are even able to predict whether other compounds might work without knowing why just based on structural similarity.
It’s ideal to know the full mechanism and it obviously aids engineering but there’s no reason you have to wait for that to use something. People used fire for millennia before oxidation reduction reactions were understood.
We are even able to predict whether other compounds might work without knowing why just based on structural similarity.
It’s ideal to know the full mechanism and it obviously aids engineering but there’s no reason you have to wait for that to use something. People used fire for millennia before oxidation reduction reactions were understood.
All these types of comments are taking about using the results oh physical phenomena without fully understanding them. This is not the same thing as building human-like intelligence. We are using a Turing machine and we are bound by the limits of Turing machines. Human intelligence is nothing like that or no extraordinary evidence to that effect has been presented. Now, if you create a new computational model that is not equivalent to a Turing machine then maybe you will be on the road to something like but that’s not really where things are at.
In this case, it’s not so much the mechanisms of pharma we wouldn’t know, but what it is we’re even trying to cure - not having a definition of the disease or it’s symptoms and yet trying to engineer a cure would be pretty insane.
We have a very clear objective for a lot of this AI stuff. For the rest like the Stable Diffusion stuff it’s play.
Where could I find a good definition of intelligence? At least the intelligence aimed for when we talk about general ai?
In response to your questions, ChatGPT says, "Intelligence is the ability to process information, think abstractly, and learn from experiences. In the context of artificial intelligence (AI), intelligence typically refers to the ability of a machine or system to perform tasks that typically require human-like thinking, such as learning, problem solving, and decision making. There is no one specific definition of intelligence that is universally accepted, and different researchers and practitioners may have slightly different interpretations of what intelligence means in the context of AI."
Seems a bit circular.
"What's a good definition of intelligence?" "Human intelligence." "What's human intelligence?" "What we want AI intelligence to be like."
Seems a bit circular.
"What's a good definition of intelligence?" "Human intelligence." "What's human intelligence?" "What we want AI intelligence to be like."
We must improve this thing so it can finally tell us definitively what we’re building :)
Just imagine if someone actually have success and an AGI just start happily chatting about everything, properly solving general problems like a 30 something engineer, mathematician or whatever.
No one in the planet would know how to be sure it's not just emulating human behavior.
No one in the planet would know how to be sure it's not just emulating human behavior.
Sorry but your argument doesn’t make sense. Engineering/building is different than understanding how things works. Surely is related: if you build something you usually can learn how things work - and vice-versa.
Insane to me is to affirm that something has a limit when you don’t fully understand it.
Insane to me is to affirm that something has a limit when you don’t fully understand it.
What are other examples of things we engineer before knowing what they are? I’m honestly having trouble thinking of any.
And I think it’s a shame we haven’t considered what the limits are on “intelligence” very much. Knowing them has been immeasurably valuable in software engineering and algorithms, for one.
And I think it’s a shame we haven’t considered what the limits are on “intelligence” very much. Knowing them has been immeasurably valuable in software engineering and algorithms, for one.
I agree with you. I also think it is useful to understand limits. This is not what I’m arguing against, it is affirming limits without the knowledge to do so. Until we don’t really figure out how our intelligence works we can’t say we can’t reproduce it algorithmically.
We can find plenty of examples where engineering advanced beyond our understanding: bridges, boats, tables, etc. When humanity started building those we didn’t have the full picture of our physics yet we build and used those for more than 1000s years.
Again, I’m not saying we can’t engineer something from knowledge, I’m saying we can’t affirm limits onto something we don’t fully understand yet.
IMHO this paper is speculative and biased towards our human intelligence. From all the advances I have seen so far in the AI landscape, I’m growing more and more skeptic that our intelligence is something so complex that we can’t replicate.
We can find plenty of examples where engineering advanced beyond our understanding: bridges, boats, tables, etc. When humanity started building those we didn’t have the full picture of our physics yet we build and used those for more than 1000s years.
Again, I’m not saying we can’t engineer something from knowledge, I’m saying we can’t affirm limits onto something we don’t fully understand yet.
IMHO this paper is speculative and biased towards our human intelligence. From all the advances I have seen so far in the AI landscape, I’m growing more and more skeptic that our intelligence is something so complex that we can’t replicate.
We built airplanes before really understanding biological flight (especially e.g. bees).
We didn't know what electricity really was when we created electric lightbulbs.
That analogy doesn’t work. We knew we were trying to build a light bulb. There were properties of electricity, a complex physical phenomena, that we did not understand. However, we have a rigorous understanding of Turing machines. We have a nascent understanding of human intelligence.
We don't have a rigorous understanding of neural networks.
Hadn’t it been thoroughly studied and understood for over 100 years at that point? Ben Franklin was studying it, after all.
Electron was discovered 1897 but the first lightbulb was 1802.
There are more examples as well, as another user has commented, we didn't know what fire is until relatively recently but we have been using it for thousands of years.
There are more examples as well, as another user has commented, we didn't know what fire is until relatively recently but we have been using it for thousands of years.
That sounds a bit early - whats your source? This one says at least mid 19th century for not practical electric lights https://www.energy.gov/articles/history-light-bulb
But either way, is knowing the electron knowing electricity? There are so many properties of it that can be known and manipulated without that insight- and indeed they built up that understanding to reach practical engineering and use of electricity. That’s what I think is being gotten at wrt intelligence.
“Knowing” something isn’t necessarily about being aware of its smaller parts.
But either way, is knowing the electron knowing electricity? There are so many properties of it that can be known and manipulated without that insight- and indeed they built up that understanding to reach practical engineering and use of electricity. That’s what I think is being gotten at wrt intelligence.
“Knowing” something isn’t necessarily about being aware of its smaller parts.
> That sounds a bit early - whats your source?
The source you linked already cites it as the first arc lamp.
> But either way, is knowing the electron knowing electricity? There are so many properties of it that can be known and manipulated without that insight- and indeed they built up that understanding to reach practical engineering and use of electricity. That’s what I think is being gotten at wrt intelligence.
Yes. That is exactly my point. We don't need to entirely understand what intelligence is in order to be able to create it. The same way we didn't know what fire is, but we created it with no problem.
The source you linked already cites it as the first arc lamp.
> But either way, is knowing the electron knowing electricity? There are so many properties of it that can be known and manipulated without that insight- and indeed they built up that understanding to reach practical engineering and use of electricity. That’s what I think is being gotten at wrt intelligence.
Yes. That is exactly my point. We don't need to entirely understand what intelligence is in order to be able to create it. The same way we didn't know what fire is, but we created it with no problem.
Ok, I’ll give you the lightbulb point :).
But we can hardly define intelligence, let alone “entirely understand” it. A child could give a good , practical definition of fire and manipulate it skillfully thousands of years ago. Not so much us grown adults wrt intelligence today.
But we can hardly define intelligence, let alone “entirely understand” it. A child could give a good , practical definition of fire and manipulate it skillfully thousands of years ago. Not so much us grown adults wrt intelligence today.
Here is a child-like, practical definition of intelligence: "ability to solve problems".
That’s a good start, but doesn’t help us since even the most basic programs can fit that definition and are not what we mean when we say AGI. I have yet to see anything approaching a useful definition of intelligence across several discussions of impressive new language, and other statistical models - it feels like we should have that before talking about making it real!
With fire, even “when things turn from themselves to ashes and produce heat” (which I imagine a prehistoric child could come up with) distinguishes fire usefully from most other phenomena in the world
With fire, even “when things turn from themselves to ashes and produce heat” (which I imagine a prehistoric child could come up with) distinguishes fire usefully from most other phenomena in the world
You are equivocating throughout this thread: you accept that we developed an understanding of fire incrementally, and concurrently developed fire-based technologies, yet you insist it is different for intelligence, without giving any good reason to think so.
The basic problem is that it seems pretty clear that human intelligence is not anything like a Turing machine and no one has presented any computational system not equivalent to a Turing machine. Some might conclude this is a foundational problem.
It does not follow, merely from noting that there are differences between human intelligence and a Turing machine, that no "merely" Turing-equivalent device could display human-level intelligence. Every attempt I have seen so far to carry through that argument either ends up begging the question or becoming an argument from incredulity.
Note that this is not an argument that it is possible, which certainly has not been conclusively established.
Note that this is not an argument that it is possible, which certainly has not been conclusively established.
I think it is on those making the claim that a system equivalent to a Turing machine can have human level or superior intelligence. Humans are vastly more intelligent than any other organism we have encountered. I may be incredulous at the idea but the scale of difference between human intelligence and programs based on Turing machines is enormous.
The wholw point of turing machines is that they can emulate any computable system. You could have your a turing machine simulate every atom inside a human, and then you trivially have a turing machine that shows intelligence equivalent to a human.
Their argument that Turing machines cannot be free from immediacy doesn't make sense to me.
They say that "there are no actions that emanate from the historicity of internal organization". I have no idea what that means. Is this an argument about free will? If so, what would be the magic ingredient that saves human brains from immediacy? It can't be memory, because Turing machines have memory as well.
From the article:
> it is generally assumed that input-output processing is performed by some sort of algorithm that can be implemented on a universal Turing machine. The problem is that such algorithmic systems have no freedom from immediacy, since all their outputs are determined entirely—even though often in intricate and probabilistic ways—by the inputs of the system. There are no actions that emanate from the historicity of internal organization. There is, therefore, no agency at all in an AI “agent.”
They say that "there are no actions that emanate from the historicity of internal organization". I have no idea what that means. Is this an argument about free will? If so, what would be the magic ingredient that saves human brains from immediacy? It can't be memory, because Turing machines have memory as well.
From the article:
> it is generally assumed that input-output processing is performed by some sort of algorithm that can be implemented on a universal Turing machine. The problem is that such algorithmic systems have no freedom from immediacy, since all their outputs are determined entirely—even though often in intricate and probabilistic ways—by the inputs of the system. There are no actions that emanate from the historicity of internal organization. There is, therefore, no agency at all in an AI “agent.”
>If so, what would be the magic ingredient that saves human brains from immediacy? It can't be memory, because Turing machines have memory as well.
The universe is not deterministic. There is one brain state at time T, but at time T+1 there are many possible brain states, all with different possibilities. What determines which state at time T+1 the consciousness at time T perceives itself as progressing to? If we define this as a function, which associates states at time T with states at times T+1, then this function loosely fits the definition of "free will". We can't actually construct this function or say anything about it, but we can say it can't not exist (as consciousness perceives itself as progressing to a single future state), which by the law of the excluded middle implies it must exist. The definition doesn't however prevent it from essentially being completely random.
The universe is not deterministic. There is one brain state at time T, but at time T+1 there are many possible brain states, all with different possibilities. What determines which state at time T+1 the consciousness at time T perceives itself as progressing to? If we define this as a function, which associates states at time T with states at times T+1, then this function loosely fits the definition of "free will". We can't actually construct this function or say anything about it, but we can say it can't not exist (as consciousness perceives itself as progressing to a single future state), which by the law of the excluded middle implies it must exist. The definition doesn't however prevent it from essentially being completely random.
"The universe is not deterministic." Maybe not. I think that's a big claim, and it's being made, not proved.
But let's say it's right, ignoring the massive burden of proof. We can hook up a quantum random number generator to GPT-3, and suddenly it can be conscious? If that's the whole argument, it seems like a useless point anyways.
But let's say it's right, ignoring the massive burden of proof. We can hook up a quantum random number generator to GPT-3, and suddenly it can be conscious? If that's the whole argument, it seems like a useless point anyways.
> The universe is not deterministic. There is one brain state at time T, but at time T+1 there are many possible brain states, all with different possibilities.
There is no proof of this. There are deterministic interpretations of quantum mechanics, for instance.
Also, there is little reason to think our brains depend on quantum indeterminacy, even if it exists. After all, a brain that behaves erratically/non-deterministically is the exact opposite of useful. Once you learn how to walk, you want to be able to rely on that ability. Once you learn what food is, you don't want to erratically start eating rocks or twigs.
Non-determinism in the brain would be selected against, not selected for. Flexibility would be selected for, but flexibility can be deterministic.
There is no proof of this. There are deterministic interpretations of quantum mechanics, for instance.
Also, there is little reason to think our brains depend on quantum indeterminacy, even if it exists. After all, a brain that behaves erratically/non-deterministically is the exact opposite of useful. Once you learn how to walk, you want to be able to rely on that ability. Once you learn what food is, you don't want to erratically start eating rocks or twigs.
Non-determinism in the brain would be selected against, not selected for. Flexibility would be selected for, but flexibility can be deterministic.
Non-deterministic Turing machines are still computationally equivalent to Turing machines. That is, there is no problem that a Non-deterministic Turing machine can solve for which there doesn't exist a deterministic Turing machine that solves it as well, and vice versa. Non-deterministic Turing machines are exponentially faster as far as we know (P ≠ NP seems to be true) but they are no more powerful. And note that Non-deterministic algorithms are far more powerful than anyone imagines human brains could be - as they can perform an infinite number of operations in parallel (equivalently, they have an oracle that gives them the best possible choice at any step).
Even worse for your argument, probabilistic Turing machines and algorithms (classical bounded-error probabilistic machines) don't even seem to be faster than fully deterministic ones (P = BEP).
Even worse for your argument, probabilistic Turing machines and algorithms (classical bounded-error probabilistic machines) don't even seem to be faster than fully deterministic ones (P = BEP).
let's talk about real stuff, a live running model right now. chatGPT is currently storing chats with it, if somehow openAI got to re-feed - even taking offline and redeploying the model after that - those previous chats into the tokens, wualá, you've got an AI fred from immediacy.
Practical stuff like "store memories", "replace human senses", "AIs can't walk the world as humans", etc. are feasible to be solved with current commodity technology, even without hypothetical future highly capable robots.
I'll divert a little from the exact answer to the comment..
I've said it in another comment, it looks like Tiktok could be exactly this kind of "eyes and ears", it could be the - one of the - perfect continuously information collection systems "looking into the world" for an strong AI (yeah, right now), but most certainly is just an incredible business model turned into an almost free giant datasets generator.
I cannot get to think that someone could have created tiktok to "see into the world" for AIs / create datasets, from the scratch, but it is possible too.
Practical stuff like "store memories", "replace human senses", "AIs can't walk the world as humans", etc. are feasible to be solved with current commodity technology, even without hypothetical future highly capable robots.
I'll divert a little from the exact answer to the comment..
I've said it in another comment, it looks like Tiktok could be exactly this kind of "eyes and ears", it could be the - one of the - perfect continuously information collection systems "looking into the world" for an strong AI (yeah, right now), but most certainly is just an incredible business model turned into an almost free giant datasets generator.
I cannot get to think that someone could have created tiktok to "see into the world" for AIs / create datasets, from the scratch, but it is possible too.
That might be like a chat with itself, it seems trivial to me, implementation wise and it seems nigh impossible ethics wise
If I were Timnit Gebru and I saw that Google was trying to implement reflection in an AI when i felt the ethics engineering hadn’t caught up to the software engineering, I might quit
If I were Timnit Gebru and I saw that Google was trying to implement reflection in an AI when i felt the ethics engineering hadn’t caught up to the software engineering, I might quit
If you get bored by biological exceptionalism and hairsplitting you can skip this article. There's nothing new here.
[deleted]
I think we mostly recognise pattern of the world. We train ourselves by minimizing surprise, improving our predictive capabilities.
The same is true of AI at the moment.
The affordance is another pattern (tool) we train ourselves all our life with many tools: hammer, knife, cisors, tweezers and more complex ones: car, backhoe, computers. There is a lot of overlap from one tool to another one, most of the pattern transfer easily from one to another. That’s why we don’t have to retrain ourselves if we change the size of the knife we use, it’s a tiny adjustment done in the first few seconds of use.
The same is true of AI at the moment.
The affordance is another pattern (tool) we train ourselves all our life with many tools: hammer, knife, cisors, tweezers and more complex ones: car, backhoe, computers. There is a lot of overlap from one tool to another one, most of the pattern transfer easily from one to another. That’s why we don’t have to retrain ourselves if we change the size of the knife we use, it’s a tiny adjustment done in the first few seconds of use.
See also: On the Impossibility of Supersized Machines https://arxiv.org/abs/1703.10987
Great parody paper.
Limited speed of signal transmission, limited amount of matter and energy per volume of space, topology of physical space, quantum uncertainty, tunneling, and other quantum effects.
The authors make a classic mistake: the brain is not comparable to a Turing Machine, it is a finite state machine. There’s no infinite tape in a brain.
The infinite tape doesn't have to be physically inside the brain. The tape is the universe itself, we read it through senses and write to it through actions.
Brain is not the Turing Machine, but it's the controller part of it, which we often treat as the same thing.
We have multiple controllers (brains) operating on a shared tape (universe).
Brain is not the Turing Machine, but it's the controller part of it, which we often treat as the same thing.
We have multiple controllers (brains) operating on a shared tape (universe).
The universe is finite, so Turing machines are still more powerful. We know the universe is finite because it has a finite volume, and even if it were to expand infinitely, which is debatable, it still has finite extent in spacetime because of heat death.
The brain is not a finite state machine. If it is then it is clearly equivalent to a Turing machine. It seems like your position maybe is just based on there not being “infinite tape” or something but human intelligence is not anything like any computational model we have conceived, or at least no one has presented any such evidence.
> The brain is not a finite state machine.
Yes it is, all finite volumes contain finite information. This is a consequence of physics called the Bekenstein Bound. This means the brain can be fully captured by a finite state machine. It has a very large state space, but it's still finite.
Yes it is, all finite volumes contain finite information. This is a consequence of physics called the Bekenstein Bound. This means the brain can be fully captured by a finite state machine. It has a very large state space, but it's still finite.
I mean you can quote those bounds all you like but this is not what a “finite state machine” is. You are using the words in a nominalistic way but for people in computing, we know very well what a finite state machine is and it encompasses things far greater than the “Bekenstein Bound”. It’s no problem to consider more states in an FSM than there are particles in the universe. Of course, this is silly because what you are describing as an FSM is not.
So you're saying that a set of finite states governed by a set of finite state transitions is not a finite state machine. Ok buddy.
Don't confuse expressiveness and state space. Even if there are more faithful encodings that better preserve other properties, what I said is strictly true and it's important for people to understand that human cognition is not as powerful as some think: a finite state space means a human can be fully captured by a finite state machine.
Don't confuse expressiveness and state space. Even if there are more faithful encodings that better preserve other properties, what I said is strictly true and it's important for people to understand that human cognition is not as powerful as some think: a finite state space means a human can be fully captured by a finite state machine.
Brain can create new states, so by induction it is not a fsm
What exactly is an example of a new state?
Learning, creativity, imagination etc. (Yeah I'm pulling things out of my ass, but that's discussion about brain for you)
Your brain has a finite number of particles. The information content of your brain must be encoded in those particles. Thus, the brain's state space is finite.
A finite set of configurations is enumerable, and can be mapped to any other finite set of same or larger size, like a computer's memory, with no loss of information.
Therefore, any state your brain can enter can similarly be created within a computer, in principle.
A finite set of configurations is enumerable, and can be mapped to any other finite set of same or larger size, like a computer's memory, with no loss of information.
Therefore, any state your brain can enter can similarly be created within a computer, in principle.
I think you need to revisit your proof. Human intelligence is more complicated than that. It is not using encoding in the way you are claiming or if it is this is such a bizarre concept that it requires evidence. We don’t really understand how humans think but experientially it is a phenomena that is more complicated than a process that follows from a small set of simple rules.
More complicated than what? Rule 110 is sufficient to compute all computable functions. All the richness of the internet today can be produced from Rule 110 alone. You'd be surprised how much complexity can arise from simple rules.
Furthermore, the proof I laid out is a corollary of simple physics. If you want a more rigorous version, look up the Bekenstein Bound, which proves conclusively that any finite volume can only contain finite information. Your body is a finite volume, therefore it contains finite information, therefore it can be captured by a finite state machine.
Edit: I just realized I also replied to you above, so I'll take up the thread there.
Furthermore, the proof I laid out is a corollary of simple physics. If you want a more rigorous version, look up the Bekenstein Bound, which proves conclusively that any finite volume can only contain finite information. Your body is a finite volume, therefore it contains finite information, therefore it can be captured by a finite state machine.
Edit: I just realized I also replied to you above, so I'll take up the thread there.
> any state your brain can enter can similarly be created within a computer, in principle
Not necessarily: https://en.wikipedia.org/wiki/Orchestrated_objective_reducti...
Not necessarily: https://en.wikipedia.org/wiki/Orchestrated_objective_reducti...
All arguments disputing a computable brain reduce to claiming that something exists beyond physics or claiming that physics is non-computable in some way. There is no evidence for either of these, Penrose's theory included.
Quantum physics is still physics. The fact that it's not computable (non-deterministic) is not even disputable at this point, it's the nature of reality.
The only questionable thing is whether quantum effects are essential in brain activity.
The only questionable thing is whether quantum effects are essential in brain activity.
> The fact that it's not computable (non-deterministic) is not even disputable at this point, it's the nature of reality.
Of course it's disputable. There are at least two well known deterministic interpretations of quantum mechanics that are indistinguishable from orthodox QM, Many Worlds and Bohmian mechanics.
Of course it's disputable. There are at least two well known deterministic interpretations of quantum mechanics that are indistinguishable from orthodox QM, Many Worlds and Bohmian mechanics.
One doesn’t have to claim it is beyond physics, but no one has shown it can be simulated by a Turing machine.
The brain isn't a fsm simply because I can create a game with an aribtrary number of rules. If I can do that, then there are countably infinite fsm just for making games in my brain, "making a game with one rule fsm", "making a game with two rules fsm" etc.
> The brain isn't a fsm simply because I can create a game with an aribtrary number of rules.
And so can a finite state machine with the same state space as your brain. Size of state space is not a difference of kind but of degree.
And so can a finite state machine with the same state space as your brain. Size of state space is not a difference of kind but of degree.
The simplest response to that is that as soon as a machine (including a digital computer) is interacting with the real world, it is tapping into a source of entropy and its behavior is no longer determined fully by its programming. Alternatively, if one holds that the universe is deterministic, then living organisms are, which also defeats the author's last stand.