Three questions that keep me up at night(blog.evjang.com)
blog.evjang.com
Three questions that keep me up at night
https://blog.evjang.com/2020/04/three-questions-that-keep-me-up-at-night.html
59 comments
Question 2: This is better left unanswered. We've been building superhuman AIs for at least 300 years which are exploiting reality to their advantage and have started out-competing humans for resources (including by changing laws in their favour). These things we've been building are called corporations, and they're certainly smarter than any individual, richer than any human can become and are now working hard to figure out how their humans can be replaced by more reliable components.
Ask HN: what are the questions that keep you up at night ?
For me: How do viruses really work? How to antiviruses work? I’ve gone into basics of DNA -> RNA -> Ribosome -> protein -> cell organelles -> Cells. We know how to sequence DNA and it’s pretty cheap right now. How do we simulate compiling DNA -> proteins and how they interact with each other? How do you find weak spots in a virus and figure out what proteins will inactivate them ?
2) what is the best course of action for people who lost their jobs? Taking <20k from 401k/IRA should be free of taxes and penalties. Everyone has lost >30% of value since Feb already. Sure most people don’t have 401ks, but those who do could weather this storm a bit better. Should we stop payments for all loans for 2 months. No mortgage, auto loans, student loans etc. stop those foreclosures from happening. No rent.
3) will US have the most deaths? Most likely. A large population + most states haven’t locked down. This means unless >80% of population gets infected and recovers to build herd immunity, or we have a vaccine, we’re in deep trouble. Should we infect everyone >10 and under <30 and in clusters so we build up herd immunity in weeks? And get most of the young population back to work? Like if you’re gonna be sitting at home for two weeks, may as well get sick and build up immunity while we’re at it.
For me: How do viruses really work? How to antiviruses work? I’ve gone into basics of DNA -> RNA -> Ribosome -> protein -> cell organelles -> Cells. We know how to sequence DNA and it’s pretty cheap right now. How do we simulate compiling DNA -> proteins and how they interact with each other? How do you find weak spots in a virus and figure out what proteins will inactivate them ?
2) what is the best course of action for people who lost their jobs? Taking <20k from 401k/IRA should be free of taxes and penalties. Everyone has lost >30% of value since Feb already. Sure most people don’t have 401ks, but those who do could weather this storm a bit better. Should we stop payments for all loans for 2 months. No mortgage, auto loans, student loans etc. stop those foreclosures from happening. No rent.
3) will US have the most deaths? Most likely. A large population + most states haven’t locked down. This means unless >80% of population gets infected and recovers to build herd immunity, or we have a vaccine, we’re in deep trouble. Should we infect everyone >10 and under <30 and in clusters so we build up herd immunity in weeks? And get most of the young population back to work? Like if you’re gonna be sitting at home for two weeks, may as well get sick and build up immunity while we’re at it.
> Many AI researchers, myself included, believe that competitive survival of "living organisms" is the only true way to implement general intelligence.
My gut translates that idea as: Progress is made though War not Markets.
Which seems dubious to say the least.
My gut translates that idea as: Progress is made though War not Markets.
Which seems dubious to say the least.
When you put it that way it makes me more sympathetic to the idea, horrifying as it is. War has been one of the biggest things that led to progress, unfortunately. And we don't seem to have learned how to do without it. Walter Scheidel's work has been interesting about this.
https://www.economist.com/open-future/2018/09/10/can-inequal...
https://www.economist.com/open-future/2018/09/10/can-inequal...
Isn't a market a kind of competitive environment?
Given an appropriate reward function, and multiple independent actors seeking advantage, some might converge on an optimal strategy for attaining AGI.
Anyway, that's the way I read it.
Given an appropriate reward function, and multiple independent actors seeking advantage, some might converge on an optimal strategy for attaining AGI.
Anyway, that's the way I read it.
Sure but exchange is a more important part of it. Even a predator offers protection services.
I know it’s just the moment we’re living in right now but as I read those questions I just shrug, they couldn’t feel less relevant.
The ones on my mind are “how long will large populations be locked down?” “How many people are going to die?” and “how long until this all happens again?”
The ones on my mind are “how long will large populations be locked down?” “How many people are going to die?” and “how long until this all happens again?”
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."
https://news.ycombinator.com/newsguidelines.html
Your comment might be just fine in an in-person conversation, but if you respond to someone this way in a broadcast medium [1], it comes across as a kind of social enforcement and shaming, especially if (like this one was) it sits at the top of a thread accruing mass—which happens easily because because indignation routinely attracts upvotes [2]. The OP has every right to think about what interests him, nor does it somehow follow from this that he doesn't care about people dying.
[1] Which HN is: https://news.ycombinator.com/item?id=22780144
[2] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
https://news.ycombinator.com/newsguidelines.html
Your comment might be just fine in an in-person conversation, but if you respond to someone this way in a broadcast medium [1], it comes across as a kind of social enforcement and shaming, especially if (like this one was) it sits at the top of a thread accruing mass—which happens easily because because indignation routinely attracts upvotes [2]. The OP has every right to think about what interests him, nor does it somehow follow from this that he doesn't care about people dying.
[1] Which HN is: https://news.ycombinator.com/item?id=22780144
[2] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
“The greats never sacrifice the important for the urgent. They handle the immediate problem and still make sure to secure the future.”
The media is incentivized to distract you. Turn off your computer, read a book, and don’t forget to live.
The media is incentivized to distract you. Turn off your computer, read a book, and don’t forget to live.
What an absurd statement. This crisis matters. Social distancing matters. Lockdowns matter. “Stick your fingers in your ears” is awful advice.
But at some point, the medias are rehashing the same statements: Stay home, and follows good practices.
As an individual, there's only so much you can do and there's no need to keep yourself distracted from other matters.
As an individual, there's only so much you can do and there's no need to keep yourself distracted from other matters.
The advice around wearing face masks has changed in the last 48 hours.
You can stay tuned in to the world around you without collapsing on the ground in hysterics.
You can stay tuned in to the world around you without collapsing on the ground in hysterics.
[deleted]
no i think you misunderstand. those three questions the commenter posted aren’t about running from fear or anxiety — some would simply call “questions about how we collectively overcome massive and pressing challenges” more relevant, in the short and long term.
OP here. I agree these 3 are not the most important problems of collective society; these are just ones that I have a personal interest and curiosity in :)
>>> “how long will large populations be locked down?” “How many people are going to die?” and “how long until this all happens again?”
Is it really the large populations that worry you or is it just the risk that you might be next or even more hypocritic, that your investment portfolio is going south? Very bad and innevitable horrors have been happenning for a while(i.e the war Yemen, Iraq) and the "large populations" that you mention had little to no insomnia.
Is it really the large populations that worry you or is it just the risk that you might be next or even more hypocritic, that your investment portfolio is going south? Very bad and innevitable horrors have been happenning for a while(i.e the war Yemen, Iraq) and the "large populations" that you mention had little to no insomnia.
> Very bad and innevitable horrors have been happenning for a while(i.e the war Yemen, Iraq) and the "large populations" that you mention had little to no insomnia.
And they’ve had very little effect on me. It’s not irrational to care at different levels depending on that.
And they’ve had very little effect on me. It’s not irrational to care at different levels depending on that.
It's hipocritic to dissmiss OP's concerns/questions as worthless compared with your own personal concerns disguised as a great care about "large populations". It would not be totally irrational to stay on topic without derailing the conversation.
Humans evolve and adapt, we are very resourceful. We will overcome this and learn how to respond faster and more correctly in the future. Also the future president of the USA will likely be much more intelligent, this was a once in a lifetime concentration of ineptitude in power.
We're usually not very proactive though. Although we adapt and evolve it's usually as a reaction to pressure that can't be ignored, and by that time the cost is high if we do adapt and higher if we don't.
For those who make it through, they'll be wiser and more prepared for similar situations - and their descendants may be too, at least for a while.
For those who make it through, they'll be wiser and more prepared for similar situations - and their descendants may be too, at least for a while.
One could go even shorter term than that.
But life is a long game.
But life is a long game.
1. What is your name?
2. What is your quest?
3. What is your favorite color?
Watch out for that last one.
2. What is your quest?
3. What is your favorite color?
Watch out for that last one.
1. Are you really the head of the Kwik-E-Mart?
2. Really?!
3. You?
https://www.youtube.com/watch?v=FiDY2nXVLZ0
2. Really?!
3. You?
https://www.youtube.com/watch?v=FiDY2nXVLZ0
>Watch out for that last one.
This must be a reference I’m not getting?
This must be a reference I’m not getting?
Monty Python and the Holy Grail
https://youtu.be/0D7hFHfLEyk
https://youtu.be/0D7hFHfLEyk
Watch "Monty Python and The Holy Grail"
Blue!
No, YELLLLLLOOoooooowwwwwwww!
No, YELLLLLLOOoooooowwwwwwww!
What is the capital of Assyria?
Lame, only true question is will this universe die out and all information in it be lost or can it be maintained indefinitely/transfer information to the next universe iteration.
> only true question
you mean like, the last question?
you mean like, the last question?
I guess, the only question that matters.
Unfortunately it does not matter for us, we are too transient.
Sometimes I wonder if we are the critters related to the second question. Of course the next question is how far away are we from meeting the simulation’s spec.
Tldr:
>Q1: Can we imitate "thinking" from only observing behavior?
>Q2: What is the computationally cheapest "organic building block" of an Artificial Life simulation that could lead to human-level AGI?
>Q3: Loschmidt's Paradox: How does T-asymmetry in macroscopic phenomena arise from T-symmetric (microscopic) physical laws?
>Q1: Can we imitate "thinking" from only observing behavior?
>Q2: What is the computationally cheapest "organic building block" of an Artificial Life simulation that could lead to human-level AGI?
>Q3: Loschmidt's Paradox: How does T-asymmetry in macroscopic phenomena arise from T-symmetric (microscopic) physical laws?
> If you don't know how to grasp an object, you can't bring food to your mouth.
Counterpoint: hummingbirds.
It might sound cheap, but discussions of the "basic necessities" of anything so often just overlook obvious counterexamples.
Counterpoint: hummingbirds.
It might sound cheap, but discussions of the "basic necessities" of anything so often just overlook obvious counterexamples.
Hummingbirds actually proves my point here, on how life and death solve the symbol grounding problem.
What is a "stable grasp"? An entire body of classical robotics literature is devoted to answering this problem. I work on robotic grasping research and defining what it means to grasp something successfully is a little tricky unless you have a broader context of what the grasping is for.
Nature doesn't care whether hands exist or not - only that the animal can feed. That is why we see such a spectrum of appendages in the animal kingdom. Some are hand-like, others not at all, and some in between. In some sense, "hand" is a mental construct of human language.
What is a "stable grasp"? An entire body of classical robotics literature is devoted to answering this problem. I work on robotic grasping research and defining what it means to grasp something successfully is a little tricky unless you have a broader context of what the grasping is for.
Nature doesn't care whether hands exist or not - only that the animal can feed. That is why we see such a spectrum of appendages in the animal kingdom. Some are hand-like, others not at all, and some in between. In some sense, "hand" is a mental construct of human language.
> Q1: Can we imitate "thinking" from only observing behavior?
If you believe that the human mind is the result of Darwinian evolution, then you implicitly believe that thinking can arise from interacting with the environment. That does not exactly answer the question, but if anything, it goes beyond it.
On the other hand:
"This could also be addressed via other high-data environments:
1) Observing trader orders on markets and seeing if we can recover the trader's deductive reasoning and beliefs about the future. See if we can observe rational thought (if not rational behavior).
2) Recovering intent and emotions and desire from social network activity."
Before you could even begin to get anything from these streams of symbols, you would have to have some way to give them semantics - to understand what they signify to the actors. I suspect that these would be among the last things you would achieve, not a means to the end.
If you believe that the human mind is the result of Darwinian evolution, then you implicitly believe that thinking can arise from interacting with the environment. That does not exactly answer the question, but if anything, it goes beyond it.
On the other hand:
"This could also be addressed via other high-data environments:
1) Observing trader orders on markets and seeing if we can recover the trader's deductive reasoning and beliefs about the future. See if we can observe rational thought (if not rational behavior).
2) Recovering intent and emotions and desire from social network activity."
Before you could even begin to get anything from these streams of symbols, you would have to have some way to give them semantics - to understand what they signify to the actors. I suspect that these would be among the last things you would achieve, not a means to the end.
Do we already posess the hardware required to run full AI?
Deep fakes can be done on older hardware, we just didn't have the code. Does the same thing apply to AI?
Deep fakes can be done on older hardware, we just didn't have the code. Does the same thing apply to AI?
Q1: Can we imitate "thinking" from only observing behavior?
NO as you are measuring actions and not the process that instigated those actions.
One example - chess - you see the move, but the thinking behind that move may elude. More so a bad move, as their thinking was that it is a good move when it is not.
[EDIT format/spelling]
NO as you are measuring actions and not the process that instigated those actions.
One example - chess - you see the move, but the thinking behind that move may elude. More so a bad move, as their thinking was that it is a good move when it is not.
[EDIT format/spelling]
Q1) By the mere copying of behaviors, no. It is clear that there are at least the a priori intuitions of space, time, and causality necessary for humans to obtain any knowledge. It is within that a priori framework where concepts and then decisions are derived from.
Q2) This would require the knowledge of what is sufficient & necessary for AGI.
Q3) Minsky, Neumann, and most sophisticatedly, Norbert Weiner answered this question. Read page 33 of Weiner's Cybernetics. The real insight, where in what I will now explain in brevity is at the expense of the insight's resolution, is that in the nature of microscopic phenomena there are an exorbitant quantity of fickle components ("equal particles") whose positions, accelerations, velocities are all impossible to record because of that sheer quantity by the nature of this sphere of study. More so, there are then at least second-order coupling effects between all these fickle components due to their constant interaction. As a resolve, we have developed terms of statistical understanding. Weiner cites "cloud, temperature, turbulence, etc." all being concepts which are specifically defined in a statistical context.
Where as in the macroscopic sphere there are far fewer particles much further apart from each other with near certain initial masses, positions, and velocities.
The result? In the microscopic systems, because their properties are precisely defined statistically the distribution which predicts the future nor past for a given system isn't unique. That is, the future state (predicted by the distribution provided by some initial system) could have also been predicted by other distributions describing many other initial systems unless by "some miracle" there is a "tightly defined statistical range".
So are microscopic physics condemned to this paradox? Not necessarily, there may be some way to construct scientific objects from systems phenomena which don't require a statistical conception because they can be precisely defined from a microscopic systems point of view temporally & spatially. However, this would require a revolution in theory and likely mathematics
Q2) This would require the knowledge of what is sufficient & necessary for AGI.
Q3) Minsky, Neumann, and most sophisticatedly, Norbert Weiner answered this question. Read page 33 of Weiner's Cybernetics. The real insight, where in what I will now explain in brevity is at the expense of the insight's resolution, is that in the nature of microscopic phenomena there are an exorbitant quantity of fickle components ("equal particles") whose positions, accelerations, velocities are all impossible to record because of that sheer quantity by the nature of this sphere of study. More so, there are then at least second-order coupling effects between all these fickle components due to their constant interaction. As a resolve, we have developed terms of statistical understanding. Weiner cites "cloud, temperature, turbulence, etc." all being concepts which are specifically defined in a statistical context.
Where as in the macroscopic sphere there are far fewer particles much further apart from each other with near certain initial masses, positions, and velocities.
The result? In the microscopic systems, because their properties are precisely defined statistically the distribution which predicts the future nor past for a given system isn't unique. That is, the future state (predicted by the distribution provided by some initial system) could have also been predicted by other distributions describing many other initial systems unless by "some miracle" there is a "tightly defined statistical range".
So are microscopic physics condemned to this paradox? Not necessarily, there may be some way to construct scientific objects from systems phenomena which don't require a statistical conception because they can be precisely defined from a microscopic systems point of view temporally & spatially. However, this would require a revolution in theory and likely mathematics
I dislike the cybernetic approach because I find that it first commits to epistemic helplessness, saying that we know nothing of particles. Perhaps that was reasonable when QM was so young, but these days we can use QM for a better answer. This isn't complete, but it helps a lot.
First, note that CPT-symmetry can be extended up to the macroscopic level. Einstein's thought experiments about time involve macroscopic entanglement. For example, given that a red ball and a blue ball are each in their own boxes, and one possesses one box, then the box is entangled to have either a red or blue ball inside. We can use linear logic to reason about these situations; one holds the multiplicative disjunction of red and blue.
Second, note that in order to make MWI's numbers work, we need not just branching of possibilities, but also coalescing. Imagine that, in empty space, we suddenly have a single virtual-particle pair; the diagram would look like () with time vertical. At the beginning, we have one universe; during the split, two universes, one for each chirality of the pair; afterwards, one universe. When we split, the probabilities of each possible universe decrease. Now, imagine doing this with more pairs, and verify that the Binomial Theorem manifests. This gives the same rise to macrostate-like entropy as Boltzmann's approach to thermodynamics, but only using QM particles and QM logic; a QM state is likelier when more of the many worlds in MWI would support its existence.
Putting these together, we may conclude that Loschmidt's paradox is an incomplete part of a bigger paradoxical complaint: We can see that there are time-reversible and time-irreversible behaviors, both in particles and in macroscopic systems. The time-reversible behaviors seem to govern resources; the time-irreversible behaviors seem to govern entropy. The two logics for these two sets of behaviors are distinct; linear logic and intuitionistic logic can intertwine, but here they don't appear to do so. Why not?
First, note that CPT-symmetry can be extended up to the macroscopic level. Einstein's thought experiments about time involve macroscopic entanglement. For example, given that a red ball and a blue ball are each in their own boxes, and one possesses one box, then the box is entangled to have either a red or blue ball inside. We can use linear logic to reason about these situations; one holds the multiplicative disjunction of red and blue.
Second, note that in order to make MWI's numbers work, we need not just branching of possibilities, but also coalescing. Imagine that, in empty space, we suddenly have a single virtual-particle pair; the diagram would look like () with time vertical. At the beginning, we have one universe; during the split, two universes, one for each chirality of the pair; afterwards, one universe. When we split, the probabilities of each possible universe decrease. Now, imagine doing this with more pairs, and verify that the Binomial Theorem manifests. This gives the same rise to macrostate-like entropy as Boltzmann's approach to thermodynamics, but only using QM particles and QM logic; a QM state is likelier when more of the many worlds in MWI would support its existence.
Putting these together, we may conclude that Loschmidt's paradox is an incomplete part of a bigger paradoxical complaint: We can see that there are time-reversible and time-irreversible behaviors, both in particles and in macroscopic systems. The time-reversible behaviors seem to govern resources; the time-irreversible behaviors seem to govern entropy. The two logics for these two sets of behaviors are distinct; linear logic and intuitionistic logic can intertwine, but here they don't appear to do so. Why not?
> I dislike the cybernetic approach because I find that it first commits to epistemic helplessness
The confusion here is that my statement has nothing to do with cybernetics. Although it’s taken from his book titled Cybernetics, almost 60% of the book isn’t about cybernetics but instead a philosophy and analysis of Science as it progresses through history.
Though it’s fair to Wiener to preemptively note there is no failure in epistemology in one of the most successful theories which holds ground in neurons, to electrical circuits, traffic engineering, the abstract feedback loop. There’s no assumption that “we know nothing of particles”. Wiener originally wrote the book in 1948 and completed the second edition in 1961. He made significant contributions to quantum mechanics[0]. It’s quite suspect to make such a defamatory remark about Wiener’s cybernetics given his quantum knowledge and the field as a whole.
In fact, what would cybernetics possibly be predicting about the interactions between a particular system of which each component cannot exhibit a property? Surely some property must be being reinforced by means of itself or another auxiliary property.
QM & Cybernetics answer completely separate questions.
You’re missing the bigger picture. The irreversibility in any physical phenomenon is only through scientific paradigms which are probabilistic—such as thermodynamics, temperature, cloud, turbulence etc. These concepts are simply more powerful in dynamical systems, which are always due to microscopic phenomena, where there are no analytical, deterministic solutions for the otherwise favored deterministic paradigms (e.g. three-body problem).
“while when a great number of atoms (of the order of Avogadro’s number) is considered, this energy related to irreversibility becomes so large that its order of magnitude must be taken into account. Consequently, macroscopic irreversibility results related to microscopic irreversibility by flows of photons and amount of atoms involved in the processes.”
[0] https://www.ams.org/books/psapm/052/psapm052-endmatter.pdf [1] https://www.nature.com/articles/srep35796
The confusion here is that my statement has nothing to do with cybernetics. Although it’s taken from his book titled Cybernetics, almost 60% of the book isn’t about cybernetics but instead a philosophy and analysis of Science as it progresses through history.
Though it’s fair to Wiener to preemptively note there is no failure in epistemology in one of the most successful theories which holds ground in neurons, to electrical circuits, traffic engineering, the abstract feedback loop. There’s no assumption that “we know nothing of particles”. Wiener originally wrote the book in 1948 and completed the second edition in 1961. He made significant contributions to quantum mechanics[0]. It’s quite suspect to make such a defamatory remark about Wiener’s cybernetics given his quantum knowledge and the field as a whole.
In fact, what would cybernetics possibly be predicting about the interactions between a particular system of which each component cannot exhibit a property? Surely some property must be being reinforced by means of itself or another auxiliary property.
QM & Cybernetics answer completely separate questions.
You’re missing the bigger picture. The irreversibility in any physical phenomenon is only through scientific paradigms which are probabilistic—such as thermodynamics, temperature, cloud, turbulence etc. These concepts are simply more powerful in dynamical systems, which are always due to microscopic phenomena, where there are no analytical, deterministic solutions for the otherwise favored deterministic paradigms (e.g. three-body problem).
“while when a great number of atoms (of the order of Avogadro’s number) is considered, this energy related to irreversibility becomes so large that its order of magnitude must be taken into account. Consequently, macroscopic irreversibility results related to microscopic irreversibility by flows of photons and amount of atoms involved in the processes.”
[0] https://www.ams.org/books/psapm/052/psapm052-endmatter.pdf [1] https://www.nature.com/articles/srep35796
I can imagine many people would not be comfortable answering that question, particularly depending on the company and their culture. I can easily see this type of question leading to dramatically different conclusions ranging from “this person is a deep thinker on varied topics” all the way to “this person wouldn’t be a good cultural fit”. Don’t think too deeply about the wrong topics.
#2 is explored by John Searle's Chinese Room argument, which is unfortunately completely misunderstood by most technologists working in AI, to their own peril.
This article is an excellent introduction:
http://www.dreamsongs.com/Searle.html
A relevant excerpt:
Searle goes on to contend that intentional behavior must be in response to real contingencies, not formal simulations of reality. If so, it would seem to follow that the capacity for intentional behavior is the result of what might be called phylogenic contingencies, and so would only be found in products of real evolution. In short, what Searle is saying is that, just like other biological activities such as digestion and photosynthesis, thought is intrinsically dependent on the biochemistry of its origin. Just as a formal simulation of digestion is not really digesting, a formal simulation of thought is not really thinking. And to tie this to the gender thought experiment, a formal simulation of gender is not gender, because it lacks biological structural and functional characteristics.
The "Simulation, Duplication and Evolution" section of the SEP article is also useful:
https://plato.stanford.edu/entries/chinese-room/#SimuDuplEvo...
This article is an excellent introduction:
http://www.dreamsongs.com/Searle.html
A relevant excerpt:
Searle goes on to contend that intentional behavior must be in response to real contingencies, not formal simulations of reality. If so, it would seem to follow that the capacity for intentional behavior is the result of what might be called phylogenic contingencies, and so would only be found in products of real evolution. In short, what Searle is saying is that, just like other biological activities such as digestion and photosynthesis, thought is intrinsically dependent on the biochemistry of its origin. Just as a formal simulation of digestion is not really digesting, a formal simulation of thought is not really thinking. And to tie this to the gender thought experiment, a formal simulation of gender is not gender, because it lacks biological structural and functional characteristics.
The "Simulation, Duplication and Evolution" section of the SEP article is also useful:
https://plato.stanford.edu/entries/chinese-room/#SimuDuplEvo...
I agree with Searle insomuch that Life and Death is the real contingency with which gives a reality meaning. If the simulation had some arbitrary cellular-automata ruleset, there is no guarantee that the simulation would ever evolve into something isomorphic to our reality. I have a strong belief that 3D physics + collision/contact forces, conservation of energy, linearity of perspective, and a nutrient cycle are important pieces.
If thought is indeed "intrinsically dependent on biochemistry...", then we have a hard journey ahead.
However, AI philosophers should try their own medicine and present claims with real contingencies, i.e. make claims that are experimentally falsifiable.
Put another way, I'm less interested in debating what "understanding" is and more interested in figuring out how to build a Chinese room to begin with. Once we have such a system, perhaps we might realize that the Chinese Room thought experiment was ill-posed to begin with.
If thought is indeed "intrinsically dependent on biochemistry...", then we have a hard journey ahead.
However, AI philosophers should try their own medicine and present claims with real contingencies, i.e. make claims that are experimentally falsifiable.
Put another way, I'm less interested in debating what "understanding" is and more interested in figuring out how to build a Chinese room to begin with. Once we have such a system, perhaps we might realize that the Chinese Room thought experiment was ill-posed to begin with.
See Mike Adler answer to Searle. To use your digestion metaphor, as long as the input is food and the output is energy and byproducts, AI researchers don't care whether philosophers would call it "real digestion" or a "formal simulation of digestion". Same goes for thinking.
https://philosophynow.org/issues/46/Newtons_Flaming_Laser_Sw...
https://philosophynow.org/issues/46/Newtons_Flaming_Laser_Sw...
This essay is filled with so many falsehoods, assumptions, and lack of knowledge that I don't know where to begin. It's also written with a snarky, arrogant attitude which is not helpful. To list a few:
- "there is every reason to believe that a human brain is a machine"
Given what evidence?
- Platonism died for mathematicians some centuries ago, and simply looks silly.
And yet Frege and Gödel, two of the most accomplished mathematicians of the past century, could be described as Platonists. Clearly this isn't a closed question.
https://en.wikipedia.org/wiki/Philosophy_of_mathematics#Plat...
https://plato.stanford.edu/entries/platonism-mathematics/
Regarding digestion: if the Turing Test is determining whether a computer is thinking like a human being, then focusing only on input and output, and not the 'being', sort of misses the entire point. Remember that Searle's Chinese Room is primarily a criticism of the Turing Test.
As always, the truth is: it's more complicated than it seems.
- "there is every reason to believe that a human brain is a machine"
Given what evidence?
- Platonism died for mathematicians some centuries ago, and simply looks silly.
And yet Frege and Gödel, two of the most accomplished mathematicians of the past century, could be described as Platonists. Clearly this isn't a closed question.
https://en.wikipedia.org/wiki/Philosophy_of_mathematics#Plat...
https://plato.stanford.edu/entries/platonism-mathematics/
Regarding digestion: if the Turing Test is determining whether a computer is thinking like a human being, then focusing only on input and output, and not the 'being', sort of misses the entire point. Remember that Searle's Chinese Room is primarily a criticism of the Turing Test.
As always, the truth is: it's more complicated than it seems.
I certainly agree that Adler's provocative tone is not helpful and his strokes are sometimes too broad to be fully correct as you have rightfully pointed out, but the general point stands.
To me the Turing test gives a falsifiable experiment that defines what "thinking like a human being" means.
It does so by focusing only on the input output. To do that, it puts the system under test out of view of the experimenter and mediates the experiment with written text.
So yes, the input output is the important part, the Turing test, albeit flawed, is interesting, and any claim that machines can't think based on a definition of "think" that does not lend itself to a falsifiable experiment, such as Searle's, is uninteresting.
I will leave it at that and make a quick exit, as Adler suggests.
To me the Turing test gives a falsifiable experiment that defines what "thinking like a human being" means.
It does so by focusing only on the input output. To do that, it puts the system under test out of view of the experimenter and mediates the experiment with written text.
So yes, the input output is the important part, the Turing test, albeit flawed, is interesting, and any claim that machines can't think based on a definition of "think" that does not lend itself to a falsifiable experiment, such as Searle's, is uninteresting.
I will leave it at that and make a quick exit, as Adler suggests.
> Given what evidence?
Well, one could just as well pose the same question to any dualist.
The evidence for minds being physical phenomena comes, at least in part, from the observation that minds appear to be the result of physical brains, which have evolved through a physical process from much simpler precursors, doing physical things. This is far from proof, and is only a hint of an explanation of how it all actually works, but it is as least as good evidence as anything offered by dualists, who do not seem to have anything to offer in the way of a constructive explanation of minds -- they seem to spend all their time arguing against materialism, rather than for a particular idea of how minds work.
Well, one could just as well pose the same question to any dualist.
The evidence for minds being physical phenomena comes, at least in part, from the observation that minds appear to be the result of physical brains, which have evolved through a physical process from much simpler precursors, doing physical things. This is far from proof, and is only a hint of an explanation of how it all actually works, but it is as least as good evidence as anything offered by dualists, who do not seem to have anything to offer in the way of a constructive explanation of minds -- they seem to spend all their time arguing against materialism, rather than for a particular idea of how minds work.
> Searle goes on to contend that intentional behavior must be in response to real contingencies, not formal simulations of reality. If so...
He does indeed, but the Chinese Room paper presents no compelling argument for that view; it ultimately depends on an appeal to one's intuitions about the feasibility of a remarkable and unrealistic situation that is far from our own experiences, and one, moreover, where he misunderstood the most common response (the 'systems reply'). Dennett has objected that these sort of arguments (or, as he calls them, 'intuition pumps') are simply unsuitable for proving anything, comparing them to denying heliocentricity on the grounds of it being just obvious that the Earth is not moving.
He does indeed, but the Chinese Room paper presents no compelling argument for that view; it ultimately depends on an appeal to one's intuitions about the feasibility of a remarkable and unrealistic situation that is far from our own experiences, and one, moreover, where he misunderstood the most common response (the 'systems reply'). Dennett has objected that these sort of arguments (or, as he calls them, 'intuition pumps') are simply unsuitable for proving anything, comparing them to denying heliocentricity on the grounds of it being just obvious that the Earth is not moving.
Searle either misunderstood AI (and computers) or explained himself poorly (as philosophers often do), so misunderstanding Searle isn't a major problem.
I seriously doubt that Searle misunderstands computers, and probably not what passes for AI either then or now. I also regard that paper as one of the more accessible works of the genre, though, as always, you have to pay close attention to what claims are being made.
On the other hand, at least at the time of writing the original paper, he clearly misunderstood the 'systems reply', as his response to it -- supposing that the actor had memorized the instructions -- is beside the point, and makes no difference.
On the other hand, at least at the time of writing the original paper, he clearly misunderstood the 'systems reply', as his response to it -- supposing that the actor had memorized the instructions -- is beside the point, and makes no difference.
One important nuance to the Chinese Room is to keep in mind that, even as a black box, there must be limitations on its behavior. We can infer those limitations.
We start with Aaronson's observation [0] that the Chinese Room's complexity can be analyzed. Let us assume that the Room responds in (N)P time, and in particular not EXP time. Then the dictionary must be poly-sized. Further, the dictionary is finite but the input strings are unbounded in length, so that we must be able to divide and conquer somehow. Therefore we only need log-many steps to walk the dictionary. The dictionary is ordered, and so indexed, and so walked in log-time.
We can conclude that Searle's definition of intelligence is basically the ability to execute certain kinds of basic katamorphisms, akin to pregroup grammars [1] or other categorical grammars; in other words, Searle says that intelligence is merely the ability to talk convincingly like a human, and not necessarily any of the other features that we see across Mammalia.
Searle might retort that the Room knows facts about the world. This would require us to augment our pregroup/category, enriching it with relational information. For example, in the utterance "I like cats", we might annotate each word with a relation or type, and then look them up in a secondary database-book. We can see here that it would then be the database-book which has an opinion on cats, since "I" becomes merely another symbol in the database. This is reminiscent of Hofstadter's critique of Searle (supported by his other work), in which he suggests that "I" is indeed merely a self-referent symbol which leads to a strange loop, or as he puts it, "I am a strange loop." Here, the strangeness of the loop is provided by the ability of "I" to be both a plain word in the textual language and also a referent symbol in the symbolic database.
Edit: Forgot the main takeaway: When Searle and Turing measure intelligence based on a person's ability to respond to textual queries with textual answers, but the entire interface between the tester and the person is the ability to give textual answers to textual questions, then we aren't measuring intelligence anymore. We've struck Goodhart's Law [2] and we can imagine that the most successful people in these tests will be those who can abuse glitches in text-processing systems.
[0] https://www.scottaaronson.com/papers/philos.pdf
[1] https://en.wikipedia.org/wiki/Pregroup_grammar
[2] https://en.wikipedia.org/wiki/Goodhart's_law
We start with Aaronson's observation [0] that the Chinese Room's complexity can be analyzed. Let us assume that the Room responds in (N)P time, and in particular not EXP time. Then the dictionary must be poly-sized. Further, the dictionary is finite but the input strings are unbounded in length, so that we must be able to divide and conquer somehow. Therefore we only need log-many steps to walk the dictionary. The dictionary is ordered, and so indexed, and so walked in log-time.
We can conclude that Searle's definition of intelligence is basically the ability to execute certain kinds of basic katamorphisms, akin to pregroup grammars [1] or other categorical grammars; in other words, Searle says that intelligence is merely the ability to talk convincingly like a human, and not necessarily any of the other features that we see across Mammalia.
Searle might retort that the Room knows facts about the world. This would require us to augment our pregroup/category, enriching it with relational information. For example, in the utterance "I like cats", we might annotate each word with a relation or type, and then look them up in a secondary database-book. We can see here that it would then be the database-book which has an opinion on cats, since "I" becomes merely another symbol in the database. This is reminiscent of Hofstadter's critique of Searle (supported by his other work), in which he suggests that "I" is indeed merely a self-referent symbol which leads to a strange loop, or as he puts it, "I am a strange loop." Here, the strangeness of the loop is provided by the ability of "I" to be both a plain word in the textual language and also a referent symbol in the symbolic database.
Edit: Forgot the main takeaway: When Searle and Turing measure intelligence based on a person's ability to respond to textual queries with textual answers, but the entire interface between the tester and the person is the ability to give textual answers to textual questions, then we aren't measuring intelligence anymore. We've struck Goodhart's Law [2] and we can imagine that the most successful people in these tests will be those who can abuse glitches in text-processing systems.
[0] https://www.scottaaronson.com/papers/philos.pdf
[1] https://en.wikipedia.org/wiki/Pregroup_grammar
[2] https://en.wikipedia.org/wiki/Goodhart's_law
> Searle might retort that the Room knows facts about the world.
IIRC, Searle's claims to show that the instruction book could not be written, and therefore the room could not exist, so he presumably does not attribute any properties or capabilities to it.
IIRC, Searle's claims to show that the instruction book could not be written, and therefore the room could not exist, so he presumably does not attribute any properties or capabilities to it.
Richard Feynman has a good explanation for 3.
http://www.informationphilosopher.com/solutions/scientists/f...
http://www.informationphilosopher.com/solutions/scientists/f...
Thanks! I had a chat with a mathematician friend today, and we came up with a fairly simple resolution actually that I'm pretty satisfied with: the trick here is to stop thinking of particles as T-symmetric billiard balls, and just to assume that there exists microscopic T-asymmetric interactions.
If we assume molecular chaos hypothesis (independent particle velocities prior to collision) and take into account probabilistic collisions occurring at the 2-particle microscopic level, then post-collision velocities are now dependent, thereby imposing an ordering to time.
I have to think more about this though, and I'm still struggling with understanding the circular dependency between entropy "being time" and entropy being "caused" by "motion" (with respect to ... time?)
If we assume molecular chaos hypothesis (independent particle velocities prior to collision) and take into account probabilistic collisions occurring at the 2-particle microscopic level, then post-collision velocities are now dependent, thereby imposing an ordering to time.
I have to think more about this though, and I'm still struggling with understanding the circular dependency between entropy "being time" and entropy being "caused" by "motion" (with respect to ... time?)
Why can't you assume time symmetric billiard balls? If you take a lot of them, black and white, put all the black ones on one side and the white on the other, give them random velocities, and wait a bit, you will end up with them all mixed together. The entropy has increased even though all interactions are reversible. The macroscopic transformation is time asymmetric (the balls will not sort themselves back again) even though the microscopic transformations are time symmetric (elastic collisions and frictionless movement).
The way I think about this (from Feynman) is that entropy measures "how special" a macroscopic state is, on average. When you apply many microscopic independent fluctuations to a given state you are unlikely to end up with a "more special" state. That's how states are ordered at the macroscopic level giving the arrow of time a clear direction.
The way I think about this (from Feynman) is that entropy measures "how special" a macroscopic state is, on average. When you apply many microscopic independent fluctuations to a given state you are unlikely to end up with a "more special" state. That's how states are ordered at the macroscopic level giving the arrow of time a clear direction.
The macroscopic transformation is time asymmetric (the balls will not sort themselves back again) even though the microscopic transformations are time symmetric (elastic collisions and frictionless movement).
I agree with your observation of what happens to the black and white billiards. My question is - if the interactions are all time-symmetric (implies conservation of entropy), where does the entropy come from in the macroscopic system? My belief is that the microscopic model is flawed; if we simply introduce some randomness into the collision dynamics of two particles (which is believable given particle accelerator experiments / QM), then microscopic interactions are no longer T-symmetric.
It comes down to a philosophical interpretation of whether you can treat a system of two particles probabilistically (i.e. thermodynamically in aggregate) or not.
I agree with your observation of what happens to the black and white billiards. My question is - if the interactions are all time-symmetric (implies conservation of entropy), where does the entropy come from in the macroscopic system? My belief is that the microscopic model is flawed; if we simply introduce some randomness into the collision dynamics of two particles (which is believable given particle accelerator experiments / QM), then microscopic interactions are no longer T-symmetric.
It comes down to a philosophical interpretation of whether you can treat a system of two particles probabilistically (i.e. thermodynamically in aggregate) or not.
In this set up, the answer to q1- Intent: No, intent can’t be inferred.
If intent could be inferred- the meaningful and material portion of human privacy would cease to exist.
If intent could be inferred- the meaningful and material portion of human privacy would cease to exist.