AI and Robots Are a Minefield of Cognitive Biases(spectrum.ieee.org)
spectrum.ieee.org
AI and Robots Are a Minefield of Cognitive Biases
https://spectrum.ieee.org/automaton/robotics/robotics-software/humans-cognitive-biases-facing-ai
41 comments
Some interesting ideas, but it’s all conjecture and very little data. I’m sure Dr Kim knows this stuff inside out, but I felt like the content lacked support. Also, there are somewhat contradictory statements. Eg. The point about evolution being equivalent to millions of years of AI training is absurd - In a later sentence, the author states that computers were designed for computation (duh) and so it shouldn’t be a surprise that they are good at that one thing. The whole thing reads as being rushed and superficial.
> The whole thing reads as being rushed and superficial.
Journalism tends to reward this. I used to work in a much more public-facing field where we would regularly get calls from local journalists about projects we were involved in.
The journalists never printed the quotes from the engineers who gave thoughtful, measured, and accurate answers. They always quoted the blowhards and the charlatans in our office who'd inevitably toss together a bunch of meaningless buzzwords, make a series of vague statements that sounded insightful, and play fast and loose with the facts. A journalist's job is to sell newspapers, or clicks, not to keep anyone informed.
Journalism tends to reward this. I used to work in a much more public-facing field where we would regularly get calls from local journalists about projects we were involved in.
The journalists never printed the quotes from the engineers who gave thoughtful, measured, and accurate answers. They always quoted the blowhards and the charlatans in our office who'd inevitably toss together a bunch of meaningless buzzwords, make a series of vague statements that sounded insightful, and play fast and loose with the facts. A journalist's job is to sell newspapers, or clicks, not to keep anyone informed.
"Journalism" doesn't tend to reward this. It is a tautological statement than faster is better, all else being equal, for any type of work.
In this case, the article is by a guest author. They are not a journalist and this isn't, technically, "journalism".
"Journalism" also isn't monolithic enough to warrant the accusation that the trade itself is lowering its standards across-the-board. There are plenty of examples of deeply researched stories or elaborately crafted features on anything from pop culture to science to politics. While some publications may have succumbed to the easy fix that is viral content, and many local papers are but a shade of their former glory because nobody cares, the are plenty of publishers who escaped the other way, to quality.
Your complaint about selective quoting also really isn't really about anything being rushed. I presume you could lock the author in a dungeon for a few hours and they might be willing to say whatever you want, but not because they suddenly agree with it.
If you disagree with the author and work in AI, I would suggest that it isn't entirely impossible that the topic of cognitive biases is relevant here in more than one way.
In this case, the article is by a guest author. They are not a journalist and this isn't, technically, "journalism".
"Journalism" also isn't monolithic enough to warrant the accusation that the trade itself is lowering its standards across-the-board. There are plenty of examples of deeply researched stories or elaborately crafted features on anything from pop culture to science to politics. While some publications may have succumbed to the easy fix that is viral content, and many local papers are but a shade of their former glory because nobody cares, the are plenty of publishers who escaped the other way, to quality.
Your complaint about selective quoting also really isn't really about anything being rushed. I presume you could lock the author in a dungeon for a few hours and they might be willing to say whatever you want, but not because they suddenly agree with it.
If you disagree with the author and work in AI, I would suggest that it isn't entirely impossible that the topic of cognitive biases is relevant here in more than one way.
> While some publications may have succumbed to the easy fix that is viral content, and many local papers are but a shade of their former glory because nobody cares, the are plenty of publishers who escaped the other way, to quality.
Since the former has become the norm, and the latter the exception, the former has become, technically, "journalism". Just another case where a word's meaning has changed due to ubiquity.
Since the former has become the norm, and the latter the exception, the former has become, technically, "journalism". Just another case where a word's meaning has changed due to ubiquity.
> “Journalism" also isn't monolithic enough to warrant the accusation that the trade itself is lowering its standards across-the-board.
It certainly feels that way to me. There are still holdouts of really good journalism, like Reuters, but even the likes of the New York Times seems to have significantly degraded. Good journalism seems to be the exception rather than the rule.
It certainly feels that way to me. There are still holdouts of really good journalism, like Reuters, but even the likes of the New York Times seems to have significantly degraded. Good journalism seems to be the exception rather than the rule.
> A journalist's job is to sell newspapers, or clicks, not to keep anyone informed.
Yes. And to propagandize on behalf of his and his organization's preferred political views. Seldom do they probably consciously realize they are doing this. The echo chamber synergizes with their prejudices and steers them in that direction. And for them, it's a job. Their reward is the editor's approval, or at least the absence of his disapproval.
Yes. And to propagandize on behalf of his and his organization's preferred political views. Seldom do they probably consciously realize they are doing this. The echo chamber synergizes with their prejudices and steers them in that direction. And for them, it's a job. Their reward is the editor's approval, or at least the absence of his disapproval.
There's a Noam Chomsky quote for this:
“I’m sure you believe everything you’re saying. But what I’m saying is that if you believe something different, you wouldn’t be sitting where you’re sitting.”
I’m not sure why you’re being downvoted. Is it controversial that publications have become increasingly partisan in recent years? I thought that was widely agreed upon? Or maybe people disagree with your implication that this is a bad thing (I imagine progressives are pleased that the majority of the old reputable outlets are burning through their credibility reserves to advance progressivism).
I don't know what sort of "data" you might want. There are modes of argumentation that do not rely on double-blind randomised trials.
The argument is, essentially:
- people used to believe that being a chess grandmaster is universally considered proof of high intelligence
- people also believed that opening a refrigerator requires strictly less intelligence than becoming a chess grandmaster
- we now have build artificial "intelligence" that easily beats any human in chess. Ask it to open a refrigerator and see what happens.
-> Therefore, our concept of "intelligence", as a single, ordered, continuous, and measurable quantity, is demonstrably wrong.
You can proof plenty with a single datapoint or two. As long as you don't care about the color, half a sheep proofs sheep exist. There's no n and no p-value attached to that because it is not subject to measuring errors and confounders. Sheep just is.
They article isn't quite clear on how our biases are potentially dangerous, because it happens to be the point of warnings to come before the actual damage occurs. But when someone is setting out to revolutionise all our lives with something that uses the term "intelligence", and neither they nor we can actually come up with definition of the concept, maybe the burden of proof should be on those saying it's all fine, and not those asking for a bit of occasional reflection.
I also just don't get what's controversial about the article. It doesn't even ask for anything dramatic, or specific. It's the MIT robot tech leader, who knows what they're talking about. And when people start invoking the human rights of abused robots, I am confident the robotics community wouldn't object to that being either a joke or misguided anthropomorphising.
The argument is, essentially:
- people used to believe that being a chess grandmaster is universally considered proof of high intelligence
- people also believed that opening a refrigerator requires strictly less intelligence than becoming a chess grandmaster
- we now have build artificial "intelligence" that easily beats any human in chess. Ask it to open a refrigerator and see what happens.
-> Therefore, our concept of "intelligence", as a single, ordered, continuous, and measurable quantity, is demonstrably wrong.
You can proof plenty with a single datapoint or two. As long as you don't care about the color, half a sheep proofs sheep exist. There's no n and no p-value attached to that because it is not subject to measuring errors and confounders. Sheep just is.
They article isn't quite clear on how our biases are potentially dangerous, because it happens to be the point of warnings to come before the actual damage occurs. But when someone is setting out to revolutionise all our lives with something that uses the term "intelligence", and neither they nor we can actually come up with definition of the concept, maybe the burden of proof should be on those saying it's all fine, and not those asking for a bit of occasional reflection.
I also just don't get what's controversial about the article. It doesn't even ask for anything dramatic, or specific. It's the MIT robot tech leader, who knows what they're talking about. And when people start invoking the human rights of abused robots, I am confident the robotics community wouldn't object to that being either a joke or misguided anthropomorphising.
I thought it made a good point.
I remember Microsoft introducing an early AI in a game and they made it a young boy. This seemed like an artful/deceptive attempt to take advantage of human biases to make it seem more inpressive than it was.
Which is fine, up to a point, for showbiz, but probably not ideal for wider applications.
I'm not sure if this technique has a name but I see it more and more like "<unhealthy thing> has more <good quality> than <healthy thing>". They're not lying but they are taking advantage of human biases to very specifically talk about a minor truth to distract you from the overall truth.
"Rinsing your recycling (in water heated by electricity generated by coal) generates more carbon than the recycling saves". Again true, but carefully crafted to make you think the opposite of the truth.
Notably, no evil, mustache-twirling villain is required for this. The people who make boring, reasonable assertions will be automatically filtered out in favour of the people making wild and sloppy claims that appeal to people's biases and when they get caught out they'll be replaced by another similar person.
But a few villains will intentionally take advantage of it too.
I remember Microsoft introducing an early AI in a game and they made it a young boy. This seemed like an artful/deceptive attempt to take advantage of human biases to make it seem more inpressive than it was.
Which is fine, up to a point, for showbiz, but probably not ideal for wider applications.
I'm not sure if this technique has a name but I see it more and more like "<unhealthy thing> has more <good quality> than <healthy thing>". They're not lying but they are taking advantage of human biases to very specifically talk about a minor truth to distract you from the overall truth.
"Rinsing your recycling (in water heated by electricity generated by coal) generates more carbon than the recycling saves". Again true, but carefully crafted to make you think the opposite of the truth.
Notably, no evil, mustache-twirling villain is required for this. The people who make boring, reasonable assertions will be automatically filtered out in favour of the people making wild and sloppy claims that appeal to people's biases and when they get caught out they'll be replaced by another similar person.
But a few villains will intentionally take advantage of it too.
I think your given example could benefit from spelling it out further.
Let’s say that I read "Rinsing your recycling (in water heated by electricity generated by coal) generates more carbon than the recycling saves", investigate it, and convlude it’s true.
What is the additional truth that statement is making me think the opposite of? I lost you there.
Let’s say that I read "Rinsing your recycling (in water heated by electricity generated by coal) generates more carbon than the recycling saves", investigate it, and convlude it’s true.
What is the additional truth that statement is making me think the opposite of? I lost you there.
That multiple, independant, well-regarded and thorough lifecycle analyses of recycling have consistently shown it to be both financially and environmentally (including but not limited to carbon output specifically) beneficial.
edit: I removed a "not" to clear up the ambiguous double-negative. Summary, my position is that science says recycling is a good thing. Numerous media stories will focus relentlessly on any small flaw in it, either to get a reaction from the people who do it and think it's a good thing, or support the people who fight against it for political reasons as it represents (like global warming) a massive failure of free markets.
edit: I removed a "not" to clear up the ambiguous double-negative. Summary, my position is that science says recycling is a good thing. Numerous media stories will focus relentlessly on any small flaw in it, either to get a reaction from the people who do it and think it's a good thing, or support the people who fight against it for political reasons as it represents (like global warming) a massive failure of free markets.
Brains are survival engines, not truth detectors. If self deception promotes fitness, the brain lies. Stops noticing irrelevant things. Truth never matters. Only fitness. By now you don’t experience the world as it exists at all.
- Peter Watts "Blindsight"
Responses to this have been interesting...
There is a lot of evidence that humans are over agressively pattern matchers. Another poster pointed out some examples like the limitations of human vision (there is a hole in your vision that your brain fills in with whatever is thinks is there for example). Look up cognitive biases sometimes, or any old optical illusion. Those are just the fun edge cases where you get to see examples of the lies the over agressively pattern matchers of your sensorium tell you.
I didn't expect this to be controversial. It is a basic assumption of my life that what I perceive is only roughly correlated with "the truth".
Really I meant this as a statement about machine learning doing whatever it's fitness function tells it to, and that the fitness function we select only hopefully correlates with reality.
There is a lot of evidence that humans are over agressively pattern matchers. Another poster pointed out some examples like the limitations of human vision (there is a hole in your vision that your brain fills in with whatever is thinks is there for example). Look up cognitive biases sometimes, or any old optical illusion. Those are just the fun edge cases where you get to see examples of the lies the over agressively pattern matchers of your sensorium tell you.
I didn't expect this to be controversial. It is a basic assumption of my life that what I perceive is only roughly correlated with "the truth".
Really I meant this as a statement about machine learning doing whatever it's fitness function tells it to, and that the fitness function we select only hopefully correlates with reality.
Then the same applies to Mr. Watts’ statement, and we are therefore under no obligation to take it as true.
Not sure I follow your logic. Watts is essentially using deductive reasoning:
1. The set of beliefs that are true are not equivalent to the set of beliefs that promote fitness
2. The brain is designed to promote fitness
3. The brain may hold beliefs that are not true
You are saying that Watts' statement is undermining itself, that the conclusion undermines either of the premises. But neither 1 or 2 is in contradiction to 3.
Now he takes the conclusion a bit further and argues that not only may we hold untrue beliefs but that with time we will hold a substantial number of untrue beliefs. He does not justify this in this quote but again, this statement is not contradicted by 3.
Or are you saying that since he argues that the brain lies to us then we can't trust our brains which in turn invalidates any reasoning demonstrated by a brain?
1. The set of beliefs that are true are not equivalent to the set of beliefs that promote fitness
2. The brain is designed to promote fitness
3. The brain may hold beliefs that are not true
You are saying that Watts' statement is undermining itself, that the conclusion undermines either of the premises. But neither 1 or 2 is in contradiction to 3.
Now he takes the conclusion a bit further and argues that not only may we hold untrue beliefs but that with time we will hold a substantial number of untrue beliefs. He does not justify this in this quote but again, this statement is not contradicted by 3.
Or are you saying that since he argues that the brain lies to us then we can't trust our brains which in turn invalidates any reasoning demonstrated by a brain?
In what way is the object in your first premise perceptible to the subject in Mr. Watts’ quote? It would require a faculty “higher” than the brain, which is restricted in access to only a subset of the object, no?
Ah, ok. So you are saying that we cannot know whether or not the first premise is true because the brain does not have full knowledge of its own beliefs.
I would argue that this is irrelevant. We would only need to demonstrate one belief that is useful for fitness but that is factually untrue in order to prove the first premise. The belief that bad spirits cause disease is untrue but useful as it compels you to avoid dead bodies which actually may cause disease. We have now (in an admittedly loose sense of the word) proven the first premise.
Unless we assume 3 implies a Pascal's demon scenario where all attempts to identify any set of beliefs and evaluate their truth value or usefulness for fitness will only provide illusory insights I believe that 3 does not contradict 1.
I would argue that this is irrelevant. We would only need to demonstrate one belief that is useful for fitness but that is factually untrue in order to prove the first premise. The belief that bad spirits cause disease is untrue but useful as it compels you to avoid dead bodies which actually may cause disease. We have now (in an admittedly loose sense of the word) proven the first premise.
Unless we assume 3 implies a Pascal's demon scenario where all attempts to identify any set of beliefs and evaluate their truth value or usefulness for fitness will only provide illusory insights I believe that 3 does not contradict 1.
Did you write this with your “knows things” brain, or your “fitness maxing” brain? ;)
I think, from talking to you, that we both believe in a “knows things” brain: we can know things, including that there are limitations to our knowledge, and certain vulnerabilities that can lead us to know wrongly. The kinds of arguments you have been making are predicated on our thoughts having truth value, as such.
The “fitness machine” brain’s thoughts have no truth value, only fitness value. It does not experience “reality as it exists, at all.” As such it cannot even determine that there is truth value to its self-evaluation as a “fitness machine.” Even worse, it has no access to its objective function as a fitness maximizer, so it can’t even tell if its own thoughts are, indeed, fitness-enhancing! So yeah, fitness brain is susceptible to the “demon” scenario you described.
If we want to posit that the “fitness machine” brain can indeed “know things,” we need to specify how. This doesn’t seem obvious at all to me.
I think, from talking to you, that we both believe in a “knows things” brain: we can know things, including that there are limitations to our knowledge, and certain vulnerabilities that can lead us to know wrongly. The kinds of arguments you have been making are predicated on our thoughts having truth value, as such.
The “fitness machine” brain’s thoughts have no truth value, only fitness value. It does not experience “reality as it exists, at all.” As such it cannot even determine that there is truth value to its self-evaluation as a “fitness machine.” Even worse, it has no access to its objective function as a fitness maximizer, so it can’t even tell if its own thoughts are, indeed, fitness-enhancing! So yeah, fitness brain is susceptible to the “demon” scenario you described.
If we want to posit that the “fitness machine” brain can indeed “know things,” we need to specify how. This doesn’t seem obvious at all to me.
Peter Watts is an author of fiction and "Blindsight" is a scifi novel, so you better not take it as true!
That said, it's consistent that both the statement could be deceptive (in the sense it describes) but still assert something that could generally be true. It could even be deceptive in tone or some details, but the gist of what it says still be true.
That said, it's consistent that both the statement could be deceptive (in the sense it describes) but still assert something that could generally be true. It could even be deceptive in tone or some details, but the gist of what it says still be true.
Yeah it is not about truth but about cost and the cost can sometimes be catastrophic (death). For example it is better to mistake a rock for a bear than a bear for a rock.
Buddhist philosophers would agree with you. See Yogacara and Madhyamaka schools, who both agree that the ordinary appearance of the world is akin to a magical illusion.
This seems illogical.
This boils everything down to brains, and discards the human individual, and how different human individuals react differently to the same things depending on their pre-existing understandings of their worlds.
So this statement cannot be true.
This boils everything down to brains, and discards the human individual, and how different human individuals react differently to the same things depending on their pre-existing understandings of their worlds.
So this statement cannot be true.
> This boils everything down to brains, and discards the human individual
The individual is the brain-in-a-body? There isn't a claim that all brains are identical.
The individual is the brain-in-a-body? There isn't a claim that all brains are identical.
Can you expand more on your distinction between talking about brains being bad and talking about individuals being good? Are you saying all brains are the same, and something else makes us different?
Is that a true statement? Has Peter Watts transcended his humanity and entered the ethereal realm of gods where he is no longer subject to that which he claims humans are subject?
Suppose you have at an old analog clock with a second hand that ticks. Look somewhere else in the room, then look at the clock. The first tick as you're looking at it takes much longer to occur, or at least that's the way it will seem to you.
The human eye doesn't move continuously, but moves in short rapid jumps. [0]. Your eye isn't perfect, and during those rapid jumps it produces blurry images. Rather than feeding those blurry images to the image recognition centers of the brain, those images are thrown out. That prevents your brain from using known-faulty data, but now you have a problem because motion tracking requires continuity. So you cheat. You take some of the images that arrived after the blurry images, extrapolate backwards to make an estimation of how things looked before, and then feed those to the motion tracking [1].
The neat thing is that this depends on how the second hand of your analog clock moves. If it ticks every second, then the brain's back-extrapolation to fill in the missing images sees a stationary second hand, and assumes that it was stationary in the past. If it rotates smoothly instead of ticking, then the brain's back-extrapolation sees the motion, assumes that the motion was constant during the past, and back-extrapolates accordingly. So if you glance over to a ticking clock, the first tick feels like it takes longer, but you don't have the same effect glancing over to a smooth motion.
[0] https://en.wikipedia.org/wiki/Saccade [1] https://en.wikipedia.org/wiki/Saccadic_masking
The human eye doesn't move continuously, but moves in short rapid jumps. [0]. Your eye isn't perfect, and during those rapid jumps it produces blurry images. Rather than feeding those blurry images to the image recognition centers of the brain, those images are thrown out. That prevents your brain from using known-faulty data, but now you have a problem because motion tracking requires continuity. So you cheat. You take some of the images that arrived after the blurry images, extrapolate backwards to make an estimation of how things looked before, and then feed those to the motion tracking [1].
The neat thing is that this depends on how the second hand of your analog clock moves. If it ticks every second, then the brain's back-extrapolation to fill in the missing images sees a stationary second hand, and assumes that it was stationary in the past. If it rotates smoothly instead of ticking, then the brain's back-extrapolation sees the motion, assumes that the motion was constant during the past, and back-extrapolates accordingly. So if you glance over to a ticking clock, the first tick feels like it takes longer, but you don't have the same effect glancing over to a smooth motion.
[0] https://en.wikipedia.org/wiki/Saccade [1] https://en.wikipedia.org/wiki/Saccadic_masking
Aside of what the other commenters replied (in support of the assertion), please bear in mind "Blindsight" is a scifi novel. Peter Watts is not a scientist, he's a writer of fiction. Whether he believes what he wrote in his book is uncertain, but remember this is primarily a work of fiction.
He's a marine biologist, but not a neurologist.
It's a work of fiction but it is one that folks in graduate neurophysiology courses have been encouraged to read.
and it has a bibliography at the end
It's a work of fiction but it is one that folks in graduate neurophysiology courses have been encouraged to read.
and it has a bibliography at the end
Yes. Pretty much. For example your conscious awareness processes all the time just 2% of the image your eyes actually see.
And how would you solve the compression problem of dealing with all that data? Our brains are running a similar algorithm to mpeg-4 where we discard redundant data and make some basic predictions about movement. If they didn’t we’d waste a lot of time paying attention to unimportant things in our visual field.
Those are not necessarily unimportant things. These are just things we don't have a capacity to notice because of crap hardware we have to process this information. We don't conciously process 2% of what we see because only 2% is relevant. We do that because hardware needed to process more wouldn't pass through birth canal.
The 2% off visual data that we process is essentially just the stuff that is changing. It’s not some secret hidden knowledge, we can still intentionally examine things closely and see all there is to see.
How many dots are in this image?: https://pbs.twimg.com/media/CsG6MSpWAAAdTob?format=jpg&name=...
After you give up counting them, here is an explanation: https://www.sciencealert.com/here-s-why-you-can-t-see-all-12...
After you give up counting them, here is an explanation: https://www.sciencealert.com/here-s-why-you-can-t-see-all-12...
tl;dr- Humans may consider their own intelligence as a reference-model from which to conceptualize AI. Biases in self-conception may undermine that reference-model, thus undermining conceptions of AI.
Earlier version: https://www.naverlabs.com/en/storyDetail/212
Earlier version: https://www.naverlabs.com/en/storyDetail/212
[deleted]
The article makes some acceptable points, but this article seems disorganized and there seems to be a disconnect between the content and the conclusion. The author devotes most of the article to supporting the idea that anthropomorphizing "AI" is a category mistake. But at the end of the article, he claims that this category mistake needs to be taken into account by producers of AI?