Machine Learning Confronts the Elephant in the Room(quantamagazine.org)
quantamagazine.org
Machine Learning Confronts the Elephant in the Room
https://www.quantamagazine.org/machine-learning-confronts-the-elephant-in-the-room-20180920/
115 comments
Yes but children can pass these tests when presented with a single photo as well (if I understand the setup correctly), so I'd say it's a fair comparison. The algorithm can "look" at the picture as long as it wants as well, e.g. using many different convolutional filters that can "see" features in different parts of the image and then combine these in later stages of the analysis.
How close are those convolutional filters to the physical sight of animals? Were they designed specifically to mimic nature or they are artificial concept with similar purpose?
They may be close to first stages of sight pipeline. Maybe even close to full pipe for some simple animals. For humans, they're pretty much input processing stage. Human vision (and presumably animal vision too) has runtime feedback that uses the the output of the vision process to fix up the input. That's what makes you suddenly see things if someone tells you what to look for, or suddenly unsee things. That's why you didn't notice I repeated the the word "the" in few places in this comment.
As the intro to the the article says (you now noticed, didn't you?), the way DNN vision systems are different is that "unlike humans, they can’t do a double take". And those "double takes" is what our vision pipeline does all the time.
As the intro to the the article says (you now noticed, didn't you?), the way DNN vision systems are different is that "unlike humans, they can’t do a double take". And those "double takes" is what our vision pipeline does all the time.
Very clever young man, but it's turtles all the way down!
Actually the artificial neural networks used for vision tasks are not so different from our visual cortex, there's an interesting Nature paper on this:
https://www.nature.com/articles/s42003-018-0110-y
Of course the neurons in our brain work in a very different way, but at an architecture level the strategies of natural and artificial neural networks seem to match.
https://www.nature.com/articles/s42003-018-0110-y
Of course the neurons in our brain work in a very different way, but at an architecture level the strategies of natural and artificial neural networks seem to match.
Thank you for the link! Indeed, that's very interesting reading.
First they were designed to mimic nature, then these patterns (gabor filters) astonishingly automatically developed themselves via backpropagation (which gave rise to the recent enthusiasm).
Modern networks are instantiated with random noise but end up pretty much the same as previously hardcoded filters.
Modern networks are instantiated with random noise but end up pretty much the same as previously hardcoded filters.
[deleted]
This is what the article says but it's not really correct. Modern CV systems perform "segmenting" before identification.
Yes, check out "attention" mechanisms. The network can use wide convolutions for the image as a whole, effectively mixing data kind of like a blur function, and also use finer convolutions for a small piece of the image that it chooses to focus on.
Does the problem of elephants still exist with such approach? It doesn't seem right that an unexpected feature in a corner of the image would have a dramatic impact on the recognition of the features elsewhere.
I don't see any evidence of "dramatic" impact. The objects visible in both images only show minor changes in the confidence score . The objects missing in one image are close to the visualization threshold (50% confidence) in the other. And for the "chair turning into a couch", we don't see the likelihood of the chair, because it's hidden behind the box for the couch.
So the dramatic changes seen in the image seem to be artifacts of the visualization and not of the network's performance.
So the dramatic changes seen in the image seem to be artifacts of the visualization and not of the network's performance.
Well if there's a thing/elephant/chair floating over someone's head in one part of the room, that does change the context for interpreting other things in the room.
I know very little about machine vision, so forgive the naïvete of this question:
> an ability humans have that AI lacks: the ability to understand when a scene is confusing and thus go back for a second glance.
Wouldn't the machine return a low confidence score when a scene is confusing? If not, why is this difficult to get around?
If so, why can't we just call this a computation difficulty problem, simply requiring a way to go back and spend more effort later when required? A problem like this would simply require better computers, and patience.
In other words, Is this problem as deeply rooted as the article suggests? Or is it simply a problem with the popular approach to machine vision?
> an ability humans have that AI lacks: the ability to understand when a scene is confusing and thus go back for a second glance.
Wouldn't the machine return a low confidence score when a scene is confusing? If not, why is this difficult to get around?
If so, why can't we just call this a computation difficulty problem, simply requiring a way to go back and spend more effort later when required? A problem like this would simply require better computers, and patience.
In other words, Is this problem as deeply rooted as the article suggests? Or is it simply a problem with the popular approach to machine vision?
> simply requiring a way to go back and spend more effort later when required
Present neural nets have no "more effort" or "less effort" knob. For the same input they always produce the same output.
The article says a different thing: humans rerun the "algorithm" again, but this time with the knowledge that something was wrong in the previous run (lets say color), and this knowledge will tweak the way it runs again, maybe even running a completely different algorithm the second time (a network more specialized in color but which is worse at shapes)
Present neural nets have no "more effort" or "less effort" knob. For the same input they always produce the same output.
The article says a different thing: humans rerun the "algorithm" again, but this time with the knowledge that something was wrong in the previous run (lets say color), and this knowledge will tweak the way it runs again, maybe even running a completely different algorithm the second time (a network more specialized in color but which is worse at shapes)
You can easily add a "knob" like this to any model. For example, you can first run an input through a fast, but less accurate object detector (e.g. YOLO), and if not satisfied with results (or automatically triggered by low confidence scores), run the same input through a slower, but more accurate model (RCNN).
You can also have a "generalist" model, trained to differentiate major object classes (e.g. furniture from animals), and multiple "specialist" models, each trained to differentiate between variants of the same object category (e.g. breeds of dogs). That approach was described in Hinton's knowledge distillation paper.
Naively, doing an ensemble of different networks is effectively this. I don't know if folks doing that, though. Pretty sure these edge cases don't matter nearly as much as lack of data for most of us.
Consider, can you recognize that distant relative you have never seen? Why not?
Consider, can you recognize that distant relative you have never seen? Why not?
> a low confidence score
Neural nets should return a low confidence score. But, the popular approach (described below) ignores that. Neural nets ignore confidence because of a technique called softmax [1].
This happens as the final operation of a neural net, and is required for training.
Softmax is a tool to make an array of positive numbers look like a probability distribution:
But then in inference/prediction, the confidence is ignored. What if x.sum() is small? That would imply that the network is uncertain.
[1]: https://en.m.wikipedia.org/wiki/Softmax_function
Neural nets should return a low confidence score. But, the popular approach (described below) ignores that. Neural nets ignore confidence because of a technique called softmax [1].
This happens as the final operation of a neural net, and is required for training.
Softmax is a tool to make an array of positive numbers look like a probability distribution:
out = x / x.sum()
But then in inference/prediction, the confidence is ignored. What if x.sum() is small? That would imply that the network is uncertain.
[1]: https://en.m.wikipedia.org/wiki/Softmax_function
No. Regardless if the outputs for cat/dog are [0.03, 0.01] or [0.75, 0.25], the network is still three times more confident it's a cat. The uncertainty (entropy) of the outputs is exactly the same in both cases.
In other words, if you only have two object classes, the magnitude of the outputs does not matter, the uncertainty is measured by the relative difference of the outputs.
The only way to measure the confidence of the model that the output is "cat OR dog", is to have another class (e.g. "chair"), only then, looking at all three outputs you can estimate the confidence of the model regarding "cat OR dog" predictions (vs 'NOT (cat OR dog)"). For example, if [cat, dog, chair] outputs are [0.03, 0.01, 0.05] then we know the model is not confident that it's either a cat or a dog, but if the outputs are [0.75, 0.25, 0.05], then it's clear it is.
In other words, if you only have two object classes, the magnitude of the outputs does not matter, the uncertainty is measured by the relative difference of the outputs.
The only way to measure the confidence of the model that the output is "cat OR dog", is to have another class (e.g. "chair"), only then, looking at all three outputs you can estimate the confidence of the model regarding "cat OR dog" predictions (vs 'NOT (cat OR dog)"). For example, if [cat, dog, chair] outputs are [0.03, 0.01, 0.05] then we know the model is not confident that it's either a cat or a dog, but if the outputs are [0.75, 0.25, 0.05], then it's clear it is.
Is this what softmax is? Simply dividing a vector by sum of its components? If so, then how does it deserve a name, not to mention a long Wikipedia page full of formulas?
Softmax has two components:
1. Transform the components to e^x. This allows the neural network to work with logarithmic probabilities, instead of ordinary probabilities. This turns the common operation of multiplying probabilities into addition, which is far more natural for the linear algebra based structure of neural networks.
2. Normalize their sum to 1, since that's the total probability we need.
One important consequence of this is that bayes' theorem is very natural to such a network, since it's just multiplication of probabilities normalized by the denominator.
The trivial case of a single layer network with softmax activation is equivalent to logistic regression.
The special case of two component softmax is equivalent to sigmoid activation, which is thus popular when there are only two classes. In multi class classification softmax is used if the classes are mutually exclusive and component-wise sigmoid is used if they are independent.
1. Transform the components to e^x. This allows the neural network to work with logarithmic probabilities, instead of ordinary probabilities. This turns the common operation of multiplying probabilities into addition, which is far more natural for the linear algebra based structure of neural networks.
2. Normalize their sum to 1, since that's the total probability we need.
One important consequence of this is that bayes' theorem is very natural to such a network, since it's just multiplication of probabilities normalized by the denominator.
The trivial case of a single layer network with softmax activation is equivalent to logistic regression.
The special case of two component softmax is equivalent to sigmoid activation, which is thus popular when there are only two classes. In multi class classification softmax is used if the classes are mutually exclusive and component-wise sigmoid is used if they are independent.
Thanks for the detailed explanation!
It also includes the exponentiation step before the vector normalizations. There are connections to statistical mechanics here, where the relative energy population numbers are proportional to the softmax of the energy levels divided by temperature. (so as temperature goes up, the relative energy differences get smaller and the states are more equally populated.) That idea has been ported over as "softmax temperature" in some places.
No, it's not. It's actually e ^ x_i / sum(e ^ x_j for x_j in x), which is in fact different. Simply dividing by the sum wouldn't work for "squashing to a probability distribution" in a large number of cases.
So pointwise exponentiation composed with dividing by the sum. Still don't need a new word.
Coming from pure math, I often feel this way now learning statistics and ML. In pure math, it feels like the threshold for how a novel a concept should be before it gets its own word is much higher.
E.g, we have "regression" and "classification" instead of "supervised continuous prediction" and "supervised discrete prediction".
E.g, we have "regression" and "classification" instead of "supervised continuous prediction" and "supervised discrete prediction".
If you don't undestand where the name "softmax" came from, you don't really understand what it is. Softmax is a differentiable approximation of the max function.
Plot max(0, x) and softmax(0, x) functions, and it should become clear.
Plot max(0, x) and softmax(0, x) functions, and it should become clear.
Nit: it seems it's more like a smooth approximation to maxarg than max.
Yeah it makes sense that this is a super important function, but I still feel like one could just remember the principle that "exponentiation followed by normalization is a smooth approximation to maxarg."
Yeah it makes sense that this is a super important function, but I still feel like one could just remember the principle that "exponentiation followed by normalization is a smooth approximation to maxarg."
Basic building blocks of most deep learning models are convolutional layer, pooling layer, fully connected layer, and softmax layer. How do you propose we call "softmax layer" instead?
Normalization layer?
This opens up possibility of using something else than softmax in there.
This opens up possibility of using something else than softmax in there.
Well, there are other building blocks, such as batch normalization layer, or local contrast normalization layer (not to mention a dozen of batchnorm alternatives, e.g. group normalization, weight normalization, layer normalization, instance normalization, etc).
If you just say "normalization layer" how am I supposed to know which normalization you're talking about?
If you just say "normalization layer" how am I supposed to know which normalization you're talking about?
Because grad students need to publish papers.
Note that you can get a form of confidence by just not applying softmax to the output during inference. Softmax is primarily to aid in training.
[deleted]
How well do neural networks train with no normalization at all, compared with softmax?
It's not about normalization, it's about loss function. Softmax is required by cross entropy minimization (negative log-likelihood to be precise), which works somewhat better in practice than mean squared error (MSE) minimization (which needs no normalization of outputs).
You need to perform some kind of normalization, since probability must be between 0 and 1 (and being wrong on a confident prediction gives huge penalties using the popular maximum likelyhood loss functions).
But you can use component wise normalization (sigmoid) instead of combined normalization (softmax). These correspond to the assumption that the classes are independent (component wise sigmoid) or mutually exclusive (softmax).
But you can use component wise normalization (sigmoid) instead of combined normalization (softmax). These correspond to the assumption that the classes are independent (component wise sigmoid) or mutually exclusive (softmax).
"probability must be between 0 and 1" - why? (I get it's used in mathematics, but I see no reason why a NN would have to output probability that way.)
"and being wrong on a confident prediction gives huge penalties using the popular maximum likelyhood loss functions" - It should.
"and being wrong on a confident prediction gives huge penalties using the popular maximum likelyhood loss functions" - It should.
I see no reason why a NN would have to output probability that way
For classification tasks, the labels are usually encoded as a one hot vector (one in the position of the correct class output, zeros everywhere else). If you don't normalize outputs to be between zero and one, it becomes a regression task - you are essentially asking the model to fit your one hot encoded label. That's not desirable, because we don't care about the actual value of the output for the correct class. Whether it is 0.1, 1.1 or 1001 it is the correct output as long as it's larger than outputs for other classes. That's why we want to take the largest output, and scale it in a way that it's always less than one. Its distance from one depends on how much larger it is than other outputs (the confidence of the model in this prediction).
Without normalization, the model that outputs 1000 for the correct class and tiny values for all other classes would get severely penalized because the labels says it should be 1 in that position (so the error is 1000-1=999), even though the model made the correct prediction.
There's some confusion about this (e.g. https://news.ycombinator.com/item?id=18054447 ), so hopefully my explanation makes sense.
For classification tasks, the labels are usually encoded as a one hot vector (one in the position of the correct class output, zeros everywhere else). If you don't normalize outputs to be between zero and one, it becomes a regression task - you are essentially asking the model to fit your one hot encoded label. That's not desirable, because we don't care about the actual value of the output for the correct class. Whether it is 0.1, 1.1 or 1001 it is the correct output as long as it's larger than outputs for other classes. That's why we want to take the largest output, and scale it in a way that it's always less than one. Its distance from one depends on how much larger it is than other outputs (the confidence of the model in this prediction).
Without normalization, the model that outputs 1000 for the correct class and tiny values for all other classes would get severely penalized because the labels says it should be 1 in that position (so the error is 1000-1=999), even though the model made the correct prediction.
There's some confusion about this (e.g. https://news.ycombinator.com/item?id=18054447 ), so hopefully my explanation makes sense.
Each is harder:
* return a result
* return a list of results
* return a list of results with a confidence weightins
* return a list of results with multiple weightings (confidence and complexity)
how many if statements would you need to write for each if this was applied to a 3x3 grid like tic-tac-toe? how would you know you are correct?
* return a result
* return a list of results
* return a list of results with a confidence weightins
* return a list of results with multiple weightings (confidence and complexity)
how many if statements would you need to write for each if this was applied to a 3x3 grid like tic-tac-toe? how would you know you are correct?
I think the article is referring to a scene that would be unlikely, not make sense given the context. Like identify a grey mass in the middle of the street as en elephant.
"Confusing" would be a poor choice of word here though.
"Confusing" would be a poor choice of word here though.
Well - you want to believe the best method you have, if the best method produces a low confidence approach then that's that really - you can't "believe" the next best classifier instead - because by definition it's likely to be wrong. Humans re-appraise the scene in light of their surprise or confusion.
We need a cognitive model for vision... This is all leading back to David Marr, who wuda thunk.
We need a cognitive model for vision... This is all leading back to David Marr, who wuda thunk.
First, the chair wasn't replaced with a couch - you can see there is yet another rectangle just a few pixels to the left, that likely says "chair".
Second, even many people are surprisingly bad at decoding incongruous scenes, which is why hidden object games are a thing.
Second, even many people are surprisingly bad at decoding incongruous scenes, which is why hidden object games are a thing.
The disappearing cup and book are caused by a similar issue. They "only show detection results with confidence value that exceeds 0.5". The cup is shown at 50% in the original image, so it "disappears" if the presence of the elephant lowers its score even slightly. Same of disappearing books.
Presumably the couch showed up because it rose from <0.5 to 0.57. They don't tell us what the score is for the chair, it might even be higher than the score for the couch, since their visualization clearly doesn't sort by probability.
Presumably the couch showed up because it rose from <0.5 to 0.57. They don't tell us what the score is for the chair, it might even be higher than the score for the couch, since their visualization clearly doesn't sort by probability.
Agreed. Also, what exact architecture were they using, how were they segmenting, etc. This feels like a rehashing of known challenges using a random implementation struggle to make some broad "AI" judgement.
Why can't you just train the network on artificially altered photos, with "elephants" randomly scattered around, until it is robust to them?
Because there's always another type of 'elephant' to be thrown in. The networks are just not inherently robust against unexpected features (I'll call them outliers for want of a better term), and this is what needs to be fixed, and not increasing the variety and frequency of outliers.
>Because there's always another type of 'elephant' to be thrown in.
Well, you could randomly generate and render 3D objects, and paste them into photos. This will give you a huge space of 'elephants'.
Well, you could randomly generate and render 3D objects, and paste them into photos. This will give you a huge space of 'elephants'.
If what you propose was possible to do, if it was possible to overcome the problem described in the above article by using 3d models of real-world objects, then it would also be possible to train on those models in the first place, i.e. we wouldn't need to train machine vision algorithms like CNNs on large datasets of digital stills or photographs- we'd just generate millions of 3d models of objects and scenes of interest, already fully annotated for supervised learning, and train on those. But, as you can probably tell, this is not what happens and instead we have to make do with "real" images, painstakingly collected and annotated with the classes of objets within them "by hand".
What you propose is, essentially, training on a simulation. That doesn't work very well because statistical machine learning algorithms generalise very poorly from training data to unseen data and when your training data is a simulation (such as a 3d model of an object) and the unseen data is the real world (such as a video still, or a digitial photograph of that object) the resulting model will be very bad at predicting the unseen data, so identifying objects in images of the real world.
It might seem to you (and to me!) that modern 3d simulations (as in high-end CGI and video games) is a very faithful simulation of the real world, but that's primarily because we look at modern 3d through human eyes and perceive it with our (presumably) human brains. We don't know how that works exactly, so we can't reproduce the process in computers, yet, therefore machine vision systems essentially "see" and "perceive" something completely different than what we do. In the end, what looks like a very close approximation to you, is pretty much useless for them- or in any case, they are not able to learn to generalise from that to real objects.
What you propose is, essentially, training on a simulation. That doesn't work very well because statistical machine learning algorithms generalise very poorly from training data to unseen data and when your training data is a simulation (such as a 3d model of an object) and the unseen data is the real world (such as a video still, or a digitial photograph of that object) the resulting model will be very bad at predicting the unseen data, so identifying objects in images of the real world.
It might seem to you (and to me!) that modern 3d simulations (as in high-end CGI and video games) is a very faithful simulation of the real world, but that's primarily because we look at modern 3d through human eyes and perceive it with our (presumably) human brains. We don't know how that works exactly, so we can't reproduce the process in computers, yet, therefore machine vision systems essentially "see" and "perceive" something completely different than what we do. In the end, what looks like a very close approximation to you, is pretty much useless for them- or in any case, they are not able to learn to generalise from that to real objects.
>It might seem to you (and to me!) that modern 3d simulations (as in high-end CGI and video games) is a very faithful simulation of the real world, but that's primarily because we look at modern 3d through human eyes and perceive it with our (presumably) human brains. We don't know how that works exactly, so we can't reproduce the process in computers, yet, therefore machine vision systems essentially "see" and "perceive" something completely different than what we do. In the end, what looks like a very close approximation to you, is pretty much useless for them- or in any case, they are not able to learn to generalise from that to real objects.
This really is not what I expected, being a layman.
Do you have any more data on this topic of training on a simulation?
This really is not what I expected, being a layman.
Do you have any more data on this topic of training on a simulation?
"Data", not so much, but you can explore the subject by searching for "reality gap in machine learning". To be honest there are not many references to it that I could find online, but this post by the Google AI blog starts with a very good summary (then proceeds to propose one way to overcome the difficulties of training on simulations, though of course the problem is still far from solved):
https://ai.googleblog.com/2017/10/closing-simulation-to-real...
Simulating many years of robotic interaction is quite feasible with modern parallel computing, physics simulation, and rendering technology. Moreover, the resulting data comes with automatically-generated annotations, which is particularly important for tasks where success is hard to infer automatically. The challenge with simulated training is that even the best available simulators do not perfectly capture reality. Models trained purely on synthetic data fail to generalize to the real world, as there is a discrepancy between simulated and real environments, in terms of both visual and physical properties. In fact, the more we increase the fidelity of our simulations, the more effort we have to expend in order to build them, both in terms of implementing complex physical phenomena and in terms of creating the content (e.g., objects, backgrounds) to populate these simulations. This difficulty is compounded by the fact that powerful optimization methods based on deep learning are exceptionally proficient at exploiting simulator flaws: the more powerful the machine learning algorithm, the more likely it is to discover how to "cheat" the simulator to succeed in ways that are infeasible in the real world. The question then becomes: how can a robot utilize simulation to enable it to perform useful tasks in the real world?
The difficulty of transferring simulated experience into the real world is often called the "reality gap." The reality gap is a subtle but important discrepancy between reality and simulation that prevents simulated robotic experience from directly enabling effective real-world performance. Visual perception often constitutes the widest part of the reality gap: while simulated images continue to improve in fidelity, the peculiar and pathological regularities of synthetic pictures, and the wide, unpredictable diversity of real-world images, makes bridging the reality gap particularly difficult when the robot must use vision to perceive the world, as is the case for example in many manipulation tasks.
Note the bit about deep learning algorithms being very proficient "cheaters", which I missed in my comment above. Indeed, one way to fail to generalise from a simulation to the real world is to "overfit" to the defects in the simulation!
Like the linked blog post, most material you are likely to find online focus on training robots with (deep) Reinforcement Learning, I think because that just happens to be one domain where it is even harder to collect training data than good old supervised learning for image recognition. I can find virtually no source referring to the "reality gap" in the context of purely machine vision research- it's just not the done thing to train vision algorithms on simulated data, for the reasons described above, consequently it's very difficult to find hard data on why it's not done.
A good related source (with a bazillion references) is the following blog post, discussing the difficulties of deep RL, which is primarily trained on simulated environments:
https://www.alexirpan.com/2018/02/14/rl-hard.html
The post makes only passing referenct to the "reality gap" but it should give a good idea about the ins and outs of training in simulated environments.
https://ai.googleblog.com/2017/10/closing-simulation-to-real...
Simulating many years of robotic interaction is quite feasible with modern parallel computing, physics simulation, and rendering technology. Moreover, the resulting data comes with automatically-generated annotations, which is particularly important for tasks where success is hard to infer automatically. The challenge with simulated training is that even the best available simulators do not perfectly capture reality. Models trained purely on synthetic data fail to generalize to the real world, as there is a discrepancy between simulated and real environments, in terms of both visual and physical properties. In fact, the more we increase the fidelity of our simulations, the more effort we have to expend in order to build them, both in terms of implementing complex physical phenomena and in terms of creating the content (e.g., objects, backgrounds) to populate these simulations. This difficulty is compounded by the fact that powerful optimization methods based on deep learning are exceptionally proficient at exploiting simulator flaws: the more powerful the machine learning algorithm, the more likely it is to discover how to "cheat" the simulator to succeed in ways that are infeasible in the real world. The question then becomes: how can a robot utilize simulation to enable it to perform useful tasks in the real world?
The difficulty of transferring simulated experience into the real world is often called the "reality gap." The reality gap is a subtle but important discrepancy between reality and simulation that prevents simulated robotic experience from directly enabling effective real-world performance. Visual perception often constitutes the widest part of the reality gap: while simulated images continue to improve in fidelity, the peculiar and pathological regularities of synthetic pictures, and the wide, unpredictable diversity of real-world images, makes bridging the reality gap particularly difficult when the robot must use vision to perceive the world, as is the case for example in many manipulation tasks.
Note the bit about deep learning algorithms being very proficient "cheaters", which I missed in my comment above. Indeed, one way to fail to generalise from a simulation to the real world is to "overfit" to the defects in the simulation!
Like the linked blog post, most material you are likely to find online focus on training robots with (deep) Reinforcement Learning, I think because that just happens to be one domain where it is even harder to collect training data than good old supervised learning for image recognition. I can find virtually no source referring to the "reality gap" in the context of purely machine vision research- it's just not the done thing to train vision algorithms on simulated data, for the reasons described above, consequently it's very difficult to find hard data on why it's not done.
A good related source (with a bazillion references) is the following blog post, discussing the difficulties of deep RL, which is primarily trained on simulated environments:
https://www.alexirpan.com/2018/02/14/rl-hard.html
The post makes only passing referenct to the "reality gap" but it should give a good idea about the ins and outs of training in simulated environments.
Oh, and, regarding the issue of discrepancies in perception between humans and machine vision systems, this is a good source, albeit on the short side (well, the bit that talks about perception anyway):
https://rodneybrooks.com/forai-steps-toward-super-intelligen...
https://rodneybrooks.com/forai-steps-toward-super-intelligen...
That would not seem computationally efficient if for every class (i.e. object) you were to create a 3D modeling and paste it into a training set
Couldn't you train multiple nets with somewhat different training sets, and for each classification see how many agrees?
If they all agree you have a high confidence, if they mostly disagree you have a low confidence.
If they all agree you have a high confidence, if they mostly disagree you have a low confidence.
That’s actually the concept of ensemble learning! [1] The question is, how would that lead to predicting a randomly placed elephant as an elephant with high confidence?
[1] https://en.m.wikipedia.org/wiki/Ensemble_averaging_(machine_...
[1] https://en.m.wikipedia.org/wiki/Ensemble_averaging_(machine_...
This study shows something more fundamental about the way that neural networks process images, other than just that random elephants are a problem. Rather than break the image down into constituent parts and reason about them individually, they use global context to help them classify. This lets them outperform humans on some tasks. The problem is if you show it something completely different to anything it has ever seen before (like a T-rex with a cane and monocle riding a bear), it will not reliably output something sensible.
To an extent it shows the gap between neural networks being able to classify things, and actually being able to understand what they are seeing, which will always limit their utility beyond classifying images that look like other images they've seen before.
To an extent it shows the gap between neural networks being able to classify things, and actually being able to understand what they are seeing, which will always limit their utility beyond classifying images that look like other images they've seen before.
I just spent 15 minutes putting a picture of a bear with a top hat riding Abraham Lincoln with laser eyes into all the online demos I could find. They weren't unintelligible responses, just a bit crap. They said "weapon", "animal", "person", etc. I'm not sure how much global context they really use, in my experience it has been latching onto a weird bit of local texture. More work to do...
Try this one : https://i.ytimg.com/vi/I11Vlcpz3jc/hqdefault.jpg
I tried it on a couple of online demos, neither spotted that there was a car in the picture.
I tried it on a couple of online demos, neither spotted that there was a car in the picture.
Is a car still a car if it's taken out of context? To us, yes. To an AI, it might not matter.
A space collision avoidance AI that identifies a road vehicle as a road vehicle and expects it to behave as one normally behaves would create problems. A human can look at a 1936 Ford truck floating in space and know it's not going to make a sudden left turn. An AI working in space would still treat it as debris if a human told it "that's a truck, it's not going to do anything different unless acted upon."
A space collision avoidance AI that identifies a road vehicle as a road vehicle and expects it to behave as one normally behaves would create problems. A human can look at a 1936 Ford truck floating in space and know it's not going to make a sudden left turn. An AI working in space would still treat it as debris if a human told it "that's a truck, it's not going to do anything different unless acted upon."
>> Is a car still a car if it's taken out of context? To us, yes. To an AI, it might not matter.
A system that can't recognise objects out of context is either completely useless, or only useful to the extent that it can recognise a very large number of objects in a very large number of varied contexts. Essentially, it's a system that learns primarily by memorising specific cases, rather than memorising and generalising to new cases.
This kind of system is limited by the number of objects and contexts with which it can be trained in practice. Which is why you need large datasets to train machine vision algorithms- because you basically have to show them many examples of everything you want them to learn.
In your example, the problem is not that the "AI" would not be able to recognise a car in space as a car, in space; it's that it would not be able to recognise it even as debris - unless it had already seen such "debris" before. Obviously, if it hadn't observed the behaviour of such debris it would also be incapable of predicting its behaviour, also.
The inability of statistical machine learning models to abstract and extrapolate from available observations, in the same way humans do, can be very surprising, especially in light of their othewise excellent abilities to recognise objects they have seen before.
A system that can't recognise objects out of context is either completely useless, or only useful to the extent that it can recognise a very large number of objects in a very large number of varied contexts. Essentially, it's a system that learns primarily by memorising specific cases, rather than memorising and generalising to new cases.
This kind of system is limited by the number of objects and contexts with which it can be trained in practice. Which is why you need large datasets to train machine vision algorithms- because you basically have to show them many examples of everything you want them to learn.
In your example, the problem is not that the "AI" would not be able to recognise a car in space as a car, in space; it's that it would not be able to recognise it even as debris - unless it had already seen such "debris" before. Obviously, if it hadn't observed the behaviour of such debris it would also be incapable of predicting its behaviour, also.
The inability of statistical machine learning models to abstract and extrapolate from available observations, in the same way humans do, can be very surprising, especially in light of their othewise excellent abilities to recognise objects they have seen before.
Sure, but this is a narrow view of the issue, which is that neural nets are solving a somewhat superficial version of the problem. This may not be an issue for simple image classification, but if we're looking to attack more complex tasks (e.g. describing what happens in a movie), it's clear that we need neural nets with deeper understanding.
If the net stops thinking that unicyclists are objects when it rains that could present a challenge for unicyclists.
That is interesting. This is what I find most annoying about neural nets though, they are impossible to debug. Can you work out if they are failing because of global context (car in water) or local texture (weird swirling patterns and odd colors)?
Because that's not the problem. The issue is that networks can't recognize their output doesn't make sense, reconfigure themselves and try again, and repeat that process until the output starts matching expectations (while simultaneously weakening those expectations the longer this continues). The way our vision does.
Are you going to repeat this with every object that exists, or could exist?
What I'd do is train it on a large diverse set of foreign objects, then test it with a different diverse set of foreign objects, to see if it is possible to make it generally robust. Basically, the goal is that any feature which is not overlapping with the "elephant" should not be influenced by it.
Well, CNNs are already trained on "a large diverse set" of objects, in many different configurations.
By the way, something that is not described in the article- the referenced paper also describes experiments where objects where translated inside the source image itself - so not in another image. And the tested networks were still confused.
So it's not "foreign" objects that are the problem. It's that neural nets generalise very poorly. Show them something they haven't seen before and their ability to recognise it falls off a cliff.
By the way, something that is not described in the article- the referenced paper also describes experiments where objects where translated inside the source image itself - so not in another image. And the tested networks were still confused.
So it's not "foreign" objects that are the problem. It's that neural nets generalise very poorly. Show them something they haven't seen before and their ability to recognise it falls off a cliff.
No, you see neural networks are specially afraid of elephants. Just like elephants are especially afraid of mice. And so the circle continues, all the way around to humans who are especially afraid of neural networks and what they will do to society.
I am sorry to inform you that this post is not that funny.
[deleted]
I am not an expert in AI/ML, just a casual observer, but I didn't like the tone of the article. Not only it fixates on the speed of processing, it also seems smug and acts as if we know how human brain sees and processes images, while all we have is a bunch of conjectures. That was distracting.
Well, we do know that the brain does something that CV systems don't, and that it was a part of a recent self-driving car accident.
I'm confused about what's so special here.
This is analogous to a person from the early days of photography having no knowledge of the possibility of image doctoring, and us feeling "smug" because a photo like this would elicit confusion.
It seems to me you just have to run your NN on pairs of doctored and undoctored images, and create a separation of concern between scene continuity and object recognition. It's likely that a lot of implementations currently rely too much on surrounding context for successful categorization.
It's also worth exploring certain image analysis techniques during pre-processing to draw out certain attenuated aspects. Edge detection is par for the course, but things like shading are also important to attenuate so that the NN has an easier time learning.
If we're dealing with multiple frames and not just a single image, this opens up a whole dimension of temporal analysis that should make it trivial to separate the background from juxtaposed images.
This is analogous to a person from the early days of photography having no knowledge of the possibility of image doctoring, and us feeling "smug" because a photo like this would elicit confusion.
It seems to me you just have to run your NN on pairs of doctored and undoctored images, and create a separation of concern between scene continuity and object recognition. It's likely that a lot of implementations currently rely too much on surrounding context for successful categorization.
It's also worth exploring certain image analysis techniques during pre-processing to draw out certain attenuated aspects. Edge detection is par for the course, but things like shading are also important to attenuate so that the NN has an easier time learning.
If we're dealing with multiple frames and not just a single image, this opens up a whole dimension of temporal analysis that should make it trivial to separate the background from juxtaposed images.
>It seems to me you just have to run your NN on pairs of doctored and undoctored images, and create a separation of concern between scene continuity and object recognition.
This may fix this problem, but what other visual issues will we uncover that might be fixable with the right kind of training but which we haven't yet accounted for.
For example, we see so many optical illusions because of how our eyes and brain (and the neurons between them) work. These are often times caused by our sensation or perception incorrectly picking up data, but on purpose (well, as much purpose as an evolved system could have). For example, how neurons in the eye exaggerate differences in brightness or how we are overly sensitive to perceiving faces so that we see them even where they don't exist. What impact does the lack of these have on an artificial vision system? For starters, it would likely indicate the system wouldn't recognize the optical illusion unless trained for it, but are there any other impacts that we need to take into consideration, especially when considering having human lives depending upon the system?
This may fix this problem, but what other visual issues will we uncover that might be fixable with the right kind of training but which we haven't yet accounted for.
For example, we see so many optical illusions because of how our eyes and brain (and the neurons between them) work. These are often times caused by our sensation or perception incorrectly picking up data, but on purpose (well, as much purpose as an evolved system could have). For example, how neurons in the eye exaggerate differences in brightness or how we are overly sensitive to perceiving faces so that we see them even where they don't exist. What impact does the lack of these have on an artificial vision system? For starters, it would likely indicate the system wouldn't recognize the optical illusion unless trained for it, but are there any other impacts that we need to take into consideration, especially when considering having human lives depending upon the system?
Can't they implement a "virtual representation" or sort of ideal or archetypical scene in an AIs memory that's quickly accessible, and then use that to diff "new unexpected stuff" and refocus on that difference somehow?
If we had such a representation and the mapping from real world (images) to it, we'd basically solved all image recognition tasks because the above is the definition of an image recognition task
In the real world you wouldn't need to know the exact shape of an object to determine what it is, unique features could be enough. For an elephant this could be just a range of skin colors that are unique to elephants.
The problem with videos and images vs. the real world is the fidelity, you can't trust images or videos that are previously captured through other equipment. This barrier wouldn't exist if the AI can "see" into the real world, the skin colors of animals can be trusted there!
The problem with videos and images vs. the real world is the fidelity, you can't trust images or videos that are previously captured through other equipment. This barrier wouldn't exist if the AI can "see" into the real world, the skin colors of animals can be trusted there!
This is an important, and worrying, problem, but I'd be more impressed if the elephant wasn't added as a bad photoshop. That kind of manipulation is going to screw up a lot of the processing.
> A visual prank exposes an Achilles’ heel of computer vision systems: Unlike humans, they can’t do a double take.
Thanks for giving us ML researchers a TODO. We'll get to work right away.
Thanks for giving us ML researchers a TODO. We'll get to work right away.
Wouldn't humans who haven't seen Elephants before or who are not trained to recognize the general category of mammals / animals be prone to make similar errors?
I don't think so - I think that the human may fail to find the correct label for the Elephant (although if they have some knowledge about Elephants they might), but the machine fails to see it - to identify it as a thing, and also it changes the labels it gives other things; so cats become dogs, mugs become books and so on. Humans are not prone to this.
>who are not trained to recognize the general category of mammals / animals
Does such a human exist? Other than perhaps someone blind from birth who had their sight restored, I don't think you could fine such a person give how conditioned we are to recognize animals, perhaps even on a biological level. Closest I can think of is animals that are camouflaged, and I think we would see many humans making mistakes on those. I've definitely have, especially with regards to insects that look like plants. But those animals specifically evolved to throw off predator detection systems.
Does such a human exist? Other than perhaps someone blind from birth who had their sight restored, I don't think you could fine such a person give how conditioned we are to recognize animals, perhaps even on a biological level. Closest I can think of is animals that are camouflaged, and I think we would see many humans making mistakes on those. I've definitely have, especially with regards to insects that look like plants. But those animals specifically evolved to throw off predator detection systems.
In this case the object detector has been trained to recognize elephants. It's just never seen elephants in such a setting (living room).
It would be similar to a situation where a human, who sees an improbable object for a given environment, reevaluates his initial perception/understanding of that environment. I can't come up with a good example of such a situation, but you get the idea.
It would be similar to a situation where a human, who sees an improbable object for a given environment, reevaluates his initial perception/understanding of that environment. I can't come up with a good example of such a situation, but you get the idea.
Is this not a whole new attack surface?
Adversarially generated inputs have been known for some time [1], so I wouldn’t call it a new class of attacks unless 1. there are other and older ways of attacking neural networks 2. new as in the last couple of years.
[1] https://blog.openai.com/adversarial-example-research/
[1] https://blog.openai.com/adversarial-example-research/
Yes, this can be used for generating adversarial inputs, and yes, this is a new attack surface. From the paper:
The images generated here could be viewed as a variant of adversarial examples, in which small image perturbations (imperceptible to humans) cause a large shift in the network’s output. The images we generate are of a somewhat opposite flavor: while we do not limit the magnitude of the difference between the original and modified image, the detectors are sometimes “blind” to the inserted object. In addition, our examples are not “targeted” in the sense that no optimization process is required to generate them; they seem prevalent enough so that a simple scan of transplanting translated versions of one object in the other can give rise to multiple wrong interpretations.
The images generated here could be viewed as a variant of adversarial examples, in which small image perturbations (imperceptible to humans) cause a large shift in the network’s output. The images we generate are of a somewhat opposite flavor: while we do not limit the magnitude of the difference between the original and modified image, the detectors are sometimes “blind” to the inserted object. In addition, our examples are not “targeted” in the sense that no optimization process is required to generate them; they seem prevalent enough so that a simple scan of transplanting translated versions of one object in the other can give rise to multiple wrong interpretations.
Not really. In addition to object classes, object detection networks learn to ID teh context of an object's presence. In almost every scenario, this context is a desirable learning trait.
How about a scenario where an image of an elephant ( or any other out of context example ) is projected on purpose with intent to cause damage?
I'm aware of the vision algorithm in my head to a certain extent and I'm not sure if machine vision does the same. You can run simple thought experiments to see what your brain actually does to analyze an image.
First of all when I look at a scene I am 100% aware of geometry. Irregardless of meaning, words and symbols I can trace out the three dimensional shape of things without associations to words.
How do I know I can do this? Simple. Every scene I look at I can basically translate or imagine that scene in my head as wireframe scene or some low poly scene as if it was generated by a computer. Similar to if I look at wireframe scene generated by a computer, my mind can translate it into a scene that looks real. Try it, you can do it.
Second, I can look at an actual low poly wireframe model of an elephant and associate it with the word 'elephant.' I do not need color, or detail to know it's an elephant. In fact, with just color and detail alone it is harder for me to identify an elephant. For example if someone takes many very closeup photographs of parts of an elephant like its eye, skin, ear, etc.. and asks me to guess the subject by interpreting the pictures... I become fully aware that I would be accessing a slower, different part of my brain to deduce the meaning. This is a stark contrast to the instantaneous word association established when I look at a wireframe model of an elephant. The speed difference between both ways of identifying an elephant indicate to me that geometric interpretation is the primary driver behind our visual analysis and details like color or texture are tertiary when it comes to the identification of an elephant. I believe the visual cortex determines shape first, then subsequently determines word from shape.
If you feed a white sculpture of an elephant or a wireframe of an elephant into one of these deep learning networks it is unlikely you will get the word 'elephant' as output. But if you feed it a real picture of an elephant it can correctly identify the elephant (assuming it was trained against photos of an elephant). Because the delta between a white sculpture of an elephant and an actual picture of an elephant is just color and detail this indicates to me that when you train these deep learning networks to recognize an elephant you are training the network to recognize details. It's a form of over fitting, the training is not general enough to catch geometry. It is correlating blobs of pixels , color and detail with an elephant rather then associating a three dimensional model of it to the word... the opposite of what humans do. In fact I bet you that if you took those very closeup photographs of an elephant and fed it into the network it'd do a better job at recognition versus the picture of a white sculpture of an elephant.
This indicates to me that to improve our vision algorithms, the algorithm must first associate pixels with geometry then identify the associated word to the geometry rather than try to associate blobs of pixels to words. Train geometry recognition before word association.
My guess is that our minds have specific and genetically determined built in geometry recognition algorithms honed to turn a 2d image into a 3d shape. We do not learn to translate 2d to 3d we are born with that ability hardwired. Where learning comes in is the translation of this shape to a word. Whereas most of the machine learning we focus on in research is image recognition, I believe the brain is actually learning shape and geometry recognition.
How do I know I can do this? Simple. Every scene I look at I can basically translate or imagine that scene in my head as wireframe scene or some low poly scene as if it was generated by a computer. Similar to if I look at wireframe scene generated by a computer, my mind can translate it into a scene that looks real. Try it, you can do it.
Second, I can look at an actual low poly wireframe model of an elephant and associate it with the word 'elephant.' I do not need color, or detail to know it's an elephant. In fact, with just color and detail alone it is harder for me to identify an elephant. For example if someone takes many very closeup photographs of parts of an elephant like its eye, skin, ear, etc.. and asks me to guess the subject by interpreting the pictures... I become fully aware that I would be accessing a slower, different part of my brain to deduce the meaning. This is a stark contrast to the instantaneous word association established when I look at a wireframe model of an elephant. The speed difference between both ways of identifying an elephant indicate to me that geometric interpretation is the primary driver behind our visual analysis and details like color or texture are tertiary when it comes to the identification of an elephant. I believe the visual cortex determines shape first, then subsequently determines word from shape.
If you feed a white sculpture of an elephant or a wireframe of an elephant into one of these deep learning networks it is unlikely you will get the word 'elephant' as output. But if you feed it a real picture of an elephant it can correctly identify the elephant (assuming it was trained against photos of an elephant). Because the delta between a white sculpture of an elephant and an actual picture of an elephant is just color and detail this indicates to me that when you train these deep learning networks to recognize an elephant you are training the network to recognize details. It's a form of over fitting, the training is not general enough to catch geometry. It is correlating blobs of pixels , color and detail with an elephant rather then associating a three dimensional model of it to the word... the opposite of what humans do. In fact I bet you that if you took those very closeup photographs of an elephant and fed it into the network it'd do a better job at recognition versus the picture of a white sculpture of an elephant.
This indicates to me that to improve our vision algorithms, the algorithm must first associate pixels with geometry then identify the associated word to the geometry rather than try to associate blobs of pixels to words. Train geometry recognition before word association.
My guess is that our minds have specific and genetically determined built in geometry recognition algorithms honed to turn a 2d image into a 3d shape. We do not learn to translate 2d to 3d we are born with that ability hardwired. Where learning comes in is the translation of this shape to a word. Whereas most of the machine learning we focus on in research is image recognition, I believe the brain is actually learning shape and geometry recognition.
Wrong. They can do a second take. You should just code it in.
Is this sarcasm? The point of the second take is that humans often know they're confused and will go back to think about the image and remedy the confusion, whereas having a neural network just look at the image again isn't going to do anything (plus, existing architectures don't seem to have any capacity to say "I'm confused" anyway).
Parent is not sarcastic. Automated recognition systems are systems, which means they are made of several components (sensors, databases, hardwares, softwares) working together. Neural networks are one of those sub-components and no one ever claimed that neural networks are 100% accurate and sufficient to build automated decision systems.
The problem described in the article is taken into account when building systems (like autonomous cars) using neural networks.
The problem described in the article is taken into account when building systems (like autonomous cars) using neural networks.
The problem described in the article is taken into account when building systems (like autonomous cars) using neural networks.
Where can I find more information on this?
Where can I find more information on this?
This is a vast engineering field studied before the existence of neural networks.
Keywords are "control theory", "sensor fusion", "automated decision under uncertainty" that you can look on Google, Wikipedia and arXiv. Also "Simultaneous localization and mapping" which is a good example of using uncertain data points from different sensors to build a representation of the reality.
In those systems, a neural network is just another sensor providing augmented information.
Keywords are "control theory", "sensor fusion", "automated decision under uncertainty" that you can look on Google, Wikipedia and arXiv. Also "Simultaneous localization and mapping" which is a good example of using uncertain data points from different sensors to build a representation of the reality.
In those systems, a neural network is just another sensor providing augmented information.
Thank you, I will have a look. Just out of curiosity, what do you mean by 'uncertain' in this context?
Errors in precision and accuracy.
Hmm... there is no doubt about the sensor input, the elephant is there. The issue is with precision and accuracy of the model itself.
Edit: I need to rephrase, the interesting case for me ( and the one the article is describing) is when the model fails with accurate input data due to assumptions that are intrinsic to the model itself.
Edit: I need to rephrase, the interesting case for me ( and the one the article is describing) is when the model fails with accurate input data due to assumptions that are intrinsic to the model itself.
Optical sensors don't have anywhere near the dynamic range of a retina. I think poor low light performance has been implicated in at least one self-driving crash. Eyes can at least render objects and shapes in extreme low light even if the brain doesn't have enough information to identify them. The tiny sensors in a camera compact enough to be practical still have a long way to go.
There are doubts in both, frankly.
The title is a bit misleading / clickbait. It should be "Neural networks Confronts...". Machine Learning isn't all about neutral networks and deep learning.
As another comment said, "second take" is not what neural networks are made for. Neural networks are a building block of more complex decision systems, where the weakness of the neural networks are taken into account before automatically commiting to decisions.
Otherwise, I guess the article is good at pointing the current limitation of neural networks alone.
As another comment said, "second take" is not what neural networks are made for. Neural networks are a building block of more complex decision systems, where the weakness of the neural networks are taken into account before automatically commiting to decisions.
Otherwise, I guess the article is good at pointing the current limitation of neural networks alone.
> It should be "Neural networks Confronts...".
I disagree, for the following reasons:
- The paper is called The Elephant in the Room. Technically the paper confronted the popular machine learning approaches, so the reversal would have been a more eye catching headline. "The Elephant in the Room confronts Machine Learning" is a rare opportunity to honestly play with word order in a way that would arguably be more attention-grabbing.
- Machine learning is more general than Convolutional Neural Networks, but less general than AI. This seems appropriate for this type of popular science publication.
- There are approaches that use attention and RNNs and they are an interesting approach, but by no means mainstream nor producing state-of-the-art classification accuracies. Actually, second take is exactly what an attention-based RNN can do.
- The last time I checked the state-of-the-art solutions to image classification involve Convolutional Neural Networks.
If there was an approach that was even close to approaching ConvNet performance, I would agree that it is perhaps unfair to condemn the entire field. Even then, we would be in the territory of generalizations in headlines, which are often fair. "{party} proposing legislation" is a probably bigger generalization of headline that would generally pass without criticism.
I disagree, for the following reasons:
- The paper is called The Elephant in the Room. Technically the paper confronted the popular machine learning approaches, so the reversal would have been a more eye catching headline. "The Elephant in the Room confronts Machine Learning" is a rare opportunity to honestly play with word order in a way that would arguably be more attention-grabbing.
- Machine learning is more general than Convolutional Neural Networks, but less general than AI. This seems appropriate for this type of popular science publication.
- There are approaches that use attention and RNNs and they are an interesting approach, but by no means mainstream nor producing state-of-the-art classification accuracies. Actually, second take is exactly what an attention-based RNN can do.
- The last time I checked the state-of-the-art solutions to image classification involve Convolutional Neural Networks.
If there was an approach that was even close to approaching ConvNet performance, I would agree that it is perhaps unfair to condemn the entire field. Even then, we would be in the territory of generalizations in headlines, which are often fair. "{party} proposing legislation" is a probably bigger generalization of headline that would generally pass without criticism.
> - The paper is called The Elephant in the Room.
How is this argument related to the confusion of the terms Machine Learning and Neural Networks...?
> - Machine learning is more general than Convolutional Neural Networks, but less general than AI. This seems appropriate for this type of popular science publication.
The most appropriate term is the most precise one, in this case : neural networks
> - There are approaches that use attention and RNNs and they are an interesting approach, but by no means mainstream nor producing state-of-the-art classification accuracies. Actually, second take is exactly what an attention-based RNN can do.
Correct, but you are still talking about neural networks...
> - The last time I checked the state-of-the-art solutions to image classification involve Convolutional Neural Networks.
And again neural networks.
️
How is this argument related to the confusion of the terms Machine Learning and Neural Networks...?
> - Machine learning is more general than Convolutional Neural Networks, but less general than AI. This seems appropriate for this type of popular science publication.
The most appropriate term is the most precise one, in this case : neural networks
> - There are approaches that use attention and RNNs and they are an interesting approach, but by no means mainstream nor producing state-of-the-art classification accuracies. Actually, second take is exactly what an attention-based RNN can do.
Correct, but you are still talking about neural networks...
> - The last time I checked the state-of-the-art solutions to image classification involve Convolutional Neural Networks.
And again neural networks.
️
> How is this argument related to the confusion of the terms Machine Learning and Neural Networks...?
We're talking about the title.
> The most appropriate term is the most precise one, in this case : neural networks
That is generalizing RNNs and CNNs into "neural networks". If I take a popular CNN architecture, chop off the final dense layer and activation and attach an SVM classifier, is it an SVM or a convnet I am using?
Somehow the title "The Elephant in the Room confronts faster_rcnn_inception_resnet_v2_atrous_coco, faster_rcnn_nas_coco, ssd_mobilenet_v1_coco, mask_rcnn_inception_resnet_v2_atrous_coco, mask_rcnn_resnet101_atrous_coco." (or the reverse) doesn't have the same ring to it and is probably inappropriate for the audience.
We're talking about the title.
> The most appropriate term is the most precise one, in this case : neural networks
That is generalizing RNNs and CNNs into "neural networks". If I take a popular CNN architecture, chop off the final dense layer and activation and attach an SVM classifier, is it an SVM or a convnet I am using?
Somehow the title "The Elephant in the Room confronts faster_rcnn_inception_resnet_v2_atrous_coco, faster_rcnn_nas_coco, ssd_mobilenet_v1_coco, mask_rcnn_inception_resnet_v2_atrous_coco, mask_rcnn_resnet101_atrous_coco." (or the reverse) doesn't have the same ring to it and is probably inappropriate for the audience.
i wouldn't be so critical. It's journalism and I would intrepret the title more as "Machine Learning-community confronts...".
And while ML isn't all about NNs and deep learning, we're quite enticed by them right now and try to push them to see where their limitations are. So I think with a bit more generous interpretation the title is acceptable. I think it's certainly not clickbait, since NNs are right now a very significant part of the ML-research landscape.
And while ML isn't all about NNs and deep learning, we're quite enticed by them right now and try to push them to see where their limitations are. So I think with a bit more generous interpretation the title is acceptable. I think it's certainly not clickbait, since NNs are right now a very significant part of the ML-research landscape.
Well, to me, this is not journalism. Journalists have a strong ethic and respect for the meaning of words to relay high quality information.
This is science vulgarisation, or a magazine article. Not that there is no value in it, but it's an over-generalization.
This is science vulgarisation, or a magazine article. Not that there is no value in it, but it's an over-generalization.
In this article, learn the secret AI researchers dont want you to know, my non tech friends love talking about it! Gonna have to agree with everyone else here, the title is clickbait and you can just code around it.
There is nothing fatal here its just one more problem to solve. Havent we heard all about this issue of image recognition being trickable 1000 times, why is this the top post?
There is nothing fatal here its just one more problem to solve. Havent we heard all about this issue of image recognition being trickable 1000 times, why is this the top post?
What if an algorithm actually had a focused sight? Look at the brightest feature, blur the rest, then try to find another distinctive details, look closer into the parts of the image until the most of it becomes clear. Let other neural network control the scanning process and loop until it develops some confidence in the result. Can it work that way?