Traffic Sign Recognition with TensorFlow(medium.com)
medium.com
Traffic Sign Recognition with TensorFlow
https://medium.com/@waleedka/traffic-sign-recognition-with-tensorflow-629dffc391a6#.ifsyvpm67
27 comments
In the UK at least, there is a lot of resistance to automation from the unions. They are well organised, and able to essentially hold the rail companies to randsom until they back down on automation plans.
First thing to do is sell it as driver-assistance technology. More drivers move to the "just open and close the doors" role (aka the "man and a dog" factory of the future model.)
Then you can let drivers just not be replaced when they retire. If the drivers union kicks up a fuss and they strike, well, great -- ask them to take the next day off too.
Then you can let drivers just not be replaced when they retire. If the drivers union kicks up a fuss and they strike, well, great -- ask them to take the next day off too.
There need be only one train operator per train (or maybe two?), which can carry millions of dollars' worth of payload – which can have dozens of cars. Despite it being a seemingly more simple and discrete problem to solve, it may not be worth it – a human's judgement can come in handy from time-to-time.
You assume it has already happened. Let me remind you: all the self driving stuff right not is in tests. Still mostly in favourable driving conditions. The whole self driving thing may still not happen, if it turns out (after a few accidents) that the technology is not quite there.
But I think the model has already been proven out with many millions of miles driven and very few accidents. The tech is here. It isn't mainstream, but it is here and it is better than humans in most conditions.
Cars have driven billions on miles on cruise control, but that does not mean that cruise control can autonomously drive a car. Planes fly billions of miles on autopilot and yet we keep the pilots. What matters is not the number of miles driven, but the situations in which the system had no idea what to do and had to disengage, and moreover the variety of such situations. Indeed these situations are in the tail of the distribution, but that tail is a really challenging place.
One series of accidents, particularly in conditions which would be easy to avoid by a human, and the whole market for the self driving car will pop like soap bubble.
One series of accidents, particularly in conditions which would be easy to avoid by a human, and the whole market for the self driving car will pop like soap bubble.
I dunno, maybe? What would the security for that look like? If I found out trains had no drivers, and you could bring them to a stop just by disconnecting the iphone?
The "iPhone and suction cup" was tongue in cheek, but train drivers are already behind locked doors And anyone can already bring most trains to a stop by pulling the red "please stop this train" lever. Just install more cameras if you're worried, or get a security guard. Cheaper than a driver and takes less time to train (heh).
There is work on self driving trains. In our university we worked on a project called autoBahn which made a proof of concept self driving train and I know of a few other research projects that worked in the same direction.
However, in my opinion there are several aspects why this is developing much slower than self driving cars:
* There are incredibly many regulations for every single piece of hardware that can be used on a train. They have such strict regulations that computing hardware that is certified for use on trains is usually 2 generations behind current hardware.
* There are not that many test tracks and you can't just roll your own track without fronting huge sums to build a test track. A self driving car can at least be driven on a private land if you don't have access to a test track and in the minimal case it does not have much more requirements than basically a paved parking area.
* If you want to do the testing on a real track (we were lucky and had a partner company that enabled us to use a public track) you have to slot into the regular transit schedule. Which means a lot of planning to do a single test and then a very tight schedule to get the test system up and running. If you miss the allotted time you have to wait or scrap the test completely for that day.
* An obvious issue is passenger safety. Even in a small train you can have several dozens of passengers which is a huge regulatory hurdle.
* And the last and in my opinion very important thing is that you can get a testing vehicle for self driving cars up and running for a reasonable amount of money, sure they are more expensive than regular cars if you count all the sensors and hardware you need but it is nothing compared to a train car which is at least one order of magnitude more expensive.
Also a huge issue is weather. In good weather conditions, self-driving <cars/trains/...> can handle themselves rather well but as soon as the weather gets bad or during the night it gets quite tricky.
For example: We had situations where snow was completely covering the track for huge parts of the way. What should the algorithm that checks for obstacles on the track do if the whole track has vanished?
The human driver has to rely on his gut feeling to decide if he can continue or not. We also had situations where quite a bit of vegetation was covering the tracks. The question is: Does this count as an obstacle that requires breaking or can you drive over it. As far as I know this was also an issue in the DARPA driving challenge where a car was driving over some grass which popped up after the car passed over it causing the sensors in the back to report an obstacle and which prevented it from backing out of the situation.
However, in my opinion there are several aspects why this is developing much slower than self driving cars:
* There are incredibly many regulations for every single piece of hardware that can be used on a train. They have such strict regulations that computing hardware that is certified for use on trains is usually 2 generations behind current hardware.
* There are not that many test tracks and you can't just roll your own track without fronting huge sums to build a test track. A self driving car can at least be driven on a private land if you don't have access to a test track and in the minimal case it does not have much more requirements than basically a paved parking area.
* If you want to do the testing on a real track (we were lucky and had a partner company that enabled us to use a public track) you have to slot into the regular transit schedule. Which means a lot of planning to do a single test and then a very tight schedule to get the test system up and running. If you miss the allotted time you have to wait or scrap the test completely for that day.
* An obvious issue is passenger safety. Even in a small train you can have several dozens of passengers which is a huge regulatory hurdle.
* And the last and in my opinion very important thing is that you can get a testing vehicle for self driving cars up and running for a reasonable amount of money, sure they are more expensive than regular cars if you count all the sensors and hardware you need but it is nothing compared to a train car which is at least one order of magnitude more expensive.
Also a huge issue is weather. In good weather conditions, self-driving <cars/trains/...> can handle themselves rather well but as soon as the weather gets bad or during the night it gets quite tricky.
For example: We had situations where snow was completely covering the track for huge parts of the way. What should the algorithm that checks for obstacles on the track do if the whole track has vanished?
The human driver has to rely on his gut feeling to decide if he can continue or not. We also had situations where quite a bit of vegetation was covering the tracks. The question is: Does this count as an obstacle that requires breaking or can you drive over it. As far as I know this was also an issue in the DARPA driving challenge where a car was driving over some grass which popped up after the car passed over it causing the sensors in the back to report an obstacle and which prevented it from backing out of the situation.
If he ever tries to train deeper models in this manner and test them on real video frames from a car, he will be in for some unpleasant surprises.
Learning to map discrete snapshots of objects to labels won't yield a system that can deal with e.g., the reality of lighting conditions that a car will experience.
Learning to map discrete snapshots of objects to labels won't yield a system that can deal with e.g., the reality of lighting conditions that a car will experience.
Author here. This first part is simple by design. It's targeted to those getting started in the field.
Oh-- my comment wasn't about the simplicity of the first part. (In fact, this is a great tutorial, thanks for posting.)
My comment is about the approach of using supervised learning to map directly from images to category labels.
My comment is about the approach of using supervised learning to map directly from images to category labels.
I'm not sure I'm following. Are you saying that using supervised learning is the wrong approach here? What would you use instead?
Yes, that's right! The way you are using supervised learning here will force the neural networks to map from textures directly to human labels. A purely feedforward network, no matter how deep, can only rote memorize the effects of the world on the images (viewing angle, lighting, etc) and will not generalize.
Another shortcoming of feedforward nets is they cannot change how they interpret local features based on integrated global aspects of a scene, like ambient lighting or backlighting.
As a result the network will fail to classify on new real world images.
If instead you use recurrence to learn features that take the dynamical and global effects into account, you'll have a better chance of success. One example of how we did his is here [1].
[1] http://blog.piekniewski.info/2016/11/04/predictive-vision-in...
Another shortcoming of feedforward nets is they cannot change how they interpret local features based on integrated global aspects of a scene, like ambient lighting or backlighting.
As a result the network will fail to classify on new real world images.
If instead you use recurrence to learn features that take the dynamical and global effects into account, you'll have a better chance of success. One example of how we did his is here [1].
[1] http://blog.piekniewski.info/2016/11/04/predictive-vision-in...
I know very little of machine learning, so...
It seems that your system is supervised for the initial training. Once the system is somewhat trained, is it possible to let it free with unsupervised training, say if the confidence is in some higher range, between some frames? For example, say there was a period of frames with very high confidence, some slightly occlusion or shadow that lowered the confidence, and then another period of high confidence. With something like motion prediction, and some confidence in where the sign was, could you use that period of lower confidence to help train, maybe with some verification from a knows, complicated, supervised data set?
tldr; Are there methods to allow these systems to keep learning once they're deployed?
edit: And this may interest you, the brain appears to predict motion: https://whitneylab.berkeley.edu/people/gerrit/MausNijhawan.P...
It seems that your system is supervised for the initial training. Once the system is somewhat trained, is it possible to let it free with unsupervised training, say if the confidence is in some higher range, between some frames? For example, say there was a period of frames with very high confidence, some slightly occlusion or shadow that lowered the confidence, and then another period of high confidence. With something like motion prediction, and some confidence in where the sign was, could you use that period of lower confidence to help train, maybe with some verification from a knows, complicated, supervised data set?
tldr; Are there methods to allow these systems to keep learning once they're deployed?
edit: And this may interest you, the brain appears to predict motion: https://whitneylab.berkeley.edu/people/gerrit/MausNijhawan.P...
Its rather the opposite. It is unsupervised initially, just learns to predict its input. Note there is confusion: the unit itself is using supervised (by the future signal) but all in all nothing needs to be labeled cause the reality just unfolds. Then, once this is done, one can use the trained features (representations) to train supervised tasks, such as the street sign tracking.
PS: yes, there is a strong literature suggesting that the brain is predicting a bunch of things. Check this long review paper http://www.fil.ion.ucl.ac.uk/~karl/Whatever%20next.pdf for plenty ideas and details.
PS: yes, there is a strong literature suggesting that the brain is predicting a bunch of things. Check this long review paper http://www.fil.ion.ucl.ac.uk/~karl/Whatever%20next.pdf for plenty ideas and details.
Oh, wow. So really this allows prediction of a complex input, in a general sense?
If I understand correctly, in context of that visual example, if it were trained with a moving camera and a static scene, then its prediction would only be able to predict scene transformations caused by that moving camera. Maybe this explains how the tracking somewhat fails when the ball is moving along the grass towards the end the scene. It doesn't "know" much about moving objects, just moving cameras? So then training with moving objects, would let it predict those, as well?
In what the video is showing, if it can predict perspective transforms from camera movement, like it seems to be doing, does that means it's making something like a 3d model, or something like a depth map used for its motion prediction, somewhere in there?
I would love to see the error video of some sort of rotating 3d wireframe that it was trained to.
This whole approach of a general "predictor" seems extremely incredible.
If I understand correctly, in context of that visual example, if it were trained with a moving camera and a static scene, then its prediction would only be able to predict scene transformations caused by that moving camera. Maybe this explains how the tracking somewhat fails when the ball is moving along the grass towards the end the scene. It doesn't "know" much about moving objects, just moving cameras? So then training with moving objects, would let it predict those, as well?
In what the video is showing, if it can predict perspective transforms from camera movement, like it seems to be doing, does that means it's making something like a 3d model, or something like a depth map used for its motion prediction, somewhere in there?
I would love to see the error video of some sort of rotating 3d wireframe that it was trained to.
This whole approach of a general "predictor" seems extremely incredible.
If the shadow is predictable then the unsupervised (self supervised) part of our Predictive model will not have too much trouble learning to deal with it. But you're right, if it did have trouble, there would be good reason to for the system to continue exploring that type of event.
One way to do this is to link PVM'a prediction error output to an "instinct" that directs it towards lower confidence events that you mentioned. This could be just orienting the camera towards those events, or in a robot producing actions that led to those events again.
Note this is not necessarily reinforcement learning but it relates to some ideas there, like the idea of novelty being rewarding.
One way to do this is to link PVM'a prediction error output to an "instinct" that directs it towards lower confidence events that you mentioned. This could be just orienting the camera towards those events, or in a robot producing actions that led to those events again.
Note this is not necessarily reinforcement learning but it relates to some ideas there, like the idea of novelty being rewarding.
I know very little of machine learning, so...
It seems that your system is supervised for the initial training. Once the system is somewhat trained, is it possible to let it free with unsupervised training, say if the confidence is in some higher range, between some frames? For example, say there was a period of frames with very high confidence, some slightly occlusion or shadow that lowered the confidence, and then another period of high confidence. With something like motion prediction, and some confidence in where the sign was, could you use that period of lower confidence to help train, maybe with some verification from a knows, complicated, supervised data set?
tldr; Are there methods to allow these systems to keep learning once they're deployed?
It seems that your system is supervised for the initial training. Once the system is somewhat trained, is it possible to let it free with unsupervised training, say if the confidence is in some higher range, between some frames? For example, say there was a period of frames with very high confidence, some slightly occlusion or shadow that lowered the confidence, and then another period of high confidence. With something like motion prediction, and some confidence in where the sign was, could you use that period of lower confidence to help train, maybe with some verification from a knows, complicated, supervised data set?
tldr; Are there methods to allow these systems to keep learning once they're deployed?
is this just a write up on the udacity class project?
I was wondering the same. He doesn't present it that way.
Author here. No, it's not related to Udacity. It's my own side project.
Is it actually allowed to use ML in an autonomous vehicle?
It would be completely impossible to build an autonomous vehicle without ML. The question is how you use it in a way that is reliable and fail safe.
I think there are likely lots of legality questions regarding standards and quality requirements. I bet the requirements will eventually be parity with human performance. We need to construct standardized test sets before we can even discuss legality questions.
Loved the article, but holy hell medium.com is a bastion of dishonest journalism! I'd move the tensorflow article somewhere else!
Meanwhile the rail industry wants to delay, spend hundreds of millions of dollars to move to state-of-the-art new signalling systems. Fine, sure, do that, but in the meantime you can save a truckload of cash (and improve safety, and reduce lateness) by replacing each driver with an iPhone and a suction cup.