Tesla's Autopilot 'tricked' to operate without driver(bbc.co.uk)
bbc.co.uk
Tesla's Autopilot 'tricked' to operate without driver
https://www.bbc.co.uk/news/technology-56854417
30 comments
Hi dang , please change the URL to [1]. The current BBC one has no detail at all.
[1] https://www.consumerreports.org/autonomous-driving/cr-engine...
[1] https://www.consumerreports.org/autonomous-driving/cr-engine...
I am still sticking with my previous guess...
"I am guessing after hitting the tree the doors got jammed, so the driver got in the back seat to try to force open the back door or trunk access. They burned alive."*
This explains why front passenger was occupied (airbag and no hands to brace for impact - this person was probably was still in stock), the driver braced using steering and got in back seat to open doors, but the smoke/fire killed them both soon.
*https://news.ycombinator.com/item?id=26871493
"I am guessing after hitting the tree the doors got jammed, so the driver got in the back seat to try to force open the back door or trunk access. They burned alive."*
This explains why front passenger was occupied (airbag and no hands to brace for impact - this person was probably was still in stock), the driver braced using steering and got in back seat to open doors, but the smoke/fire killed them both soon.
*https://news.ycombinator.com/item?id=26871493
Could the real issue here be that this technology is "too alluring".
In the same way some medicines are regulated to stop people self medicating, perhaps cars need to be made less tempting to be stupid with...
For example, perhaps a $1000 deposit that you lose if you are ever found to not be paying attention at the wheel?
In the same way some medicines are regulated to stop people self medicating, perhaps cars need to be made less tempting to be stupid with...
For example, perhaps a $1000 deposit that you lose if you are ever found to not be paying attention at the wheel?
>For example, perhaps a $1000 deposit that you lose if you are ever found to not be paying attention at the wheel?
...or a $1000 fine?
...or a $1000 fine?
>> if you are ever found to not be paying attention at the wheel?
> ...or a $1000 fine?
It's a great idea except the fact that it's unenforceable and will eventually be abused. Whether or not someone has been paying attention is a transient property you can't prove or disprove after the moment has passed.
> ...or a $1000 fine?
It's a great idea except the fact that it's unenforceable and will eventually be abused. Whether or not someone has been paying attention is a transient property you can't prove or disprove after the moment has passed.
There are more accidents happening with smartphones behind the wheel.
Drunk driving and not wearing a seatbelt became socially unacceptable after decades of horrific accidents (I recall a story of a drunk driver in Belgium plowing through an entire class of school children).
Drunk driving and not wearing a seatbelt became socially unacceptable after decades of horrific accidents (I recall a story of a drunk driver in Belgium plowing through an entire class of school children).
Let me guess: was it a mannequin with its hands taped to the wheel?
This was what I was picturing:
https://xkcd.com/1559/
https://xkcd.com/1559/
Yeah. This is a ridiculous story by the BBC - it says the car can easily be tricked and then doesn't explain how. Surely that is the primary set of facts that are needed to evaluate the story?
BBC just copied the consumerreports story. It has all the detail on how they did it. It's linked from the BBC article.
https://www.consumerreports.org/autonomous-driving/cr-engine...
https://www.consumerreports.org/autonomous-driving/cr-engine...
Fisher next placed a small, weighted chain on the steering wheel, to simulate the weight of a driver’s hand, and slid over into the front passenger seat without opening any of the vehicle’s doors, because that would disengage Autopilot
I mean, I guess this counts as "tricking" the car, but there's definitely a line somewhere at which point this becomes a Tesla-hating non-story. You could also put a brick on the accelerator in a normal car and crash yourself that way, but so what? Any safety system can be worked around by a sufficiently determined person.
I mean, I guess this counts as "tricking" the car, but there's definitely a line somewhere at which point this becomes a Tesla-hating non-story. You could also put a brick on the accelerator in a normal car and crash yourself that way, but so what? Any safety system can be worked around by a sufficiently determined person.
Any article highlighting a flaw in a system can be a "hating non-story" if you're enough of a fanboy.
If Tesla hasn't implemented a fix for the problem yet, it's possible that BBC is choosing not to share the trick, as then more people would do it. Similarly to how we have "responsible disclosure" of security issues, or not writing too much about suicide in media as it'll increase overall suicides.
most cars that I drove had a weight sensor in the seat, but I guess it is just as easy to defeat...
If it's in beta then wouldn't always-on cameras be beneficial; the data from facial expressions before a potential crash might overcome the lack of users due to opt-out?
"By comparison, BMW, Ford, GM, Subaru, and others use camera-based systems that can track the movements of a driver’s eyes and/or head position to ensure that they’re looking at the road."
Guys, isn't this obvious? I can also "trick" a knife into stabbing me instead of cutting this carrot.
This is like the basic thing of giving your users power over the product they purchased.
What next? All cars can be tricked into driving into other cars with merely some steering input? "Researchers concluded that with a mild flick of the wrist, any driver could flatten 40 children in a preschool yard. Activists have called for wrist flick regulation over the years but their appeals have been ignored by all manufacturers except Daewoo - whose cars do not respond to steering input at all"
This is like the basic thing of giving your users power over the product they purchased.
What next? All cars can be tricked into driving into other cars with merely some steering input? "Researchers concluded that with a mild flick of the wrist, any driver could flatten 40 children in a preschool yard. Activists have called for wrist flick regulation over the years but their appeals have been ignored by all manufacturers except Daewoo - whose cars do not respond to steering input at all"
If my Honda can tell when I'm sitting in a seat but don't have my seat belt on then it seems like this is an easily solvable problem.
You just brought back memories of the Subaru I had that thought my backpack was a passenger. I'd throw it on the passenger seat after class sometimes without thinking about it. Then about 10 minutes down the road, it would shift and wind up triggering the sensor. So I'd be having to reach and finagle the seat belt to shut off the distracting beeping while navigating a busy stretch of road where everyone likes to drive like idiots. Very safe. Luckily the previous owners of my current vehicle were kind enough to disconnect the seat belt chime and a few other safety features that I sure haven't missed. And yes, I always wear my seat belt and so do my passengers.
Ah, the old days of buckling up back packs and fish tanks, and pretty much anything else I didn't want to throw in the back that weighed over 30 lbs.
Ah, the old days of buckling up back packs and fish tanks, and pretty much anything else I didn't want to throw in the back that weighed over 30 lbs.
A basketball player waiting for the draft literally died yesterday because he fooled his car into thinking he had the seat belt on.
Tesla's already do this, but it's easy enough to fool - just put a bag on the seat and put the seatbelt round it.
Tesla's also have an advanced on-board computer with an AI system that can drive a car. You're telling me Tesla can figure out how to make a car drive itself by detecting everything on the outside of the car but can't figure out how to tell if a person is sitting in the drivers seat of the car? If it's that easy to fool what does that say about the self driving AI?
They already have a camera pointing out of the dashboard directly at the driver so they can tell if they were paying attention after a crash.
[deleted]
>Fisher engaged Autopilot while the car was in motion on the track, then set the speed dial (on the right spoke of the steering wheel) to 0, which brought the car to a complete stop. Fisher next placed a small, weighted chain on the steering wheel, to simulate the weight of a driver’s hand, and slid over into the front passenger seat without opening any of the vehicle’s doors, because that would disengage Autopilot. Using the same steering wheel dial, which controls multiple functions in addition to Autopilot’s speed, Fisher reached over and was able to accelerate the vehicle from a full stop. He stopped the vehicle by dialing the speed back down to zero.
If anyone is curious how to do it. From https://www.consumerreports.org/autonomous-driving/cr-engine.... Personally I wouldn't call it too easy to do; That summation makes me think it would be possible on a whim with no forethought (surely a wealth of the scenarios where someone would choose to do this), but looking at the picture in the article they needed a setup with a large weight (but not too large) tied around a length of rope (also something I don't have available on demand in my car and would have to go somewhere and buy).
I'm not saying a drunk idiot at home with some unusual materials couldn't do it but it's not as simple as something you can do anytime you feel like it.
On the topic of better systems to get the driver to look at the road: "By comparison, BMW, Ford, GM, Subaru, and others use camera-based systems that can track the movements of a driver’s eyes and/or head position to ensure that they’re looking at the road. Some vehicles—including those equipped with GM’s Super Cruise—can automatically slow to a stop if they detect that drivers have ignored repeated warnings to look at the road."
We shouldn't forget that there's a lot of prior art on this area of human-machine relations. One place where we've had automated driving for ages is in train/subway and getting their conductors to pay attention even when they don't have anything to do for hours on end is something that's been struggled with. I recall that they've tried things like adding a button that they have to press periodically, but it doesn't work (the brain still can't help falling asleep and it will automatically interact with the button). Same for other more advanced things like making them input some more complicated action to show they're awake. The brain's just too good at ignoring things it realizes aren't necessary. I think one of the best solutions, maybe the only solution that's been foolproof, is https://en.wikipedia.org/wiki/Pointing_and_calling. Not sure how we would utilize that idea for self driving cars though.
If anyone is curious how to do it. From https://www.consumerreports.org/autonomous-driving/cr-engine.... Personally I wouldn't call it too easy to do; That summation makes me think it would be possible on a whim with no forethought (surely a wealth of the scenarios where someone would choose to do this), but looking at the picture in the article they needed a setup with a large weight (but not too large) tied around a length of rope (also something I don't have available on demand in my car and would have to go somewhere and buy).
I'm not saying a drunk idiot at home with some unusual materials couldn't do it but it's not as simple as something you can do anytime you feel like it.
On the topic of better systems to get the driver to look at the road: "By comparison, BMW, Ford, GM, Subaru, and others use camera-based systems that can track the movements of a driver’s eyes and/or head position to ensure that they’re looking at the road. Some vehicles—including those equipped with GM’s Super Cruise—can automatically slow to a stop if they detect that drivers have ignored repeated warnings to look at the road."
We shouldn't forget that there's a lot of prior art on this area of human-machine relations. One place where we've had automated driving for ages is in train/subway and getting their conductors to pay attention even when they don't have anything to do for hours on end is something that's been struggled with. I recall that they've tried things like adding a button that they have to press periodically, but it doesn't work (the brain still can't help falling asleep and it will automatically interact with the button). Same for other more advanced things like making them input some more complicated action to show they're awake. The brain's just too good at ignoring things it realizes aren't necessary. I think one of the best solutions, maybe the only solution that's been foolproof, is https://en.wikipedia.org/wiki/Pointing_and_calling. Not sure how we would utilize that idea for self driving cars though.
If you want more danger, put a brick on the accelerator.