Tesla vehicle in ‘Full Self-Driving’ beta mode ‘severely damaged’ after crash(theverge.com)
theverge.com
Tesla vehicle in ‘Full Self-Driving’ beta mode ‘severely damaged’ after crash
https://www.theverge.com/2021/11/12/22778135/tesla-full-self-driving-beta-crash-fsd-california
42 comments
I don't have FSD in my Model 3, but there are times when overcoming whatever Autopilot is doing with the wheel takes a surprising amount of force. I have no doubt that the motors used to control the wheel are way stronger than me.
I think that's absolutely wrong. One of the first things I started playing with when I got the car was AP authority. It's true that Tesla steering is stiff overall. It's true that Tesla AP, unlike some other driver assistance UIs, wants you to apply full override force to disengage (that is, it will fight you as hard as it can, and the car will "pop" out of AP, sometimes with a distinct swerve if you don't reverse correct).
But the force required to do that is just not high. It's not. Just try this on an empty highway. Engage AP in a context where you're doing a lane change or something and hold the wheel firmly. It won't be remotely able to overpower you. You'll feel it tug and then it will chime and give up.
But the force required to do that is just not high. It's not. Just try this on an empty highway. Engage AP in a context where you're doing a lane change or something and hold the wheel firmly. It won't be remotely able to overpower you. You'll feel it tug and then it will chime and give up.
I have never experienced that and never heard of anyone else experiencing that. I have often disengaged Autopilot accidentally due to the limited forced required. And for the record, I also don't have FSD because I don't trust it.
Are you sure you couldn't overpower it or were you subconsciously holding back because you were worried about jerking the wheel too hard and how that would impact the vehicle? If you have ever driven a car without power steering, there is a surprising amount of force required to actually steer a vehicle mechanically, but it is something every driver used to do so it shouldn't be that difficult even in the worst situations when the power steering is refusing to cooperate.
Are you sure you couldn't overpower it or were you subconsciously holding back because you were worried about jerking the wheel too hard and how that would impact the vehicle? If you have ever driven a car without power steering, there is a surprising amount of force required to actually steer a vehicle mechanically, but it is something every driver used to do so it shouldn't be that difficult even in the worst situations when the power steering is refusing to cooperate.
> I have never experienced that and never heard of anyone else experiencing that.
That's good. It's happened to me several times, and it's one of the reasons I'm not a fan of Autopilot. When it happens, the car is usually following a curve in the road. I've also had it turn off because I barely touched the wheel, and that usually happens on a straightaway.
I'm not sure if I could overpower it or not if I'm honest. Last night when it happened, it took a lot of force before it relinquished control to me. While fighting it I was definitely thinking about how it'd react once it "let go", which could have easily influenced how aggressive I was in applying force.
Regarding power steering - I have a 2006 Subaru STI that's a track-only car. The power steering pump began leaking years ago, so I removed it and the air conditioning entirely. It takes a lot of effort to steer that car, but that makes it pretty fun on the track too. A day at the track often means my upper body is sore the next day.
That said, I'm not sure how that's relevant to Tesla's FSD or Autopilot.
I've not dug into the mechanics of the Model 3, but it seems reasonable to suggest that for Autopilot or FSD to work, the mechanisms that support power steering can also be automated through software. Meaning, it's unlikely that there's a traditional power steering pump, and the automation is done with stepper or servo motors. It's more likely that it's done with a single motor. I could be wrong, of course.
That said, irregardless of how it works, it's reasonable to assume that that system has to be able to match or exceed what a driver is capable of doing, force-wise. If that's the case, then it seems totally reasonable that the car could prevent me from steering it manually if it wanted to.
That's good. It's happened to me several times, and it's one of the reasons I'm not a fan of Autopilot. When it happens, the car is usually following a curve in the road. I've also had it turn off because I barely touched the wheel, and that usually happens on a straightaway.
I'm not sure if I could overpower it or not if I'm honest. Last night when it happened, it took a lot of force before it relinquished control to me. While fighting it I was definitely thinking about how it'd react once it "let go", which could have easily influenced how aggressive I was in applying force.
Regarding power steering - I have a 2006 Subaru STI that's a track-only car. The power steering pump began leaking years ago, so I removed it and the air conditioning entirely. It takes a lot of effort to steer that car, but that makes it pretty fun on the track too. A day at the track often means my upper body is sore the next day.
That said, I'm not sure how that's relevant to Tesla's FSD or Autopilot.
I've not dug into the mechanics of the Model 3, but it seems reasonable to suggest that for Autopilot or FSD to work, the mechanisms that support power steering can also be automated through software. Meaning, it's unlikely that there's a traditional power steering pump, and the automation is done with stepper or servo motors. It's more likely that it's done with a single motor. I could be wrong, of course.
That said, irregardless of how it works, it's reasonable to assume that that system has to be able to match or exceed what a driver is capable of doing, force-wise. If that's the case, then it seems totally reasonable that the car could prevent me from steering it manually if it wanted to.
Have you talked to Tesla support about this? I don't think what you are experiencing is intended or common (my quick googling didn't find other reports of this issue). But if you are experiencing it, maybe the driver in the accident did too. But that would mean this accident is a combination of two unrelated issues.
My googling has also suggested that Tesla does have a pretty standard EPS system although it doesn't use a pump and fluid like older systems.
My googling has also suggested that Tesla does have a pretty standard EPS system although it doesn't use a pump and fluid like older systems.
Cars without power steering were designed quite a bit differently, not quite comparable to a modern car with broken power steering.
Motor that turns wheels is way stronger than human. Try to turn steering wheel with engine turned off.
If a car uses electric power steering, your resistance in the wheel will be different from a vehicle that has hydraulic power steering, or even older cars without power steering.
The designs for each of these three are different and will - while make the resistance heavier when un-powered, are not quite the same between each other.
The designs for each of these three are different and will - while make the resistance heavier when un-powered, are not quite the same between each other.
Yea, it doesn't sound like the driver is truthful in the report. It is super easy to disengage autopilot - I do it all the time on accident. Just a little nudge of the wheel and it just gives up. Of course, this is assuming there isn't a firmware bug, but such a bug would have likely been reported publicly on the dozens of Tesla forums given it's severity
[deleted]
Possible FSD disengagement problems aside...
> it wouldn't clear concerns about the safety of FSD.
If you follow FSD review channels like AI Addict, you'll know that there's no "safety" in the system. The human must maintain control at all times, as part of the agreement with Tesla for using the system, because it is clearly unsafe, if left on its own.
> it wouldn't clear concerns about the safety of FSD.
If you follow FSD review channels like AI Addict, you'll know that there's no "safety" in the system. The human must maintain control at all times, as part of the agreement with Tesla for using the system, because it is clearly unsafe, if left on its own.
Don’t they have games built into the car that let you use the steering wheel (while parked, of course) to control? Surely this wouldn’t be turning the wheels?
Yes, the wheels turn when playing those games and it makes the steering wheel controls pretty stupid in my opinion. Who wants to put that much wear on their tires to play a video game?
That can't possibly be true, can it???
Yes, it is true. Here is video evidence:
https://www.youtube.com/watch?v=_sanbI_ze8c&t=111s
https://www.youtube.com/watch?v=_sanbI_ze8c&t=111s
That's incredibly dumb. Like, wow.
True - when car is driving itself, it's relatively easy to override automatic steering and disengage.
I'd like to add my anecdote. I have been driving my Tesla for about 2 months now, and it definitely feels safer than the ICE cars I drove before. This is mainly due to more information given by the Tesla such as the distance between my car and other objects, and it also shows the lanes around the car so it's easier for me to keep a safe distance from other cars nearby.
I also tried the 'FSD' on freeways where it can navigate on autopilot. It's really feels amazing and takes a lot of burden of driving at high speed off of me.
However, recently I was going onto the acceleration ramp, and turned on navigate on autopilot, Tesla told me to confirm a lane change to the right, I did and it somehow didn't see a car to the right of me trying to change lane to its left. That would be to same spot I would have gone to. So I had to manually intervene and take control of the wheel myself.
Manual override isn't difficult at all though, and there are multiple ways to do it. It can be done through turning the wheel manually, toggle the right stick upwards, or hit the brake. So I think if you still pay attention (maybe not as much attention as driving a non-Tesla car) when driving a Tesla, it's going to be okay.
I also tried the 'FSD' on freeways where it can navigate on autopilot. It's really feels amazing and takes a lot of burden of driving at high speed off of me.
However, recently I was going onto the acceleration ramp, and turned on navigate on autopilot, Tesla told me to confirm a lane change to the right, I did and it somehow didn't see a car to the right of me trying to change lane to its left. That would be to same spot I would have gone to. So I had to manually intervene and take control of the wheel myself.
Manual override isn't difficult at all though, and there are multiple ways to do it. It can be done through turning the wheel manually, toggle the right stick upwards, or hit the brake. So I think if you still pay attention (maybe not as much attention as driving a non-Tesla car) when driving a Tesla, it's going to be okay.
Paying attention while a computer does a task for you automatically and correctly 99.9% of the time is basically impossible though. That point has been made many times before.
I wonder if this will paradoxically lead to Tesla's becoming less safe overall as the software improves, because errors will start to become uncommon enough that people really stop paying attention.
From the sound of it it is currently unreliable enough that most people know that they really do have to pay attention.
I wonder if this will paradoxically lead to Tesla's becoming less safe overall as the software improves, because errors will start to become uncommon enough that people really stop paying attention.
From the sound of it it is currently unreliable enough that most people know that they really do have to pay attention.
> That point has been made many times before.
It has been assert many times without much actual evidence.
> I wonder if this will paradoxically lead to Tesla's becoming less safe overall as the software improves
Again, something that people on HN are often totally sure is true but I have never seen any actual evidence that is true.
The believe in this is based on some specific cases in the airline industry that I am not sure necessarily translate to driving.
It has been assert many times without much actual evidence.
> I wonder if this will paradoxically lead to Tesla's becoming less safe overall as the software improves
Again, something that people on HN are often totally sure is true but I have never seen any actual evidence that is true.
The believe in this is based on some specific cases in the airline industry that I am not sure necessarily translate to driving.
The bottom line is that FSD as sold is nowhere near ready for prime time, still. As in, you always have to supervise it.
Who's wants to be their cars supervisor?
For me, it's either fully drive and enjoy your driving ... or full FSD and relax ... if it's somewhere in between, per Tesla, it's pointless (and don't get me wrong, lane assist and cruise control are handy, but they've been around for eons).
Who's wants to be their cars supervisor?
For me, it's either fully drive and enjoy your driving ... or full FSD and relax ... if it's somewhere in between, per Tesla, it's pointless (and don't get me wrong, lane assist and cruise control are handy, but they've been around for eons).
Well that's moving the goal post.
Fact is tons of people use lots of driver assistance.
Fact is tons of people use lots of driver assistance.
No, not really. I mean, I'll use lane-assist and cruise control for very simple situations on a motorway. That's it. I know what the car is going to do, but I'm still driving and that's fine.
It would drive me nuts to have to supervise an FSD car through a city or any situation beyond the above simple motorway situation.
You'll never convince me otherwise. I don't want to be my cars manager.
It would drive me nuts to have to supervise an FSD car through a city or any situation beyond the above simple motorway situation.
You'll never convince me otherwise. I don't want to be my cars manager.
> It has been assert many times without much actual evidence.
Probably because it is so obviously true surely? Do you really doubt it?
We've already seen Teslas crash due to autopilot failures in ways that definitely wouldn't have happened if people were actually paying attention (e.g. driving into the side of lorries) so there is at least some evidence anyway.
Probably because it is so obviously true surely? Do you really doubt it?
We've already seen Teslas crash due to autopilot failures in ways that definitely wouldn't have happened if people were actually paying attention (e.g. driving into the side of lorries) so there is at least some evidence anyway.
> Probably because it is so obviously true surely? Do you really doubt it?
Yeah. I do doubt it. The question is about if the fall off of attention is larger then fall of of amount incidents. And then how many of those incidents are of a critically where you can not overcome the not paying attention part.
Its not at all clear to me if its a net negative in security, in fact I would suspect the opposite.
> We've already seen Teslas crash due to autopilot failures in ways that definitely wouldn't have happened if people were actually paying attention
Not really. We have seen Tesla crash mostly in situation where they crash all the time already. And that analysis is prove of existence, not prove of significance and net negative security. You need to actually show that this happens MORE often if Autopilot is engaged.
I have seen 100s of videos were Autopilot avoids a vehicle cutting Tesla off for example. In many of those cases even a human driver would not have react so quickly. Most of those cases would not result in death but some could. You need to actually take all of that into account.
Yeah. I do doubt it. The question is about if the fall off of attention is larger then fall of of amount incidents. And then how many of those incidents are of a critically where you can not overcome the not paying attention part.
Its not at all clear to me if its a net negative in security, in fact I would suspect the opposite.
> We've already seen Teslas crash due to autopilot failures in ways that definitely wouldn't have happened if people were actually paying attention
Not really. We have seen Tesla crash mostly in situation where they crash all the time already. And that analysis is prove of existence, not prove of significance and net negative security. You need to actually show that this happens MORE often if Autopilot is engaged.
I have seen 100s of videos were Autopilot avoids a vehicle cutting Tesla off for example. In many of those cases even a human driver would not have react so quickly. Most of those cases would not result in death but some could. You need to actually take all of that into account.
That’s a good point. I think that’s a phase that we’ll go through during the development and it’s inevitable, but that’s just my opinion.
"Despite its name, FSD is not an autonomous driving system"
Really, it's criminal at this point.
Really, it's criminal at this point.
More fake news from the verge on Tesla. They are well known for this at this point.
[deleted]
To be fair, full automation just needs to be better than a buzzed frat boy texting his boys while driving 90 mph.
These things will happen, no one can write perfect code after all. But I'm still very optimistic. Right now we're in a weird Twilight Zone, where you're expecting drivers to be able to take command from an automated system at a moment's notice. Odds are the driver was zoned out or texting his partner or something.
These things will happen, no one can write perfect code after all. But I'm still very optimistic. Right now we're in a weird Twilight Zone, where you're expecting drivers to be able to take command from an automated system at a moment's notice. Odds are the driver was zoned out or texting his partner or something.
Uh, no, it needs to be far far better than that. Unless your claim is that is the majority of drivers out there.
I have no idea how you could come to the conclusion that FSD "just needs to be better than a distracted speeding drunk driver".
I have no idea how you could come to the conclusion that FSD "just needs to be better than a distracted speeding drunk driver".
This was said in jest. But given the sheer number of people who drive while texting , get mad and run red lights, or generally don't respect how dangerous driving is , the bar is pretty low.
You can drive outrageously bad, and even when you hit something you typically aren't banned for life from driving. An AI wouldn't keep inching forward at a red light and rear end someone. Even with what we have now with Waymo is much better than the typical driver.
You can drive outrageously bad, and even when you hit something you typically aren't banned for life from driving. An AI wouldn't keep inching forward at a red light and rear end someone. Even with what we have now with Waymo is much better than the typical driver.
My bar for Full Self Driving for any vehicle is better than the average driver. As long as it's better than the average driver, it's an improvement.
"As long as it's better than the average driver, it's an improvement."
Careful- this is true if interpreted juuust right, but it's subtle.
Ultimately, you care about, roughly, the net decrease in Bad Things (injuries, deaths, property damage, weighted however you want) (and possibly also the net increase in Good Things (people being able to get places, mostly; otherwise, an excellent FSD system is one which simply stays in park and refuses to move!)) that results from the deployment of an FSD system.
That looks roughly like the sum, across all trips, of "expected Bad Things if FSD was available as an option for that trip" minus "expected Bad Things if FSD was not available".
The key here is considering each trip as a choice between FSD and non-FSD and /then/ averaging. You can't just say "FSD's per-mile accident rate is lower, so it's safer"- that misses a lot of important effects!
For example, consider selection effects related to weather and road conditions. It's easy enough to imagine an FSD system that falls apart entirely in bad weather, but works alright (but worse than a human driver!) in good weather. Let's say it's got 0.1 Bad Thing Points Per Mile (BTPPM) in good weather and 100 BTPPM in bad weather. But human drivers also do worse in bad weather, naturally- maybe they've got 0.05 Bad Thing Points Per Mile in good weather and 10 Bad Thing Points Per Mile in bad weather.
Now, assuming the poor performance in bad weather is obvious to drivers, drivers will only activate FSD in good weather. So the measured BTPPM for FSD is roughly 0.1. Human drivers, though, still have to drive in bad weather sometimes - so you might see, say, 1 BTPPM from human drivers.
Even though human drivers are better than this hypothetical FSD system in all conditions, they look way worse!
And that's just one effect- here's a related, but distinct one. Take the same example from before, but now the FSD system matches the human in good weather and bad weather (0.05 and 10 BTPPM) and, in awful weather and road conditions, where the human might have 1000BTPPM, FSD manages just 100BTPPM.
The FSD system's performance matches or is superior to the human in all situations. Surely this leads to a net decrease in Bad Things?
Not necessarily- what if people take trips in awful weather at 100BTPPM which they wouldn't take with a human driver at 1000BTPPM? (And this assumes people aren't lulled into a false sense of security because they know FSD is "safe", leading them to underestimate the risk in awful conditions). This is where you have to weigh in "how much benefit does taking this trip in the first place provide".
And one more thing: remember you have to consider the vehicle. Any vehicle with FSD can and would almost certainly (if it's human-drivable at all) provide very sophisticated ADAS features when driven by a human, and would likely be new enough to have quite modern safety features. So you have to remember to not just think about "FSD accident rate per mile v.s. human"; you do have to make sure to compare within the same vehicle.
"Better than the average driver" could be interpreted to take all this into account, but the straightforwards interpretation ("lower accident rate on average") doesn't, and can be surprisingly misleading.
Careful- this is true if interpreted juuust right, but it's subtle.
Ultimately, you care about, roughly, the net decrease in Bad Things (injuries, deaths, property damage, weighted however you want) (and possibly also the net increase in Good Things (people being able to get places, mostly; otherwise, an excellent FSD system is one which simply stays in park and refuses to move!)) that results from the deployment of an FSD system.
That looks roughly like the sum, across all trips, of "expected Bad Things if FSD was available as an option for that trip" minus "expected Bad Things if FSD was not available".
The key here is considering each trip as a choice between FSD and non-FSD and /then/ averaging. You can't just say "FSD's per-mile accident rate is lower, so it's safer"- that misses a lot of important effects!
For example, consider selection effects related to weather and road conditions. It's easy enough to imagine an FSD system that falls apart entirely in bad weather, but works alright (but worse than a human driver!) in good weather. Let's say it's got 0.1 Bad Thing Points Per Mile (BTPPM) in good weather and 100 BTPPM in bad weather. But human drivers also do worse in bad weather, naturally- maybe they've got 0.05 Bad Thing Points Per Mile in good weather and 10 Bad Thing Points Per Mile in bad weather.
Now, assuming the poor performance in bad weather is obvious to drivers, drivers will only activate FSD in good weather. So the measured BTPPM for FSD is roughly 0.1. Human drivers, though, still have to drive in bad weather sometimes - so you might see, say, 1 BTPPM from human drivers.
Even though human drivers are better than this hypothetical FSD system in all conditions, they look way worse!
And that's just one effect- here's a related, but distinct one. Take the same example from before, but now the FSD system matches the human in good weather and bad weather (0.05 and 10 BTPPM) and, in awful weather and road conditions, where the human might have 1000BTPPM, FSD manages just 100BTPPM.
The FSD system's performance matches or is superior to the human in all situations. Surely this leads to a net decrease in Bad Things?
Not necessarily- what if people take trips in awful weather at 100BTPPM which they wouldn't take with a human driver at 1000BTPPM? (And this assumes people aren't lulled into a false sense of security because they know FSD is "safe", leading them to underestimate the risk in awful conditions). This is where you have to weigh in "how much benefit does taking this trip in the first place provide".
And one more thing: remember you have to consider the vehicle. Any vehicle with FSD can and would almost certainly (if it's human-drivable at all) provide very sophisticated ADAS features when driven by a human, and would likely be new enough to have quite modern safety features. So you have to remember to not just think about "FSD accident rate per mile v.s. human"; you do have to make sure to compare within the same vehicle.
"Better than the average driver" could be interpreted to take all this into account, but the straightforwards interpretation ("lower accident rate on average") doesn't, and can be surprisingly misleading.
OK so now we are looking at yet another Tesla that is on 'LSD' instead of 'FSD' which explains the erratic behaviour that is going on with these vehicles crashing and swerving when activating the 'beta' feature.
Imagine using it when driving at night. The most dangerous time to drive.
No thanks and no deal to safety critical 'beta' software.
Imagine using it when driving at night. The most dangerous time to drive.
No thanks and no deal to safety critical 'beta' software.
LSD increases my reaction times by a factor of roughly 2.5x, for anyone curious.
Anecdata against your anecdata: LSD decreases my reaction time significantly and I have a tendency to zone the fuck out
What is your reaction time to start with?
reaction time to what might be a factor here, as both AI sensor fusion and human perception are prone to hallucinations.
My anecdata was collected on humanbenchmark.com with a fast monitor.
Although saying that I think the issue is more that the action is slower than the reaction taking longer, if there's a difference...
Although saying that I think the issue is more that the action is slower than the reaction taking longer, if there's a difference...
Something doesn't add up about this report. Teslas are not fly-by-wire. Their steering wheel and brakes are mechanical. You can't turn the wheel without the wheels turning. It therefore shouldn't be possible for the computer to overrule the human. Considering that, it is unclear what intervention the driver actually did and what "the car by itself took control" means. My initial hunch is that the driver isn't being entirely truthful in an attempt to shift blame from themselves to Tesla. But even if that is the case, it wouldn't clear concerns about the safety of FSD.