Tesla Full Self Driving Beta Causes a Crash(twitter.com)
twitter.com
Tesla Full Self Driving Beta Causes a Crash
https://twitter.com/mostlyharmlessz/status/1469421762555351042
61 comments
If you have to be at the ready to take over from the car trying to kill you at all times, wouldn't you rather just drive?
The accident happened so fast. How would anybody be able to take over fast enough if your hands aren't hovering directly next to the wheel?
You have to keep your hands on the wheel, but even then it's not really a motor control issue, the problem is attention and surprise.
I should know this, but don’t… when the vehicle is in computer driving mode with your human hand on the wheel, does the wheel still turn?
I assume for mechanical and practical reasons it does, but I don’t know to what degree it will “fight you”. How much pressure is needed to override the system? If I’m holding it and refuse to let the wheel turn, I assume it gives up and computer mode turns off? But there has to be some degree of resistance it’ll overcome right?
I assume for mechanical and practical reasons it does, but I don’t know to what degree it will “fight you”. How much pressure is needed to override the system? If I’m holding it and refuse to let the wheel turn, I assume it gives up and computer mode turns off? But there has to be some degree of resistance it’ll overcome right?
I think it varies but this from today shows it aggressively fighting the driver: https://twitter.com/fly4dat/status/1469410498928156683?s=20
Yea, that’s kind of what I thought it would be like. Hard to tell how much of that is fighting, and how much is “tactile wiggle alert” where it’s not steering that input but is wiggling.
They can wiggle alert safely if it’s full electronic steering and not connected to a tie rod. IDK if that is the case, typically sucks for feeling the road.
They can wiggle alert safely if it’s full electronic steering and not connected to a tie rod. IDK if that is the case, typically sucks for feeling the road.
That’s correct. The wheel is still moving, and you are required to minimally tension it. Stronger tension will disengage the AP.
It does seem like I'd have to maintain the same level of alertness and attention as when driving, in order to have the continuity of awareness needed to react. Including having my hands on the wheel.
You'd have to be way more alert, a well maintained car is rather predictable and won't throw itself around like that
That depends on the probability (say, per mile) that you’ll need to take over.
A lower probability is even worse psychologically imho.
I guarantee reaction time to self driving is way worse than an active driver in the same situation.
Objectively only If you perform statistically better than the automatic system
Most accidents are even due to human error (even if they are reported as such) - it's self-driving propaganda fodder.
https://amp.theatlantic.com/amp/article/620808/
https://amp.theatlantic.com/amp/article/620808/
[deleted]
So far so good. I suppose there are a few acute medical situations that might cause me to fuck up driving on par with what FSD can manage to do.
The measurement is per hr or 1000 hrs of driving, I would think
Not per situation where you think you would have done better ignoring the ones where the computer would have done objectively better
Not per situation where you think you would have done better ignoring the ones where the computer would have done objectively better
Sure. So far I’ve driven maybe 500k kilometers and I’m doing pretty okay so far. Twice a deer at low speed and that’s about it.
Every time when my car issues AEB, I’m ahead of it mentally and AEB only interferes with what I want to do. But I’ve driven race cars on Nordschleife so I guess I’m an outlier here.
Every time when my car issues AEB, I’m ahead of it mentally and AEB only interferes with what I want to do. But I’ve driven race cars on Nordschleife so I guess I’m an outlier here.
What situation is the computer going to do objectively better? I don't really care to swap any number of survivable accidents for one situation like that.
Which you almost certainly do today.
There are loads of videos of Tesla self driving fails, like this one posted yesterday, where the Tesla tries to hit a pedestrian and prevents the driver from taking control, tries to drive on the train track, and stops for the Wells Fargo sign.
https://mobile.twitter.com/TaylorOgan/status/146940457943982...
https://mobile.twitter.com/TaylorOgan/status/146940457943982...
Er... Yes, this definitely seems like it'll be ready to use any day now. Bloody hell, people _pay_ for this?
Can we change this to link to the original video, at https://www.youtube.com/watch?v=7VhrG-7SBZg ?
Video description contains details about what happened, which might not be clear from the video snippet. FSD swerved into oncoming traffic, and the driver manually pulled back to the correct lane, but lost control and went off the road.
Video description contains details about what happened, which might not be clear from the video snippet. FSD swerved into oncoming traffic, and the driver manually pulled back to the correct lane, but lost control and went off the road.
For what it is worth, I want to try FSD beta, and have a safety score of 99, and can’t.
You have to jump through so many hoops to even have FSD beta right now, that this is the devil’s deal (i.e. potential manslaughter) you accept by even trying it.
Tesla should investigate, and continue to keep FSD in beta with these strong obstacles while it improves. And eventually (arguably), it needs to put up or pay up for level five FSD which it has been selling as vaporware for something like 5 years now. Maybe this is where public and private law needs to step in.
But in my opinion, casual outrage over FSD is unwarranted until Tesla rolls out software that is less safe than humans to a lot of its users that will operate it as if it is level 5 regardless of actual performance.
You have to jump through so many hoops to even have FSD beta right now, that this is the devil’s deal (i.e. potential manslaughter) you accept by even trying it.
Tesla should investigate, and continue to keep FSD in beta with these strong obstacles while it improves. And eventually (arguably), it needs to put up or pay up for level five FSD which it has been selling as vaporware for something like 5 years now. Maybe this is where public and private law needs to step in.
But in my opinion, casual outrage over FSD is unwarranted until Tesla rolls out software that is less safe than humans to a lot of its users that will operate it as if it is level 5 regardless of actual performance.
It doesn't surprise me that it's this particular scenario, I've seen a lot of Youtube videos of them going way over the center line on turns in this kind of road (2 lane rural with 55 mph) even in broad daylight
Oh dear. FSD (Fools Self Driving) Beta once again causing havoc and putting the life of this driver at risk and deciding to hit another car after the driver regaining control of the vehicle very quickly and going off road, especially in the most dangerous time to drive, which is at night.
Had the driver not kept their eyes on the road or was sleeping in the car, this system would have crashed into that incoming vehicle. This video just proves that night-vision is non-existent in this 'safety-critical' system.
Outrageous.
Had the driver not kept their eyes on the road or was sleeping in the car, this system would have crashed into that incoming vehicle. This video just proves that night-vision is non-existent in this 'safety-critical' system.
Outrageous.
Is there corroboration that this actually is self-driving? I suppose we'd need Tesla's cooperation. Is it possible to replay the "AI view" of the same thing? Can any FSD beta users chime in?
I will say that two lane highways (aka offset impact at 110mph+ IF/BIG IF they aren't speeding) are pretty damn dangerous and I'd only do self-driving on them with a LOT of repeats. I think I'm getting close to the point of trusting self-driving on divided highways under good conditions, the big caveat being stopped cars/obstacles.
It sure doesn't help that every Tesla post on HN has comments that seem like obvious astroturfs. It's like jalopnik in the comments.
Transparency is a must. The new NHTSA head doesn't help, we do need someone to hold Tesla's claim to the light, but appointing someone with publicly posted anti-Tesla screeds? That doesn't help anything.
Self driving could save tens of thousands of lives (perhaps hundreds of thousands worldwide) each year and be a legitimate boon to the economy. NHTSA needs a dedicated group of government officials with legit industry and software experience to test, regulate, and analyze incidents. Plus it would be apocalypse to one of my less favorite sectors in the world: the auto insurance industry.
NHTSA should have immediate delivery of Self-Driving accidents with all the footage and any recorder information immediately uploaded to government servers with an active software harness to replay the logs. And it's not like this would be occaisionally tested, even with high-quality theoretical self driving, there will be daily accidents to replay from every significant vendor.
I will say that two lane highways (aka offset impact at 110mph+ IF/BIG IF they aren't speeding) are pretty damn dangerous and I'd only do self-driving on them with a LOT of repeats. I think I'm getting close to the point of trusting self-driving on divided highways under good conditions, the big caveat being stopped cars/obstacles.
It sure doesn't help that every Tesla post on HN has comments that seem like obvious astroturfs. It's like jalopnik in the comments.
Transparency is a must. The new NHTSA head doesn't help, we do need someone to hold Tesla's claim to the light, but appointing someone with publicly posted anti-Tesla screeds? That doesn't help anything.
Self driving could save tens of thousands of lives (perhaps hundreds of thousands worldwide) each year and be a legitimate boon to the economy. NHTSA needs a dedicated group of government officials with legit industry and software experience to test, regulate, and analyze incidents. Plus it would be apocalypse to one of my less favorite sectors in the world: the auto insurance industry.
NHTSA should have immediate delivery of Self-Driving accidents with all the footage and any recorder information immediately uploaded to government servers with an active software harness to replay the logs. And it's not like this would be occaisionally tested, even with high-quality theoretical self driving, there will be daily accidents to replay from every significant vendor.
"Self driving could save tens of thousands of lives (perhaps hundreds of thousands worldwide) each year..."
Launching a self driving car which crashes head on into traffic is not a good start. If they used unmaned Teslas for the live traffic tests they would be risking less lives. /s
It is interesting that consumers kind of volunteer to test a potentially lethal product. I say "kind of volunteer" because they are actually paying for the car.
Launching a self driving car which crashes head on into traffic is not a good start. If they used unmaned Teslas for the live traffic tests they would be risking less lives. /s
It is interesting that consumers kind of volunteer to test a potentially lethal product. I say "kind of volunteer" because they are actually paying for the car.
Not just the car, they are dropping an additional 10k to be specifically part of the test.
> impact at 110mph
can we get the "frontal impact == impact at sum of speed" myth die already? it's been thoroughly debunked over and over. the only way it's relevant is if there's a large weight difference between vehicles.
can we get the "frontal impact == impact at sum of speed" myth die already? it's been thoroughly debunked over and over. the only way it's relevant is if there's a large weight difference between vehicles.
I'm suspicious that this may not have occurred as described for 2 reasons: the recording doesn't show the part of the screen where the autosteer indicator is, and the original uploader took down the video as soon as it got significant attention.
The original poster gave more details here: https://teslamotorsclub.com/tmc/threads/fsd-beta-attempts-to...
My amateur interpretation of this is that the vision algorithm briefly confused the yellow of the headlights of the oncoming car as the double yellow line in the road, which caused it to believe it was seriously out of lane. After verring, the algorithm reoriented, and realized an impact was imminent and diverted off the road.
I think what's interesting about this example is that in this case, an alert human driver would not be confused in this way. It shows issues with the particular machine learning approach to automated driving.
I think what's interesting about this example is that in this case, an alert human driver would not be confused in this way. It shows issues with the particular machine learning approach to automated driving.
>vision algorithm briefly confused the yellow of the headlights of the oncoming car as the double yellow line
not the lights, but the vertical lens flares captured by single front facing center mounted camera cluster - afaik Tesla still has no stereo vision.
>the algorithm reoriented
more like driver jerked to the right while having a heart attack and seeing itself going into a full frontal collision thanks to move fast and break things mentality
not the lights, but the vertical lens flares captured by single front facing center mounted camera cluster - afaik Tesla still has no stereo vision.
>the algorithm reoriented
more like driver jerked to the right while having a heart attack and seeing itself going into a full frontal collision thanks to move fast and break things mentality
Wow, you just slapped me with this idea:
- What should be the core tenet of the self driving system for a car?
- Why, of course it should be "Move fast and break things"!I don't think that a root cause analysis like this is good for anything else than improving the algorithm.
It's the same as with face detection. The algorithms are better far superior to humans, but they fail at occasions where a human would not even think twice
It's the same as with face detection. The algorithms are better far superior to humans, but they fail at occasions where a human would not even think twice
The driver intervened and lost control. It looks like Autopilot was going all in on that incoming vehicle.
[deleted]
Flagged this. Some random video with no official investigation and countless speculations.
Keep in mind that Tesla will sue your ass for defamation if you post BS video and lie about it. Just ask the Chinese dude who posted about Tesla accelerating by itself into a crash. Turns out he stepped on the pedal himself but of course when someone on twitter posts about a Tesla fuck-up, it’s taken as gospel.
Keep in mind that Tesla will sue your ass for defamation if you post BS video and lie about it. Just ask the Chinese dude who posted about Tesla accelerating by itself into a crash. Turns out he stepped on the pedal himself but of course when someone on twitter posts about a Tesla fuck-up, it’s taken as gospel.
The degree to which people are willing to jump to the defense of Tesla self-driving is truly incredible.
The experience of “your Tesla tries to do something INSANE” is shared by literally every Tesla driver I know.
The experience of “your Tesla tries to do something INSANE” is shared by literally every Tesla driver I know.
Yes, I’ll happily jump to Tesla’s defense (not that they need it) when baseless crap is thrown around, because for every 1 of me, there are 100 who can’t wait to see Tesla crash and burn.
Thankfully we have no teslas here yet, but i gotta watch out when they arrive. Is it possible to tell if a tesla has self-diving?
Not from the exterior. When full self driving beta mode is active, there is a circular, blue steering wheel icon visible at the top of the car's control screen. So unless you're inside the Tesla, you won't be able to tell.
I would not bet on this continuing to be the case, mind you. If regulators allow this sort of thing to continue at all, I'd expect warning lights visible to pedestrians and other drivers will be required.
I don't think we are far from legislation being required to mandate some sort of indicator. (if not outlawing these features)
Yes, please. Just like "Driving instructor, keep your distance", a Tesla should say "Tesla FSD, slow down"
Why should I slow down? Get away from it asap!
that wasn't a crash! the car simply decided that the turn was dangerous and took it upon itself to disrupt the road construction industrial complex by forging a new and safer path.
It looks like auto highbeam assist turned of and it might have caused a weird vision bug
There's just something I don't like about the word "bug". It suggests a condition of the software, not a condition of the developer. Maybe it's the full phrase "caused a ... bug". As if the bug wasn't present until the condition occurred. Unless that condition is actually altering the commands in the software, the bug was already there. I could even talk about what "weird" means in this context. It's not a poke at you - I heard phrases like "caused a weird bug" all the time in our industry. In generally, and especially in situations involving safety, it's important we not engage in minimization, but call it was it is: a flaw, where conditions in the operating environment weren't properly considered.
Well, given the “flaw” is a machine learning model, who was the developer and how do they relate to the training data?
Spend 10k on FSD tip jar
Fail to buy real tyres
Or is Tesla’s stability control so bad that it underperforms 2 decade old economy cars?
Fail to buy real tyres
Or is Tesla’s stability control so bad that it underperforms 2 decade old economy cars?
Don’t forget with this sensational “headline” we don’t actually know the truth. We can’t see the driver in the video and we have only his word to go by. Statistically it’s much more likely he’s lying versus FSD causing the accident in this situation. FSD has a long way to go but it’s already demonstrably safer than humans.
Regardless, Hope this guy is okay and wasn’t injured in the accident.
Regardless, Hope this guy is okay and wasn’t injured in the accident.
Sorry, but that isn't true of Tesla's FSD. It's notoriously unsafe. Here's some evidence:
https://www.motorbiscuit.com/tesla-beta-testers-say-full-sel... https://www.inputmag.com/tech/tesla-recalls-12000-cars-over-... https://www.roadandtrack.com/news/a35878363/teslas-full-self... https://electrek.co/2021/11/15/tesla-serious-phantom-braking...
https://www.businessinsider.com/tesla-full-self-driving-beta... (with video!)
Or just look on Youtube for videos of people using the FSD, and see how frequently they have to stop it from doing something dangerous.
https://www.motorbiscuit.com/tesla-beta-testers-say-full-sel... https://www.inputmag.com/tech/tesla-recalls-12000-cars-over-... https://www.roadandtrack.com/news/a35878363/teslas-full-self... https://electrek.co/2021/11/15/tesla-serious-phantom-braking...
https://www.businessinsider.com/tesla-full-self-driving-beta... (with video!)
Or just look on Youtube for videos of people using the FSD, and see how frequently they have to stop it from doing something dangerous.