Tesla FSD beta crashes into object [video](youtube.com)
youtube.com
Tesla FSD beta crashes into object [video]
https://www.youtube.com/watch?v=sbSDsbDQjSU
187 comments
I work in the self-driving space. Before I did I was super hype about what Tesla were doing with self-driving and auto-pilot, but once I actually started seriously looking at the safety ramifications of a vehicle driving with these technologies I changed tune really quickly.
A massive problem, particularly with the more capable L4 style technologies is you get lulled into a false sense of security because it'll drive a decent amount of distance perfectly fine right up until it doesn't (and it's normally spectacularly bad when it fails).
Testing purely on closed track or from simulated analysis only goes so far, you definitely need to do public road testing before you can hand it to end-users. But the driver needs to do advance driver training so that they're more aware of the hazards, they need to always be ready to take over (as with an L3 system, even if you're developing L4), they need to fully understand the capabilities, ODD and behaviour of the software, you need to keep the durations/distances short to avoid driver fatigue and ideally you have a second person in the vehicle to keep them honest.
Letting end-consumers use this technology on public roads is insane. It feels to me like the reason Tesla do it is they've boxed themselves into a corner because they've already sold it.
A massive problem, particularly with the more capable L4 style technologies is you get lulled into a false sense of security because it'll drive a decent amount of distance perfectly fine right up until it doesn't (and it's normally spectacularly bad when it fails).
Testing purely on closed track or from simulated analysis only goes so far, you definitely need to do public road testing before you can hand it to end-users. But the driver needs to do advance driver training so that they're more aware of the hazards, they need to always be ready to take over (as with an L3 system, even if you're developing L4), they need to fully understand the capabilities, ODD and behaviour of the software, you need to keep the durations/distances short to avoid driver fatigue and ideally you have a second person in the vehicle to keep them honest.
Letting end-consumers use this technology on public roads is insane. It feels to me like the reason Tesla do it is they've boxed themselves into a corner because they've already sold it.
My main concern is that Tesla will end up poisoning the well in a regulatory sense, and other SDCs that are focused on doing narrow verticals properly will get caught in the crossfire.
The common pro Tesla argument of delaying self driving costing lives is absolutely valid. But it doesn't follow that allowing Tesla FSD to run red lights is the best way to accelerate the adoption as a whole.
If the whole industry was as cautious as Waymo, I think the risk of regulation would be minimal.
This is orthogonal to how well done the system is. I am extremely impressed it works as well as it does. But it's not good enough.
The common pro Tesla argument of delaying self driving costing lives is absolutely valid. But it doesn't follow that allowing Tesla FSD to run red lights is the best way to accelerate the adoption as a whole.
If the whole industry was as cautious as Waymo, I think the risk of regulation would be minimal.
This is orthogonal to how well done the system is. I am extremely impressed it works as well as it does. But it's not good enough.
> The common pro Tesla argument of delaying self driving costing lives is absolutely valid.
It would be if self driving Teslas were safer, but the evidence seems to suggest that they crash more often than regular drivers once you control for when it is used.
It would be if self driving Teslas were safer, but the evidence seems to suggest that they crash more often than regular drivers once you control for when it is used.
Tesla doesn’t have to be safer now to follow this reasoning.
If Tesla’s approach ultimately succeeds in getting FSD working and into the mainstream, then for each year earlier that happens than it otherwise would, up to about 30,000 lives are saved, millions of injuries prevented, and roughly $1 trillion USD in total lifetime value is preserved as the worldwide accident rate goes to zero.
In that context, FSD should absolutely be civilization’s next moonshot.
That’s to say nothing of the other economic benefits besides simply killed and maiming fewer people and destroying less property.
If Tesla’s approach ultimately succeeds in getting FSD working and into the mainstream, then for each year earlier that happens than it otherwise would, up to about 30,000 lives are saved, millions of injuries prevented, and roughly $1 trillion USD in total lifetime value is preserved as the worldwide accident rate goes to zero.
In that context, FSD should absolutely be civilization’s next moonshot.
That’s to say nothing of the other economic benefits besides simply killed and maiming fewer people and destroying less property.
Interesting idea. But the only problem with it is the assumption Tesla can succeed. I am not sure about that.
The other, darker, possibility is that Tesla does so poorly in this moonshot that they poison the well for everyone else, some of whom might be more equipped to succeed.
The other, darker, possibility is that Tesla does so poorly in this moonshot that they poison the well for everyone else, some of whom might be more equipped to succeed.
To be clear, I’m not saying they will or won’t succeed! And the converse is true too!
If through their actions FSD is ultimately delayed a year, then they contributed to extended the carnage of human driven automobiles.
The point is the scale of the problem, and how valuable it is to solve it.
If through their actions FSD is ultimately delayed a year, then they contributed to extended the carnage of human driven automobiles.
The point is the scale of the problem, and how valuable it is to solve it.
There are a lot of legs to your hypothetical here. In particular assuming that Tesla's approach ultimately succeeds is a big leap.
If Tesla's approach ultimately turns into a dead end (and that is the outcome of the overwhelming majority of very ambitious tech projects) then all the time and resources spent on Tesla's approach could be seen in hindsight to have been diverted away from potentially fruitful projects and are actually making mainstream FSD happen later than they otherwise would.
FSD being civilization's next moonshot doesn't mean Tesla should necessarily get a free pass to test beta FSD tech out on public roads without scrutiny.
If Tesla's approach ultimately turns into a dead end (and that is the outcome of the overwhelming majority of very ambitious tech projects) then all the time and resources spent on Tesla's approach could be seen in hindsight to have been diverted away from potentially fruitful projects and are actually making mainstream FSD happen later than they otherwise would.
FSD being civilization's next moonshot doesn't mean Tesla should necessarily get a free pass to test beta FSD tech out on public roads without scrutiny.
How did you calculate the $1 trillion/year?
What cost would be attributed to Covid, roughly?
What cost would be attributed to Covid, roughly?
https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...
That’s just the USA
That’s just the USA
I find this line of reasoning strange - it's like if we only kind of figured out how to do heart transplants on pigs, and then immediately went full steam ahead operating on people - because otherwise it would've costed the lives of people who didn't receive the operation
Are you sure you're replying to my line of reasoning?
Because I'm saying just because the goal is extremely worthwhile, it's not carte blanche to move fast and break things.
Because I'm saying just because the goal is extremely worthwhile, it's not carte blanche to move fast and break things.
Really? That seems like a perfectly reasonable line of thinking (except of course, for uncertainty regarding pig transplant).
But lives killed by inaction should be counted!
But lives killed by inaction should be counted!
Yeah, if you look at safety standards in other industries, this is really unacceptable. It should be stopped immediately.
Also, why do we allow car manufacturers to test their own software? Shouldn't it be done by a third party? And shouldn't that third party be the only ones allowed to push updates to the car?
Also, why do we allow car manufacturers to test their own software? Shouldn't it be done by a third party? And shouldn't that third party be the only ones allowed to push updates to the car?
You don't have to look at other industries. Other auto-manufacturers do their testing on test tracks.
And their self driving is miles behind tesla
I remember literally over 5 years ago hearing from someone at a big auto-manufacturer, and they just explained, they can't afford to have their cars known for killing people. They sell a shit tonne of cars, and if they start running people over they're done. It'd be an extinction level event for their brand, and probably a serious knock to the entire industry. Apparantly Tesla is happy to take that risk. it's not that Tesla is more advanced, it's that they're happy making claims that no other company in an industry obsessed with safety would make.
Imagine Volvo, but instead of Volvo you have a company that distinguished themselves by their lack of interest in safety.
Imagine Volvo, but instead of Volvo you have a company that distinguished themselves by their lack of interest in safety.
Only recently looking at cars, what I have found interesting is collision detection and automatic breaking. It seems some manufacturers have a reputation for getting it right, and other manufacturers a reputation for a terrifying feature that drivers disable due to it going off at exactly the wrong time.
I find my father-in-law’s Volkswagen T-Cross terrifying to drive. If it’s not distracting you with shrill warning beeps and bongs, it’s getting confused and slamming on the brakes at every slick or shiny surface. It is unquestionably more dangerous than if it just left the driving to me.
Hard to understand how people have affection for this brand.
Hard to understand how people have affection for this brand.
...Because they weren't daft enough to commit to emplying blackboxes with no means of formal proofing to a safety-critical operation. Musk's approach is a massive public safety no-no. The cost of specifying and proving through trial the capabilities of what Musk is aiming for is the work of several lifetimes. Musk and Tesla just fucking YOLO it, yeeting out OTA's that substantially change the behavior of an ill-tuned system whose behavior can't even be reliably enumerated; and sinking the operational risk in drivers on the road.
Sometimes, conspicuous lack of progress is a good thing. It isn't something you necessarily appreciate until you suddenly start having to confront the law of large numbers in a very real and tangible way. Some incremental changes simply are not feasible to take until they are complete. Level 3 automation is one of those...
Sometimes, conspicuous lack of progress is a good thing. It isn't something you necessarily appreciate until you suddenly start having to confront the law of large numbers in a very real and tangible way. Some incremental changes simply are not feasible to take until they are complete. Level 3 automation is one of those...
There is no solution to self driving that doesn’t involve a black box. The safety of the system is easy to measure. When there are fewer interventions than accidents for a solid chunk of time, FSD will be safer. It could eventually reach 1 intervention per hundred thousand accidents, if you would just let them continue.
> When there are fewer interventions than accidents for a solid chunk of time, FSD will be safer. It could eventually reach 1 intervention per hundred thousand accidents, if you would just let them continue.
And in the meantime, I and other drivers, cyclists, pedestrians are subject to increased danger for what? Oh, Tesla's profits? Forgive us if we don't all see this as an acceptable tradeoff.
And in the meantime, I and other drivers, cyclists, pedestrians are subject to increased danger for what? Oh, Tesla's profits? Forgive us if we don't all see this as an acceptable tradeoff.
They aren’t in any danger. The guy driving in the video is crazy and not disengaging when the car is misbehaving. With your hand brushing the wheel, a person can regain full control of the vehicle well before there is any danger. And yes, I would like to see not only Teslas profits go up, because they are the only company doing self driving, I would also just like to see this project move forward. It’s the coolest project in the world and if it succeeds it will save millions upon millions of innocent lives.
Furthermore if you really were so edge-of-your-seat scared of traffic fatalities then Tesla would be at the bottom of your list. Why don’t you go do something about the droves of people that stream out the back of bars and into their cars every night? They kill thousands every year meanwhile Tesla has killed roughly zero people.
Furthermore if you really were so edge-of-your-seat scared of traffic fatalities then Tesla would be at the bottom of your list. Why don’t you go do something about the droves of people that stream out the back of bars and into their cars every night? They kill thousands every year meanwhile Tesla has killed roughly zero people.
It really doesn't matter whether the driver should or should not be disengaging, there are many, many studies categorically proving that "allowing the driver to be mostly relaxed and not required, only to require immediate intervention in dangerous situations" is absolutely, empirically less safe. You can't just white wash it away by "oh well, it will get better". When? And don't mention a word about Elon's opinion on when. The guy has been promising "this year" every single year for nine years now. More realistic estimates have this a decade, or two, away, at the very earliest. And I have huge doubts that when it does, Tesla will be nowhere near it. Their phantom braking fiasco proves just how horrific Tesla's approach to testing is, throwing multiple releases out into the wild with less than 72 hours between them, for absolute safety features. Anyone who claims that those releases were subject to any form of rigor in testing whatsoever is deluded, and anyone claiming that testing it on the public roads is somehow acceptable is equally deluded.
I am very, very well aware of exactly what causes traffic fatalities. According to the software at my work, I have personally responded to 378 fatality MVAs as a paramedic. Please don't try to assume everyone is ignorant about realities - we are not blindered, and only physically capable of recognizing and responding to one danger at a time.
I am very, very well aware of exactly what causes traffic fatalities. According to the software at my work, I have personally responded to 378 fatality MVAs as a paramedic. Please don't try to assume everyone is ignorant about realities - we are not blindered, and only physically capable of recognizing and responding to one danger at a time.
You can't ask people to use a driving aid and not end up less focused. With advertising, "infotainment" (which is really disguised entertainment), music, oustide environment and passengers it is already hard for a driver to focus on his driving. You can't expect any human being short of people paid for that to keep hands brushing the wheel and feet ready to slam the brakes.
Having said that I am not sure most people are safer for cyclists and pedestrians. FSD is in such a bad state right now that the tesla is driving in the streets at the speed 80y old people do. What I saw in a video is a car that drive at a similar pace to a cyclist, it is even much slower in the crossing sections.
Having said that I am not sure most people are safer for cyclists and pedestrians. FSD is in such a bad state right now that the tesla is driving in the streets at the speed 80y old people do. What I saw in a video is a car that drive at a similar pace to a cyclist, it is even much slower in the crossing sections.
And however much Tesla likes to say "Oh, yes, yes, the driver should be paying full attention", everything else they say and do says the opposite. Latest example is the update that rearranged some of the climate control and added/updated some larger hot buttons at the bottom of the screen. Not all functions are available to be pinned at the bottom. You get a limited choice, which includes Netflix.
So to be clear you can have an always available hot button for Netflix, but not for climate control. All Tesla's handwaving is entirely bullshit. "The driver is in the seat for legal purposes only. The car is driving itself."
This is horseshit of the highest order.
So to be clear you can have an always available hot button for Netflix, but not for climate control. All Tesla's handwaving is entirely bullshit. "The driver is in the seat for legal purposes only. The car is driving itself."
This is horseshit of the highest order.
> There is no solution to self driving that doesn’t involve a black box.
LIDAR greatly reduces the "black box" necessity. It basically allows you to do things like "if object is in the way then hit brake/move elsewhere", where the sensor doesn't really fail in good weather.
Given its safety over DL-only solutions, this should be step 1 to getting to FSD. Not reckless beta-testing with black box techniques.
Tesla has chosen the cheap way, which is also the irresponsible way.
LIDAR greatly reduces the "black box" necessity. It basically allows you to do things like "if object is in the way then hit brake/move elsewhere", where the sensor doesn't really fail in good weather.
Given its safety over DL-only solutions, this should be step 1 to getting to FSD. Not reckless beta-testing with black box techniques.
Tesla has chosen the cheap way, which is also the irresponsible way.
I'd rather my car's safety systems be later to market but proven safe, than early to market and have me and the others around me as unpaid beta testers.
This isn't even beta testing, it's closer to development. The driving shown in the video is that of a drunk or texting driver. For at least half of those turns it couldn't even pick a line and stick to it, instead swerving haphazardly as it became more aware of where the lane was. Never mind the constant turns into bus only lanes. I'd love to see municipalities start writing tickets on all the well-documented violations in these videos, citing Tesla itself as the driver.
Well said. I agree up to one point, I know that beta software is also tested on public roads in the industry, but only by trained drivers. As of some project I did in the past, I was drivenin in such a car, when I got a lift by a collegue to the meetings. It was about 15 years ago. From outside it was an old model, but inside the electronics where all new.
Move fast, break stuff (literally)
you forgot, kill people
some of you may die, but it is a sacrifice i'm willing to make.
[deleted]
Good thing it’s not up to you solely
That's a quotation from Lord Farquad, the villain in Shrek.
It is not up to me at all.
As long as the accident rate isn't worse than human driving, I only see this as a good thing towards future safety.
“Fake Cities” don’t have nearly enough complexity.
I agree they could start there, but you’ll graduate quickly without having learned much.
The main question is, are we willing to put people in harms way today for the benefit of future humans? The answer seems pretty obvious to me. Drunk humans are considerably worse than this and are not going away anytime soon. If we can solve self driving just 1 year earlier it’s equivalent to saving 30,000 American lives.
Put another way, if you want rules that delay the advancement of self driving driving cars, you're effectively murdering 30,000+ American lives every year.
I agree they could start there, but you’ll graduate quickly without having learned much.
The main question is, are we willing to put people in harms way today for the benefit of future humans? The answer seems pretty obvious to me. Drunk humans are considerably worse than this and are not going away anytime soon. If we can solve self driving just 1 year earlier it’s equivalent to saving 30,000 American lives.
Put another way, if you want rules that delay the advancement of self driving driving cars, you're effectively murdering 30,000+ American lives every year.
> If we can solve self driving just 1 year earlier it’s equivalent to saving 30,000 American lives.
That strawman argument works if you completely ban all human drivers the moment we solve self-driving.
The big question is, when exactly do we consider self driving solved that it can ban replace drivers? All current evidence points it's very, very far away, if ever.
That strawman argument works if you completely ban all human drivers the moment we solve self-driving.
The big question is, when exactly do we consider self driving solved that it can ban replace drivers? All current evidence points it's very, very far away, if ever.
Uh... You'll never stop running into problems that require a driver to take conttol. Automation is only as reliable as the sum of it's parts. Can't wait to see the first set of failures that prevent automated driving back of defective units under their own power without a human backup option.
>You'll never stop running into problems that require a driver to take conttol.
That's pretty much the nail in the FSD coffin.
That's pretty much the nail in the FSD coffin.
There's no guarantee that computers will end up being any better at driving than humans.
I'm not trying to defend self driving to Tesla here, like, at all, but I don't agree with this statement.
Evolution has gifted us with a phenomenal device, the brain, but evolution is extremely cautious and conservative. There is very little reason to expect that human innovation won't catch up and surpass evolution. 10,000 years from now, I expect self-driving will work perfect. 10,000 years from now, our brains will (unless modified by our technology) largely be unchanged.
Evolution has gifted us with a phenomenal device, the brain, but evolution is extremely cautious and conservative. There is very little reason to expect that human innovation won't catch up and surpass evolution. 10,000 years from now, I expect self-driving will work perfect. 10,000 years from now, our brains will (unless modified by our technology) largely be unchanged.
Every day you slow down FSD development with this kind of safetyism, a hundred people die in car accidents (in the USA).
If safety is your #1 goal, you should be advocating for busses and trains. Those are infinitely safer than cars, self-driving or otherwise.
[edit] Transit is 10x safer than private cars. [1]
> The effects extend beyond individual trip choices, too: the report notes that transit-oriented communities are five times safer than auto-oriented communities. Better public transportation contributes to more compact development, which in turn reduces auto-miles traveled and produces safer speeds in those areas. On a national scale, too, the U.S. could make large advances in safety if each American committed to replacing as few as three car trips per month with transit trips.
[1] https://mobilitylab.org/2016/09/08/transit-10-times-safer-dr...
[edit] Transit is 10x safer than private cars. [1]
> The effects extend beyond individual trip choices, too: the report notes that transit-oriented communities are five times safer than auto-oriented communities. Better public transportation contributes to more compact development, which in turn reduces auto-miles traveled and produces safer speeds in those areas. On a national scale, too, the U.S. could make large advances in safety if each American committed to replacing as few as three car trips per month with transit trips.
[1] https://mobilitylab.org/2016/09/08/transit-10-times-safer-dr...
Busses and trains are safer than cars, but certainly not infinitely so. Nonetheless the infrastructure we have isn't built for them, and that won't change any time soon. If you want a suburban house with a yard, you need a passenger car. If you want to pick blackberries at the local farm, you need a passenger car. Making passenger cars safer through autonomy is clearly a good thing.
By all means, advocate for more transit friendly urban centers. I'm with you. Just don't take away autonomy out of spite. Better cars are still better, even if they're not the solution you want.
By all means, advocate for more transit friendly urban centers. I'm with you. Just don't take away autonomy out of spite. Better cars are still better, even if they're not the solution you want.
> Making passenger cars safer through autonomy is clearly a good thing.
I'd actually disagree with this stance. Making passenger cars safer through autonomy is probably a good thing if we can actually make it safer than human drivers. I've yet to be convinced we are anywhere close to meeting the bar on that if. I assume we will eventually, but I'm not even sure I'll live to see it.
It also ignores potential knock on effects, sure in isolation safer cars are better, but the reality is nothing exists in isolation. Could we save more lives if instead of spending the money we are on self-driving cars we instead invested it into our transit systems?
As an example of knock on effects, affordable cars feels like an easy win right? Makes travel easier for everyone. But by and large affordable cars are what has allowed suburbs to exist, but there's an argument to be made that urban sprawl is far from ideal and that we'd be better of with denser communities and public transit.
I'd actually disagree with this stance. Making passenger cars safer through autonomy is probably a good thing if we can actually make it safer than human drivers. I've yet to be convinced we are anywhere close to meeting the bar on that if. I assume we will eventually, but I'm not even sure I'll live to see it.
It also ignores potential knock on effects, sure in isolation safer cars are better, but the reality is nothing exists in isolation. Could we save more lives if instead of spending the money we are on self-driving cars we instead invested it into our transit systems?
As an example of knock on effects, affordable cars feels like an easy win right? Makes travel easier for everyone. But by and large affordable cars are what has allowed suburbs to exist, but there's an argument to be made that urban sprawl is far from ideal and that we'd be better of with denser communities and public transit.
> I've yet to be convinced we are anywhere close to meeting the bar on that if.
What would convince you? Data from 60k cars isn't sufficient?
What would convince you? Data from 60k cars isn't sufficient?
> What would convince you? Data from 60k cars isn't sufficient?
It would be if the data showed they were safer than human drivers, and was independently obtained. I have yet to see any data that suggests this or anything close to this.
It would be if the data showed they were safer than human drivers, and was independently obtained. I have yet to see any data that suggests this or anything close to this.
The Tesla data shows that they are less safe than regular drivers
Uh... no? I suspect you're referring to the Goodall preprint that did the rounds a few days ago. What it purported[1] to show was not that AP was less safe than regular driver, but that it was less safe than Tesla claimed. It still showed that it was (moderately) safer than Teslas being driven without active safety measures, which are themselves about 3x safer than average vehicle.
You seem to have taken the opposite conclusion, which is exactly what the feeding frenzy over the paper wanted.
[1] The methodology is hugely suspect: you can't take an incomplete data set and then just "correct" it by inventing axes that you pull in from other incomplete data sets that weren't studied or measured in the original! That's rank P-Hacking. It seems reasonable, but I guarantee that a talented statistician can push any such data set 2x in either direction with that kind of trick.
You seem to have taken the opposite conclusion, which is exactly what the feeding frenzy over the paper wanted.
[1] The methodology is hugely suspect: you can't take an incomplete data set and then just "correct" it by inventing axes that you pull in from other incomplete data sets that weren't studied or measured in the original! That's rank P-Hacking. It seems reasonable, but I guarantee that a talented statistician can push any such data set 2x in either direction with that kind of trick.
> Just don't take away autonomy out of spite.
It's not a question of taking away autonomy, and there's certainly no spite about it. If you want to get around town, you have bikes or e-bikes. If you want to get out of urban centers, you can always rent a car at the periphery.
I've lived in SF for 10 years with no car and have never felt unable to do anything I've wanted at any time.
> If you want to pick blackberries at the local farm, you need a passenger car.
Not really, the farm can have a bus with regularly scheduled pick-ups or routes like a lot of the Napa wineries do.
It's not a question of taking away autonomy, and there's certainly no spite about it. If you want to get around town, you have bikes or e-bikes. If you want to get out of urban centers, you can always rent a car at the periphery.
I've lived in SF for 10 years with no car and have never felt unable to do anything I've wanted at any time.
> If you want to pick blackberries at the local farm, you need a passenger car.
Not really, the farm can have a bus with regularly scheduled pick-ups or routes like a lot of the Napa wineries do.
You seem to be ignoring the last and most crucial point:
> If they want to test it out on public roads, run the analysis and look at whenever it diverges from the drivers decisionmaking instead.
It would be trivial to analyze the data after the fact to see where the AI model diverges from the human driver’s actions, decide which one was right, then implement the correct action. That wouldn’t slow down testing at all, as the only difference is who’s controlling the vehicle. In any beta test, someone still has to analyze the data.
> If they want to test it out on public roads, run the analysis and look at whenever it diverges from the drivers decisionmaking instead.
It would be trivial to analyze the data after the fact to see where the AI model diverges from the human driver’s actions, decide which one was right, then implement the correct action. That wouldn’t slow down testing at all, as the only difference is who’s controlling the vehicle. In any beta test, someone still has to analyze the data.
So, for the sake of progress, we should let FSD also kill people because people die at the wheel anyway?
How about we measure and see if FSD is killing people at all first? It's not an unanswerable problem, after all. There are 60k+ of these devices on the roads now. If the statistics say it's unsafe, pull it and shut the program down.
Do they? If they don't, would you admit that it's probably a good thing to let it continue?
Do they? If they don't, would you admit that it's probably a good thing to let it continue?
You can't measure that without stopping all Beta testers' interventions, which implies allowing the system to actually kill people.
>How about we measure and see if FSD is killing people at all first?
We just saw a FSD car run a red light and nearly hit pedestrians if the driver hadn't intervened.
We just saw a FSD car run a red light and nearly hit pedestrians if the driver hadn't intervened.
Exactly. Let's measure. Is that rate higher than seen by median cars in the environment? I'd argue no, given how distressingly common that kind of incident is (certainly it's happened to me a bunch of times). But I'm willing to see data.
I think where you're going here is toward an assertion that "any failure at all is unacceptable". And that seems innumerate to me. Cars fail. We're trying to replace a system here that is already fatally (literally!) flawed. The bar is very low.
I think where you're going here is toward an assertion that "any failure at all is unacceptable". And that seems innumerate to me. Cars fail. We're trying to replace a system here that is already fatally (literally!) flawed. The bar is very low.
>I think where you're going here is toward an assertion that "any failure at all is unacceptable". We're trying to replace a system here that is already fatally (literally!) flawed. The bar is very low.
Failure is not the issue when it comes to Tesla FSD, accountability is.
For any mistakes human drivers makes, they have to pay up with money, have their license suspended, or with jail time, depending on the severity of their mistake.
You fuck up, you pay the price. That's the contract under which human drivers are allowed on the road. Humans drivers are indeed flawed, but with our law and justice systems, we have accountability to keep those who break the law in check, while allowing freedom for those who respect it. It's one of the pillars of any civilized society.
In my country, running a stop sign or a red light means you get your license suspended for a while. When a self driving Tesla does the same mistake, why doesn't Tesla's FSD AI have its "license" suspended as well? That's the issue.
Failure is not the issue when it comes to Tesla FSD, accountability is.
For any mistakes human drivers makes, they have to pay up with money, have their license suspended, or with jail time, depending on the severity of their mistake.
You fuck up, you pay the price. That's the contract under which human drivers are allowed on the road. Humans drivers are indeed flawed, but with our law and justice systems, we have accountability to keep those who break the law in check, while allowing freedom for those who respect it. It's one of the pillars of any civilized society.
In my country, running a stop sign or a red light means you get your license suspended for a while. When a self driving Tesla does the same mistake, why doesn't Tesla's FSD AI have its "license" suspended as well? That's the issue.
I take it you're the person who answers 'neither' when asked do you send the train left and kill 1 person vs sending it right and killing 100.
What's the trolley problem have to do with this situation?
Are there accidents where death is unavoidable? Yes, they happen every single day, but after the investigations and trials are over, the parties found responsible pay up for those deaths in either money or jail-time, or both.
Does that mean we should we allow machines to make deadly mistakes, especially when death IS avoidable? Absolutely not. We sentence humans for such mistakes. Machines (either their operator or their manufacturer) should also have the same liability.
Those are two different things which you're trying to spin into a strawman.
Are there accidents where death is unavoidable? Yes, they happen every single day, but after the investigations and trials are over, the parties found responsible pay up for those deaths in either money or jail-time, or both.
Does that mean we should we allow machines to make deadly mistakes, especially when death IS avoidable? Absolutely not. We sentence humans for such mistakes. Machines (either their operator or their manufacturer) should also have the same liability.
Those are two different things which you're trying to spin into a strawman.
Even this response is 'neither' :) I love it. Have a good weekend Chuck
/me roundhouse kicks out of the thread
/me roundhouse kicks out of the thread
Right, this is closer to the following scenario:
Let's say you are on an overpass above a train, and a very fat man is in front of you. The train, if it isn't stopped, will kill 10 people on the tracks. But, if you push the person in front of the train, it will kill 11 people, and one of those would be you committing homocide.
What do you do?
Let's say you are on an overpass above a train, and a very fat man is in front of you. The train, if it isn't stopped, will kill 10 people on the tracks. But, if you push the person in front of the train, it will kill 11 people, and one of those would be you committing homocide.
What do you do?
Yes, exactly!
Even if it means that a loved one of yours would get run down by one of these, it's ok in the end, because it helped improve some billionaire's beta tech-demo?
FSD isn’t a monolith, where speeding up one company gets us to the goal faster. We don’t even know if it’s possible with current tech, let alone with just cameras. Slowing down tesla might be just making a dead end safer. We don’t know, which is why we need safety standards.
That presupposes that FSD is some major societal advancement.
Exactly! I expect to see nothing but negativity here regarding this.
Making intelligence out of silicon isn't easy. Let the computers learn this way during the transition period, and finally we can remove human drivers from the road.
More than 38,000 people die every year in crashes on U.S. roadways. And Tesla makes the safest cars:
The Insurance Institute for Highway Safety (IIHS), the main independent organization that conducts crash tests on vehicles in the US, released the result of its latest tests on the Tesla Model Y and confirmed that it achieved the highest possible safety rating.
Making intelligence out of silicon isn't easy. Let the computers learn this way during the transition period, and finally we can remove human drivers from the road.
More than 38,000 people die every year in crashes on U.S. roadways. And Tesla makes the safest cars:
The Insurance Institute for Highway Safety (IIHS), the main independent organization that conducts crash tests on vehicles in the US, released the result of its latest tests on the Tesla Model Y and confirmed that it achieved the highest possible safety rating.
>Tesla makes the safest cars:
>The Insurance Institute for Highway Safety (IIHS), the main independent organization that conducts crash tests on vehicles in the US, released the result of its latest tests on the Tesla Model Y and confirmed that it achieved the highest possible safety rating.
That doesn't mean that Telsa makes the safest cars. There are roughly ~100 cars with that rating, and nothing suggests the Model Y is safer than any of the others. It's also important to note that that rating isn't based of real world data such as how often drivers actually crash and hurt other (ex, how often FSD fails), but rather how well the occupant is protected in the event of a crash.
>The Insurance Institute for Highway Safety (IIHS), the main independent organization that conducts crash tests on vehicles in the US, released the result of its latest tests on the Tesla Model Y and confirmed that it achieved the highest possible safety rating.
That doesn't mean that Telsa makes the safest cars. There are roughly ~100 cars with that rating, and nothing suggests the Model Y is safer than any of the others. It's also important to note that that rating isn't based of real world data such as how often drivers actually crash and hurt other (ex, how often FSD fails), but rather how well the occupant is protected in the event of a crash.
Making a car that has the highest crash safety rating has nothing to do with the safety of their FSD solution.
It is faster this way. And anyway it would be impossible to simulate the real world scenarios in the synthetic environment. One of the potential benefits of FSD is that it will save lives, hence should we go slow about it or take a reasonable risk and get it done. There is a risk in going slow too.
I find it hard to believe Tesla has remotely exhausted their "dry-run" training options considering how much like a drunken 8 year old their cars behave on FSD.
The FSD AI shouldn't be connected to the real-world controls until it very rarely substantially deviates from what the human drivers do in their cars while controlling a virtual car from the real-world sensor inputs. And in those cases where it deviates, the devs should manually dig in and run to ground if it would have hit something/someone/broken the law. Not until that list of cases stops growing, particularly in your dense urban areas full of edge cases, do you even start considering linking up the real car.
From what I'm seeing they're instead turning Tesla drivers into training supervisors with the real-world serving as the virtual one, while putting everyone else on/near roads at risk.
It's criminal, and I expect consequences. It's already illegal to let a child steer from your lap, and that's without even handing over the pedals. People operating "FSD Beta" on public roads should be at least as liable, where are the authorities and enforcement?
The FSD AI shouldn't be connected to the real-world controls until it very rarely substantially deviates from what the human drivers do in their cars while controlling a virtual car from the real-world sensor inputs. And in those cases where it deviates, the devs should manually dig in and run to ground if it would have hit something/someone/broken the law. Not until that list of cases stops growing, particularly in your dense urban areas full of edge cases, do you even start considering linking up the real car.
From what I'm seeing they're instead turning Tesla drivers into training supervisors with the real-world serving as the virtual one, while putting everyone else on/near roads at risk.
It's criminal, and I expect consequences. It's already illegal to let a child steer from your lap, and that's without even handing over the pedals. People operating "FSD Beta" on public roads should be at least as liable, where are the authorities and enforcement?
> One of the potential benefits of FSD is that it will save lives
But that doesn’t mean that tesla will save lives. Maybe they do a bunch of this and learn that cameras alone aren’t sufficient, and then waymo wins. Tesla wouldn’t have saved any lives, only killed a couple people unnecessarily.
Medicine is the classic example of applying this kind of thinking. You could do all sorts of unethical medical testing to speed up medical research, saving countless lives down the line, but we don’t because it doesn’t make it right.
With another medical example, you could roll out snake oil without testing it thoroughly because it’ll save lives if it works. But maybe snake oil doesn’t work, and it’ll be some other thing that works, and by rushing the snake oil, you just made things worse.
But that doesn’t mean that tesla will save lives. Maybe they do a bunch of this and learn that cameras alone aren’t sufficient, and then waymo wins. Tesla wouldn’t have saved any lives, only killed a couple people unnecessarily.
Medicine is the classic example of applying this kind of thinking. You could do all sorts of unethical medical testing to speed up medical research, saving countless lives down the line, but we don’t because it doesn’t make it right.
With another medical example, you could roll out snake oil without testing it thoroughly because it’ll save lives if it works. But maybe snake oil doesn’t work, and it’ll be some other thing that works, and by rushing the snake oil, you just made things worse.
>Medicine is the classic example of applying this kind of thinking. You could do all sorts of unethical medical testing to speed up medical research, saving countless lives down the line, but we don’t because it doesn’t make it right.
Bringing up medicine for your stance might back fire. There are tactical examples where policies to "go slow and reduce risk" have cost lives. For example, testing some medicine on pregnant women is bad for the fetus - so there are policies to "not test on anyone who might be pregnant", and as a result, there are few studies on women between the ages of 20 - 80, and women's health has suffered as a result
Bringing up medicine for your stance might back fire. There are tactical examples where policies to "go slow and reduce risk" have cost lives. For example, testing some medicine on pregnant women is bad for the fetus - so there are policies to "not test on anyone who might be pregnant", and as a result, there are few studies on women between the ages of 20 - 80, and women's health has suffered as a result
The point isn’t “go slow.” The points are “this area of ethics is well studied and much more complex than ‘wild west research saves more lives’” as well as “society has rejected this particular form of utilitarianism.”
There's a lot more examples from medicine where "move fast and break stuff" has cost lives and compromised public trust.
Why do you discard the possibility that Tesla will save lives in the long term? You may say it is unlikely, but it is not like Musk did not deliver world scale breakthroughs.
Also, regarding the medicine, do you really believe we do not do "unethical" medical testings? I guess it depends on your ethical standards and how high they are :)
But let's get back to cost benefit trade off. COVID vaccines tests were rushed. So it is obviously sometimes worth it.
There is a risk in not taking a risk.
Also, regarding the medicine, do you really believe we do not do "unethical" medical testings? I guess it depends on your ethical standards and how high they are :)
But let's get back to cost benefit trade off. COVID vaccines tests were rushed. So it is obviously sometimes worth it.
There is a risk in not taking a risk.
>One of the potential benefits of FSD is that it will save lives
When does it plan to start doing that? Safety features in cars have come for decades without trying to kill people first. Just ask Volvo.
The general non-Tesla-owning public, including pedestrians and cyclists, have not given their consent to be part of Elon's public beta test.
When does it plan to start doing that? Safety features in cars have come for decades without trying to kill people first. Just ask Volvo.
The general non-Tesla-owning public, including pedestrians and cyclists, have not given their consent to be part of Elon's public beta test.
> without trying to kill people
Do you suggest that Tesla is trying to kill people? That would be a ridiculous statement.
I bet the risk of getting injured by Tesla beta version of FSD is miniscule compared to a risk of getting into accident caused by a human driver. I am not for banning either of them. Even when we get to the point where FSD is much safer than drivers I would be against of banning humans.
I bet the risk of getting injured by Tesla beta version of FSD is miniscule compared to a risk of getting into accident caused by a human driver. I am not for banning either of them. Even when we get to the point where FSD is much safer than drivers I would be against of banning humans.
>I bet the risk of getting injured by Tesla beta version of FSD is miniscule compared to a risk of getting into accident caused by a human driver.
You can bet all you want, but human drivers, as flawed as they may be, are all fully liable by law for any mistakes they make at the wheel and have to pay with money or jail time plus losing their license.
Who is liable for the mistakes FSD makes? Who goes to jail if it runs down a pedestrian by mistake? Elon? The driver? Can the FSD loose its license like human drivers can for their mistakes?
You can't compare a human drivers to FSD safety when FSD has zero liability in front of the law and all the blame automatically goes to the driver.
You can bet all you want, but human drivers, as flawed as they may be, are all fully liable by law for any mistakes they make at the wheel and have to pay with money or jail time plus losing their license.
Who is liable for the mistakes FSD makes? Who goes to jail if it runs down a pedestrian by mistake? Elon? The driver? Can the FSD loose its license like human drivers can for their mistakes?
You can't compare a human drivers to FSD safety when FSD has zero liability in front of the law and all the blame automatically goes to the driver.
> Who is liable for the mistakes FSD makes? Who goes to jail if it runs down a pedestrian by mistake? Elon? The driver?
Yup. The driver. Aside from image, there appears to be few if any incentives for FSD to improve beyond the “it does the right thing 80% of the time” mark.
Yup. The driver. Aside from image, there appears to be few if any incentives for FSD to improve beyond the “it does the right thing 80% of the time” mark.
> there appears to be few if any incentives for FSD to improve beyond...
I think it is simply not true. Creating a FSD that could replace human drivers eg in case of trucks is potentially highly lucrative and it would make the economy more efficient.
I think it is simply not true. Creating a FSD that could replace human drivers eg in case of trucks is potentially highly lucrative and it would make the economy more efficient.
Perhaps, though, there should be some minimum bar before we allow testing on public roads. The Tesla FSD beta videos I've seen thus far are truly alarming. The system is nowhere near ready for testing in the real world, where it poses significant danger to many innocent bystanders.
> One of the potential benefits of FSD is that it will save lives
Continuing this line of logic, the most advanced FSD would save the most lives. Thus the argument could be made that Tesla should abandon their research and license Waymo FSD.
Continuing this line of logic, the most advanced FSD would save the most lives. Thus the argument could be made that Tesla should abandon their research and license Waymo FSD.
This video shows a lot of other ,,interesting" issues with the current FSD implementation. The collision with the pylon is in fact the least interesting thing.
- 2.20: Taking a left turn with a trajectory into the opposite lane. The youtuber's camera clearly was adjusting exposure from underneath the overpass and I wonder if the car's cameras had the same issues.
- 2.40: Right on red without stopping. I wonder if this is related to the stop sign rolling stop issues
- 3.30: Collision with pylon. It seems like the perception system didn't pick up the pylon at all (or if it did, its not rendering on the screen)
- 5.10: Some weird funky trajectory and then pulling into the sunken curb/sidewalk on the right turn
- 6.28: Trying to drive down the railroad tracks
I gave up after the above.
Despite Tesla's (Elon) insistence that FSD will be safer than human drivers soon enough (1 year), there is very little evidence to support that. At the very least if we are going to allow beta testing on roads then Tesla should be forced to submit their FSD incident data to the CA DMV like the rest of car manufacturers that are testing AVs.
- 2.20: Taking a left turn with a trajectory into the opposite lane. The youtuber's camera clearly was adjusting exposure from underneath the overpass and I wonder if the car's cameras had the same issues.
- 2.40: Right on red without stopping. I wonder if this is related to the stop sign rolling stop issues
- 3.30: Collision with pylon. It seems like the perception system didn't pick up the pylon at all (or if it did, its not rendering on the screen)
- 5.10: Some weird funky trajectory and then pulling into the sunken curb/sidewalk on the right turn
- 6.28: Trying to drive down the railroad tracks
I gave up after the above.
Despite Tesla's (Elon) insistence that FSD will be safer than human drivers soon enough (1 year), there is very little evidence to support that. At the very least if we are going to allow beta testing on roads then Tesla should be forced to submit their FSD incident data to the CA DMV like the rest of car manufacturers that are testing AVs.
2:20 is wild -- the street it's turning onto is clearly marked as a two-way in Google Maps [1] (which I understand provides the underlying data to Teslas). Why would the car think, no, it's best I turn onto the left side of this road?
I'm also in awe of how fickle its predicted path is, especially when turning. Why commit to a new path for a tenth of a second and then change its mind again? This suggests to me it's placing way too high weight on poor quality telemetry. Heck, at 5:14 it can't even decide which street it wants to go down. Maybe it's just a UI thing (always displays the most likely path) and the underlying model is actually keeping both options open.
At 6:45 it tries some insane passing maneuver -- I can't get over that one. Passing on the right, in a bus-only lane, planning to squeeze through the rapidly-closing gap of the car in front of it and a parked car, where there is currently a bicyclist. And it looks like the bicyclist is the only thing keeping it from doing that insane maneuver -- each time the bicyclist is occluded by the car in front (object permanence, hello?), it attempts to pass.
I've lost count of the number of things this car doesn't understand, that it should to be on a road:
* object permanence
* reading basic text in simple print
* logical connections between the map and road geometry
* patience and commitment
* understanding physics and geometry of other moving and non-moving vehicles
* seeing stationary bright orange objects immediately in front of it
[1] https://goo.gl/maps/EQfUnPrN52d9TVKPA
I'm also in awe of how fickle its predicted path is, especially when turning. Why commit to a new path for a tenth of a second and then change its mind again? This suggests to me it's placing way too high weight on poor quality telemetry. Heck, at 5:14 it can't even decide which street it wants to go down. Maybe it's just a UI thing (always displays the most likely path) and the underlying model is actually keeping both options open.
At 6:45 it tries some insane passing maneuver -- I can't get over that one. Passing on the right, in a bus-only lane, planning to squeeze through the rapidly-closing gap of the car in front of it and a parked car, where there is currently a bicyclist. And it looks like the bicyclist is the only thing keeping it from doing that insane maneuver -- each time the bicyclist is occluded by the car in front (object permanence, hello?), it attempts to pass.
I've lost count of the number of things this car doesn't understand, that it should to be on a road:
* object permanence
* reading basic text in simple print
* logical connections between the map and road geometry
* patience and commitment
* understanding physics and geometry of other moving and non-moving vehicles
* seeing stationary bright orange objects immediately in front of it
[1] https://goo.gl/maps/EQfUnPrN52d9TVKPA
AFAIK Tesla doesn't rely on map data for the driving itself, only for routing. Which sounds like the right approach considering roadworks and how often maps are out of date.
Yes, this is very far from average human level performance. Error rates need to go down by maybe 100-1000x to be viable. They are years away.
As a foreigner, can you explain at 2:40, which light is red? I can only see a green light on the right side, there was no traffic light on the road it came through. Also doesn't the US allow right turns on red?
I would totally drive on those train tracks myself.. it's pretty common here, and there was no signage. Curious to see what the AI would do next if the driver hadn't intervened.
I would totally drive on those train tracks myself.. it's pretty common here, and there was no signage. Curious to see what the AI would do next if the driver hadn't intervened.
You still have to come to a stop before turning right on red.
> - 3.30: Collision with pylon. It seems like the perception system didn't pick up the pylon at all (or if it did, its not rendering on the screen)
Wonder if LIDAR would have spotted the pylon? It sounds like neat tech.
Wonder if LIDAR would have spotted the pylon? It sounds like neat tech.
> that FSD will be safer than human drivers soon enough (1 year)
maybe while parked…
maybe while parked…
Wow, thanks for the timestamps. This wouldn't even pass a driving test.
I've owned 3 teslas. On each of them I have chosen not to pre-pay $10,000 CAD in order to get access to FSD when it is available. 2 of those cars I owned and sold before FSD was even available.
I would not have had the ability to take that license with me to the new car I purchased. I'm surprised this practice hasn't been investigated by any state justice or commerce dept (I'm not sure which it should be in the US?) as many consumers were sold something that was never provided.
I would not have had the ability to take that license with me to the new car I purchased. I'm surprised this practice hasn't been investigated by any state justice or commerce dept (I'm not sure which it should be in the US?) as many consumers were sold something that was never provided.
> I'm surprised this practice hasn't been investigated by any state justice or commerce dept ...
I'm not sure there's a way to regulate consumer stupidity. People are conned into losing money everyday, not all of those cons are crimes.
I'm not sure there's a way to regulate consumer stupidity. People are conned into losing money everyday, not all of those cons are crimes.
Many types of fraud only exist because people are "stupid". They're still illegal.
All customers are stupid relative to the manufacturer/retailer because they're not privy to internal company information regarding the product they're buying. That's why consumer protection laws exist.
I agree.
However, consumers putting down $10,000 for a promised feature with no firm deadline and severe restrictions on the said feature's portability are in a league of their own when it comes to stupidity.
Or, perhaps they are more akin to an Investor ..
However, consumers putting down $10,000 for a promised feature with no firm deadline and severe restrictions on the said feature's portability are in a league of their own when it comes to stupidity.
Or, perhaps they are more akin to an Investor ..
Consumer stupidity is one of the key elements of many frauds.
That sounds like a cleptocracy IMHO
Seems like the license might not even transfer with the car to a new owner[1]. Feels like a really expensive Kickstarter.
[1] https://www.findmyelectric.com/blog/does-full-self-driving-f...
[1] https://www.findmyelectric.com/blog/does-full-self-driving-f...
Might not? It absolutely does not.
So the FDS feature is tied to the car, rather than your personal account with Tesla? In that case, is it possible to transfer the deposit for FSD to the new Tesla owner? If so then presumably you could choose to reflect that in the price you choose to sell the car.
It's actually worse than that. The FSD is tied to your account and the car, i.e. if you sell the car to a new owner they will have to purchase FSD separately and if you buy a new Tesla you cannot transfer FSD from the old car to the new one.
That's not really true. If you sell your Tesla with FSD beta directly to me, I'll keep the FSD beta. If you trade it in to Tesla, then they might decide to remove the package and resell the car without it but I don't see how that would be your concern.
yes. It is probably one of the worst values in the entire automotive industry.
They always have this "buy now, the price will go up later!" but I had to ask myself: how much can the price go up? It's not going to be a $20,000 feature any time soon. I figured I was risking a couple thousand dollars that I would need to pay to upgrade later.
They always have this "buy now, the price will go up later!" but I had to ask myself: how much can the price go up? It's not going to be a $20,000 feature any time soon. I figured I was risking a couple thousand dollars that I would need to pay to upgrade later.
If Tesla eventually solves FSD (which is not a given) they might stop selling the ad-on package altogether and just charge a per mile subscription instead.
Then all of these old Teslas with FSD for life will become rare, similar to the Teslas witch unlimited free supercharging.
Then all of these old Teslas with FSD for life will become rare, similar to the Teslas witch unlimited free supercharging.
There really is a sucker born every minute.
While true, on a much smaller scale I experienced the same benefit from being grandfathered in to the old Blink for Home security cameras. I have full service for free for life, as opposed to $10/month. A giant online stink was raised to keep Amazon to Blink's word after the acquisition, but it worked.
Yes, but in that case you've actually bought a service that already exists, not vaporware that will never exist.
Tesla has been removing the features upon resale. So these features are really just a license you purchase to use while you own the car.
I am a Tesla owner, and yes, I regret the purchase of FSD. I did and still do consider it a bit of an investment in a future I'd like to see, and I'm quite happy I also _actually invested_ in Tesla when I bought my model 3, but over the last 4 years I've learned something that has changed my mind considerably:
AI does not step forward progressively. Normally, engineers spend ages improving things, testing, improving again. With AI, it's far more of a "let's just re-train this black-box on a totally different set of data, wind up with a totally different black-box, and see if its better!" The difference in driving quality and intelligence between even minor version updates is dramatic. I very intentionally _do not_ update my Tesla until the release is ~6 months old. This... sucks.
I am still very bullish on Tesla and EVs, and on technology in general - unlike many I am not full of hate that this didn't work out, nor do I particularly think FSD is _dramatically_ more dangerous behind the wheel than say my grand father was before we convinced him to stop driving, but I do think I have more or less zero patience for AI engineers these days. The failure by Google and others to deliver even a moderately usable voice assistant is all the proof I need that FSD is _years_ away.
AI does not step forward progressively. Normally, engineers spend ages improving things, testing, improving again. With AI, it's far more of a "let's just re-train this black-box on a totally different set of data, wind up with a totally different black-box, and see if its better!" The difference in driving quality and intelligence between even minor version updates is dramatic. I very intentionally _do not_ update my Tesla until the release is ~6 months old. This... sucks.
I am still very bullish on Tesla and EVs, and on technology in general - unlike many I am not full of hate that this didn't work out, nor do I particularly think FSD is _dramatically_ more dangerous behind the wheel than say my grand father was before we convinced him to stop driving, but I do think I have more or less zero patience for AI engineers these days. The failure by Google and others to deliver even a moderately usable voice assistant is all the proof I need that FSD is _years_ away.
Yeah, I think most people who haven't worked with machine learning may not realize this, and it's extremely important. It can be pretty easy to reach 90%, but impossible to reach 95%. People tend to assume some sort of progression, where you just "iron out" the remaining quirks, but that only applies to traditional engineering. You can gather 100x the training data and build a model with 100x the number of nodes and get a 10% average improvement with some cases getting worse and others getting better.
I think the current state of FSD is extremely impressive. There are clearly some extremely talented engineers working on that system. But my guess would be that it simply can't be incrementally improved until it's road-safe, in much the same way that GPT-3 is insanely impressive but can't be incrementally improved until it can write a whole human-quality novel. I suspect a new wave of radical rethinking of machine learning might be necessary, and that might be decades away.
I think the current state of FSD is extremely impressive. There are clearly some extremely talented engineers working on that system. But my guess would be that it simply can't be incrementally improved until it's road-safe, in much the same way that GPT-3 is insanely impressive but can't be incrementally improved until it can write a whole human-quality novel. I suspect a new wave of radical rethinking of machine learning might be necessary, and that might be decades away.
A century from now, Musk’s ability to get to people to PAY to beta test with their lives will be studied in business schools across the world.
An additional problem is that pedestrians, cyclists, etc. also pay for it.
And taxpayers?
Sad truth.
It's already documented:
https://www.youtube.com/watch?v=Dkix4UzEbjU
https://www.youtube.com/watch?v=Dkix4UzEbjU
It'll be interesting looking back whether the Musk FSD stuff cost lives or saved them. I'm guessing saved but I we'll see.
They should start studying him right now.
> They should start studying him right now.
Not only Musk. I would love to see a brain imaging study of humans as their net worth gains orders of magnitude.
I have seen and felt the effects myself to a very limited extent.
I have heard Elon and others say in regards to the internet: "We are not evolved to handle this much information, and this much choice."
My intuition leads me to believe that the same could be said for having an absurdly large net worth. We just aren't evolved for it and one's behavior changes in weird ways.
disclaimer: While I am pointing fingers, it is at our entire species, not "billionaires."
Not only Musk. I would love to see a brain imaging study of humans as their net worth gains orders of magnitude.
I have seen and felt the effects myself to a very limited extent.
I have heard Elon and others say in regards to the internet: "We are not evolved to handle this much information, and this much choice."
My intuition leads me to believe that the same could be said for having an absurdly large net worth. We just aren't evolved for it and one's behavior changes in weird ways.
disclaimer: While I am pointing fingers, it is at our entire species, not "billionaires."
The removal of radar has got to be a hindrance to getting autonomous driving working well for Tesla. Cross-comparison between multiple forms of environment surveillance (Vision, Radar, LIDAR, etc.) has to be in place for autonomous driving to work reliably IMO. This much has been stated by multiple non-Tesla AI organizations and leaders.
I think Tesla may be using the price of FSD as a way to dissuade people from buying it, while enabling it to be a funding mechanism for continued work on autonomous driving.
As evident in this video, FSD is often acting like a nervous student driver by being unable to confidently make decisions on the path to take, running through red lights, driving unpredictably at slow speeds, and more. This is probably due to the fact that there is ONLY cameras being used to inform FSD, which can suffer from contrast issues.
I've seen videos in the past where Tesla camera input is in black/white/grayscale for processing. It seems like if you converted the video to B&W, the pylon and the road are similar darkness/color, so I'm not surprised this had issues. Tesla Autopilot was suffering from issues with contrast all the way back in 2016 when Autopilot failed to detect a semi crossing a highway in Florida which lead to the death of the Tesla driver. This was due to the white trailer against a bright sky.
Ultimately, as a consumer and former Tesla owner, I don't feel confident in Tesla's ability to get autonomous driving working well at a human-capable level, let alone the "10x safer" bar they've set for themselves.
I think Tesla may be using the price of FSD as a way to dissuade people from buying it, while enabling it to be a funding mechanism for continued work on autonomous driving.
As evident in this video, FSD is often acting like a nervous student driver by being unable to confidently make decisions on the path to take, running through red lights, driving unpredictably at slow speeds, and more. This is probably due to the fact that there is ONLY cameras being used to inform FSD, which can suffer from contrast issues.
I've seen videos in the past where Tesla camera input is in black/white/grayscale for processing. It seems like if you converted the video to B&W, the pylon and the road are similar darkness/color, so I'm not surprised this had issues. Tesla Autopilot was suffering from issues with contrast all the way back in 2016 when Autopilot failed to detect a semi crossing a highway in Florida which lead to the death of the Tesla driver. This was due to the white trailer against a bright sky.
Ultimately, as a consumer and former Tesla owner, I don't feel confident in Tesla's ability to get autonomous driving working well at a human-capable level, let alone the "10x safer" bar they've set for themselves.
Crash starts from the 3:26 mark: [0]. Had they not intervened, the damage would have been much worse. Further in the video, it has repeatedly become confused in trying to drive through the pylons - again the driver intervenes.
This is just one of the many reasons why FSD is not only putting drivers at risk, but it is really turning its customers into real-life crash dummies.
A more accurate re-brand of this product should be something more like 'Fools Self Driving' than 'Full Self Driving', given the false advertising and lack of safety features for these drivers using the product.
[0] https://youtu.be/sbSDsbDQjSU?t=206
This is just one of the many reasons why FSD is not only putting drivers at risk, but it is really turning its customers into real-life crash dummies.
A more accurate re-brand of this product should be something more like 'Fools Self Driving' than 'Full Self Driving', given the false advertising and lack of safety features for these drivers using the product.
[0] https://youtu.be/sbSDsbDQjSU?t=206
Hitting a bollard is inexcusable. Even for a vision-only system. They're really lucky that it wasn't a solid bollard. New York City is replacing plastic bike lane bollards with steel ones because cars and trucks weren't taking the plastic ones seriously.[1]
There's another place where it tries to turn onto the light rail trackway and then ends up in the bus lane, but that area of First Street in San Jose is confusing.
Tesla used to make a lot of PR about how they were training their AI collecting data from vehicles in the field, although nobody seems to have tapped in and found out what they're collecting. You'd think that if they did that, they'd log all the places where there was trouble or the human overrode the system. Then transmit cautions to cars getting close to trouble spots, so the fleet as a whole never made the same mistake twice. Apparently not.
[1] https://nyc.streetsblog.org/2021/12/21/incoming-dot-commissi...
There's another place where it tries to turn onto the light rail trackway and then ends up in the bus lane, but that area of First Street in San Jose is confusing.
Tesla used to make a lot of PR about how they were training their AI collecting data from vehicles in the field, although nobody seems to have tapped in and found out what they're collecting. You'd think that if they did that, they'd log all the places where there was trouble or the human overrode the system. Then transmit cautions to cars getting close to trouble spots, so the fleet as a whole never made the same mistake twice. Apparently not.
[1] https://nyc.streetsblog.org/2021/12/21/incoming-dot-commissi...
Interestingly at the beginning of the video the route planning seems to work overtime to avoid making a specific turn. See about 25s in: https://youtu.be/sbSDsbDQjSU?t=25
That's kind of like what you're describing, but more dumb where the car tries to avoid the trouble spot entirely.
That's kind of like what you're describing, but more dumb where the car tries to avoid the trouble spot entirely.
> it is really turning its customers into real-life crash dummies.
Not just the customers, anybody else unfortunate enough to share the road with them.
As a frequent pedestrian/bicyclist in my city, Teslas scare the shit out of me. Even worse, when Tesla drivers cause accidents, the response is typically complete shock and disbelief that FSD failed them. And an utter lack of responsibility for the damage and/or injury they caused. Because obviously they weren't driving, it was the car!
Not just the customers, anybody else unfortunate enough to share the road with them.
As a frequent pedestrian/bicyclist in my city, Teslas scare the shit out of me. Even worse, when Tesla drivers cause accidents, the response is typically complete shock and disbelief that FSD failed them. And an utter lack of responsibility for the damage and/or injury they caused. Because obviously they weren't driving, it was the car!
Sanhedrin 91b applies: https://www.sefaria.org/Sanhedrin.91b?lang=bi
Both Elon Musk and the driver are equally at fault and need both go to jail, they enable each other to make the public roadways unsafe.
Both Elon Musk and the driver are equally at fault and need both go to jail, they enable each other to make the public roadways unsafe.
Why do you feel like you have to bring Judaism / religion into this conversation?
The Talmud explains Jewish legal concepts, how they were used and applied in practice. It's not an attempt to drag religion in, rather to point out that 1500 years ago the Jewish Sages recognized that folks who enable each other are both guilty and need to be judged together. The plausible denability and diffusion of responsibility that you see so often nowadays is pernicious to society.
Huh, I don’t recall seeing the bit in the law about religion being involved in the US legal system.
Seriously injecting religion by posting irrelevant scripture in a discussion of legal liability is purely proselytizing and an attempt to convert people.
Seriously injecting religion by posting irrelevant scripture in a discussion of legal liability is purely proselytizing and an attempt to convert people.
Cut it. Citing a principle from old scripture is not proselytizing. Maybe you don't like reading old stories, but some others do. I enjoyed the read and think it's relevant to the discussion.
We can choose how to look at the shared responsibility. No reason to restrict discussion to current legal doctrine.
We can choose how to look at the shared responsibility. No reason to restrict discussion to current legal doctrine.
I would add that very likely that text, like many similar ones, are proto-stories, not unlike the judgement of Solomon:
https://en.wikipedia.org/wiki/Judgement_of_Solomon
I wouldn't be surprised if a same or similar story is present in other cultures.
https://en.wikipedia.org/wiki/Judgement_of_Solomon
I wouldn't be surprised if a same or similar story is present in other cultures.
It's interesting how those green pylons were completely invisible to the car's sensor. If you look at dashboard LCD 4:30, the car seems too slow to register them as objects or misses it completely.
4:42 it makes a right onto tram track. And again at 6:24
Jesus.
4:42 it makes a right onto tram track. And again at 6:24
Jesus.
As a human, I had a hard time seeing the green pylons when watching the video. Local to me pylons are always bright orange and easy to see. My brain is not looking for dark colored narrow barriers so it doesn’t surprise me that FSD doesn’t see them either. I imagine I’d have a hard time driving those roads in bad weather or at night and maybe even under the lighting during the video.
The bollards are plainly visible in broad daylight, and wrapped with reflective tape. If you cannot see them, please stay off the road.
I think there's a huge difference between spotting them in a 2d video and spotting them live. Your brain is very good at picking out details like motion and parallax in a live environment.
The car doesn't even see an orange pylon, with bright reflective tape, at 7:24. Or for that matter, the lane markings -- after the driver pulls the car back into the lane, in continues to try driving through the parking spaces and shoulder.
This car can't even see orange pylons and lane markings. That's not even passable for a student driver on a closed course.
This car can't even see orange pylons and lane markings. That's not even passable for a student driver on a closed course.
And a great example why self driving needs more sensors than just vision. Otherwise it will keep making many of the same errors humans will. Someone is going to wipe the floor with Tesla simply by having a system that avoids collisions, rather than avoids collisions with things that can be seen.
> As a human, I had a hard time seeing the green pylons when watching the video.
I'm not doubting you, but I find that surprising. Admittedly the dark green blends into the background a bit, but the retroreflectors catching the sunlight stood out for me even in the video, which would have muted the effect somewhat.
> I imagine I’d have a hard time driving those roads in bad weather or at night and maybe even under the lighting during the video.
In bad weather or at night you should have you headlights on and those retroreflectors will make them shine brightly. They'll actually probably be easier to spot. In the event the weather is bad enough the retroreflectors can't bounce your headlights back at you, it's going to be bad enough your not seeing the paint either.
I'm not doubting you, but I find that surprising. Admittedly the dark green blends into the background a bit, but the retroreflectors catching the sunlight stood out for me even in the video, which would have muted the effect somewhat.
> I imagine I’d have a hard time driving those roads in bad weather or at night and maybe even under the lighting during the video.
In bad weather or at night you should have you headlights on and those retroreflectors will make them shine brightly. They'll actually probably be easier to spot. In the event the weather is bad enough the retroreflectors can't bounce your headlights back at you, it's going to be bad enough your not seeing the paint either.
It didn't see the orange one later in the video either.
The driver is really alert and intervenes super quickly. I don't think we can expect regular drivers to intervene this quickly.
Tesla has been full of it on FSD for a while now, at least the current beta is rolled out to a much more restricted set of people, although they still sell the general one to the general public...
But WHY is a (I'm assuming) concrete post that short and painted DARK GREEN? Green seems like a horrible choice for a post in night driving. That's not every particularly well designed for human driving. San Jose? Why isn't this post yellow? Yellow seems to be the typical concrete post color.
Surface/local street FSD driving is 10-20 years away and will require a lot of convergence, construction standards, regulatory testing, common databases of "danger locations", etc. It's so much harder. So many "target types" so much variation, so many different local standards.
In particular I don't agree with Tesla's lack of a database or route-specific training or the like. Lidar I think is actually less of an issue than that approach.
But a good divided highway automation should be doable. That has a lot more standardization, and higher amounts of traffic would lead to better statistical models and datasets for proving out algorithms.
These are the problems in highway driving: traffic jams (can be prewarned)/ stopped cars (a current tesla issue), deer/animals wandering onto the road, construction (can be signaled in a standard way and prewarned). Debris/potholes?
But WHY is a (I'm assuming) concrete post that short and painted DARK GREEN? Green seems like a horrible choice for a post in night driving. That's not every particularly well designed for human driving. San Jose? Why isn't this post yellow? Yellow seems to be the typical concrete post color.
Surface/local street FSD driving is 10-20 years away and will require a lot of convergence, construction standards, regulatory testing, common databases of "danger locations", etc. It's so much harder. So many "target types" so much variation, so many different local standards.
In particular I don't agree with Tesla's lack of a database or route-specific training or the like. Lidar I think is actually less of an issue than that approach.
But a good divided highway automation should be doable. That has a lot more standardization, and higher amounts of traffic would lead to better statistical models and datasets for proving out algorithms.
These are the problems in highway driving: traffic jams (can be prewarned)/ stopped cars (a current tesla issue), deer/animals wandering onto the road, construction (can be signaled in a standard way and prewarned). Debris/potholes?
>Green seems like a horrible choice for a post in night driving.
Those pylons have reflective strips on them for the night.
Those pylons have reflective strips on them for the night.
It doesn't see the orange post later in the video either...
Thanks for the timestamp!
Watching FSD driving feels like watching one of my side projects mostly work - it's impressive that it works at all and it's exciting and it would be wildly unethical pass off to someone without the access and ability to tweak it.
In addition to the fact that we shouldn't be testing this stuff on public roads, it's baffling to me that anyone wants to use this in it's current state. To me this seems so much more stressful than driving the car myself. Having to maintain constant vigilance of what's going on around me as with normal driving, but with the additional complexity of having to keep second guessing what that car is going to do about it as well seems so much worse to me than just driving the car.
Honestly I don’t understand why Mobileye isn’t getting more attention this is one of their more recent unedited videos https://youtu.be/50NPqEla0CQ filmed in NYC city traffic…
Intel is pushing them through an IPO and I’m pretty sure I’m going to put down a substantial investment out of all the players they seem the be the only ones that actually taking a serious evolutionary approach to consumer AVs.
Their upcoming product stack shows an impressive strategy they aren’t trying to sell L4/5 as something that is just around the corner.
They are still perfecting Level 2/3+ as a consumer targeted product (~$1000 OEM price) and focusing on full autonomy only for robotaxis where an OEM price of $10,000-20,0000 for the hardware won’t be an issue.
Intel is pushing them through an IPO and I’m pretty sure I’m going to put down a substantial investment out of all the players they seem the be the only ones that actually taking a serious evolutionary approach to consumer AVs.
Their upcoming product stack shows an impressive strategy they aren’t trying to sell L4/5 as something that is just around the corner.
They are still perfecting Level 2/3+ as a consumer targeted product (~$1000 OEM price) and focusing on full autonomy only for robotaxis where an OEM price of $10,000-20,0000 for the hardware won’t be an issue.
The traffic here seems well organized and very low. Look at this demo from RoboTaxi: https://www.youtube.com/watch?v=TFEvkmvIjVo
That is awesome. I think they are going to have to be more aggressive in this type of traffic, with pedestrians learning they can just wander in front of the car (the car can't deliberately not make eye contact and slowly move forward like a human driver can). I also like the situation at 3:48 where the oncoming human driver waves the car to overtake on the footpath, before realizing the driverless car is not going to do that and he has to back up quite a distance. Not sure what would happen there if there were more cars behind the oncoming car. Swerving towards the dog and giving it a scare at 3:10 seems a very human thing to do though, but I suspect it was probably avoiding something off camera.
If it can't see the pylons, I wonder if it would see a toddler?
I hope Tesla isn't testing that in production.
No need when it can be tested on the public in Beta.
pro tip: if there are FSD Teslas around, do not dress your toddler in a green jacket with horizontal white stripes.
Everyone is blaming FSD but this driver is just as responsible. Earlier he let the car fly through a red light.
Even if you buy the known nonsense of the human driver being expected to pay 100%attention at all times and be able to instantaneously take control why are you blaming the driver?
It’s called full self driving: even rudimentary crash avoidance from 10+ years could avoid this
It’s called full self driving: even rudimentary crash avoidance from 10+ years could avoid this
The car does a right turn on a red without stopping. I wouldn't call that "flying through a red". It's closer to the "rolling stop" behavior. I'm guessing he didn't see any cross traffic in that lane and just let it go.
This update was supposed to disable rolling stops.
Tesla throws updates over the fence with sometimes as little as 2-3 days since the initial updates. Phantom braking saw multiple updates in a couple of weeks, and some of them made it better, then worse, then better, and so on.
Their attitude to testing would be laughable if it wasn't so dangerous and reckless.
Their attitude to testing would be laughable if it wasn't so dangerous and reckless.
"Just as responsible"... for not being able to instantaneously react to the car abruptly steering itself into a bollard at the last second?
It wasn’t abrupt… they even say right after they didn’t think it would actually hit. They should have disengaged the moment it started to take the turn wrong. Not let it try to fix itself.
The driver watches as his car runs a red light and then swerves into bollards, but still allows it to come within a couple of feet of pedestrians standing in the bollards later on. I can't believe people trust their cars not to kill people.
As someone who has never driven is San Jose or the bay area, it certainly seems much better than a novice driver in a extremely complicated road layout.
The streets have not only tram/train lanes but tram specific signals, keep clear areas,etc. Not to mention a mixture of bike and park lanes.
Never driven a Tesla personally but I'd be pretty confident FSD could handle the 50 mile commute to my friends place that is just interstate, highways, and a roundabout.
My only real complaint in the video is how aggressive it seems but I think that's configurable. Perhaps they should limit the FSD speed/aggressiveness to the specific drivers own capability in the area.
The streets have not only tram/train lanes but tram specific signals, keep clear areas,etc. Not to mention a mixture of bike and park lanes.
Never driven a Tesla personally but I'd be pretty confident FSD could handle the 50 mile commute to my friends place that is just interstate, highways, and a roundabout.
My only real complaint in the video is how aggressive it seems but I think that's configurable. Perhaps they should limit the FSD speed/aggressiveness to the specific drivers own capability in the area.
Kinda worrying that someone who was paying so much attention to what the car was doing that they were actually giving a running commentary still wasn't able to react quickly enough to prevent an accident.
Lucky it was a plastic bollard and not someone's leg or baby carriage.
Lucky it was a plastic bollard and not someone's leg or baby carriage.
The immediate goal of FSD is NOT to drive perfectly, the goal (MVP) is to drive safer than the average human.
Most probably the type of accidents the FSD MVP will be having is different than a human would and vice versa.
There are 60k testers [0] of FSD and only 1 accident since September. OK, not really average drivers but that's one accident for 15k years of (some) driving. Probably FSD is already safer than most drivers. Although FSD is not yet complete as it cannot handle some situations at all and hands over to the human.
[0] https://seekingalpha.com/article/4482970-teslas-ace-in-the-h...
Most probably the type of accidents the FSD MVP will be having is different than a human would and vice versa.
There are 60k testers [0] of FSD and only 1 accident since September. OK, not really average drivers but that's one accident for 15k years of (some) driving. Probably FSD is already safer than most drivers. Although FSD is not yet complete as it cannot handle some situations at all and hands over to the human.
[0] https://seekingalpha.com/article/4482970-teslas-ace-in-the-h...
Relevant moment:
https://youtu.be/sbSDsbDQjSU&t=208
https://youtu.be/sbSDsbDQjSU&t=208
Tangential question:
Which goal is more difficult technically, FSD or Mars Settlement?
As in, which one just needs money and time versus true technical invention?
Which goal is more difficult technically, FSD or Mars Settlement?
As in, which one just needs money and time versus true technical invention?
Mars is going to be tricky.
The right turn at 4:45, I have no idea how I as a human could be expected to know where to drive the car.
Agreed, and at 6:30, more train track confusion. And LIDAR wouldn't have helped on either of these.
I've rewatched the right turn at 6:30 a few times and I'm still not really certain where a driver is supposed to go.
feels like the last 20% of this will take much more than 80% of the whole delivery time
5% will take more time than the 95%. There is zero chance that they can make it work with their potato cameras. Tesla will not stop selling this scam thought, too many dumb people with too much money.
> Full Self Driving Beta 10.10 version 2021.44.30.15
How often do these version number increment?
Was he using a version that was tested for more than a couple of months?
If not, how can that be justified?
How often do these version number increment?
Was he using a version that was tested for more than a couple of months?
If not, how can that be justified?
[deleted]
This should not be something that is allowed on public roads by end-users, but rather on closed tracks by specialists. If they want to test it out on public roads, run the analysis and look at whenever it diverges from the drivers decisionmaking instead.