Tesla’s “Full Self-Driving” sees pedestrian, chooses not to slow down(arstechnica.com)
arstechnica.com
Tesla’s “Full Self-Driving” sees pedestrian, chooses not to slow down
https://arstechnica.com/cars/2023/05/teslas-full-self-driving-sees-pedestrian-chooses-not-to-slow-down/
74 comments
I mean the tweeter is called Whole Mars Catalog and if you click on their profile it's basically all about fanboying Musk and Tesla.
Kudos to them for sharing the video to begin with, but it doesn't surprise me one bit that they're trying to spin it so it's actually the pedestrian's fault, or SF's fault, or city mentality, basically anyone's but Tesla's.
Kudos to them for sharing the video to begin with, but it doesn't surprise me one bit that they're trying to spin it so it's actually the pedestrian's fault, or SF's fault, or city mentality, basically anyone's but Tesla's.
Their previous claim to fame:
Is there anyone in the Bay Area with a child who can run in front of my car on Full Self-Driving Beta to make a point? I promise I won't run them over... (will disengage if needed)
(this is a serious request)
Unironic usage of the word "bullish" is a dead giveaway that a person is working full time at a traveling kids entertainment facility. Very convenient for the readers :) .
In car centric countries it's always the pedestrian or cyclist fault. Maybe because not being in a car is associated with poverty IDK but the hatred is bizarre.
After watching the video several times, I would definitely have slowed down seeing the pedestrian near the cross walk a hundred meters back, and would definitely have stopped in this instance. You never know when someone is going to run or not look for a car and being cautious saves a life. I live and have lived in a city as a driver for decades in multiple countries including in very dense areas and in no case would the FSD behavior be acceptable. We know from the Uber example this kind of automated disregard of pedestrians can result in death.
This is a much bigger and more serious issue, dozens of pedestrians are killed by cars every year in my city.
This is a much bigger and more serious issue, dozens of pedestrians are killed by cars every year in my city.
In the UK that would be an instant driving test failure.
The moment a pedestrian puts their feet on the crossing they have absolute right of way & it’s on you to stop. If a pedestrian is waiting to cross, you’re expected to stop.
If Tesla exports this self driving code to the UK it’s going to cause accidents because people will absolutely not be expecting this behaviour from vehicles on the road in the UK.
The moment a pedestrian puts their feet on the crossing they have absolute right of way & it’s on you to stop. If a pedestrian is waiting to cross, you’re expected to stop.
If Tesla exports this self driving code to the UK it’s going to cause accidents because people will absolutely not be expecting this behaviour from vehicles on the road in the UK.
One side-effect of self driving cars is going to be finally having to deal with the reality of impossible laws. If every single vehicle followed traffic laws perfectly, it'd likely cause chaos and gridlock.
For example, in this case, even someone standing near a crosswalk might be enough for all vehicles to refuse to enter.
For example, in this case, even someone standing near a crosswalk might be enough for all vehicles to refuse to enter.
Standing close enough to a crossing that you appear to be preparing to cross is easily enough for drivers to stop and wait in some European countries, e.g. here in Denmark.
If the pedestrian remains still, but the driver sees no reason they shouldn't cross, eventually the driver might well gesture, honk etc.
If the pedestrian remains still, but the driver sees no reason they shouldn't cross, eventually the driver might well gesture, honk etc.
An AI should never make an illegal honk like that.
Sorry, are you saying that it'd be a problem that pedestrians could cross when they intend to? I'm not following
No, that a human could be standing near but not in a crosswalk, talking. The law says you must treat them as attempting to cross, even though it is obvious they are not. Caress like this abound.
I don't think the law says that you need to treat people not entering a crosswalk as attempting to cross it. We can play this game all day if you just want to be obtuse about using words. But nothing you described was impossible to work out, nor was the scenario you most immediately described in your recent post even remotely troublesome. Do you mean if people are in the crosswalk but don't cross? How often does that really happen? Ironic considering that in this case, the pedestrian was in the crosswalk and the car drove through anyway. Pretty dangerous IMO
I also think the desire of some for an absolutist interpretation of driving regulation in this thread would result in chaos too.
How many seconds is one supposed to stop at a stop sign? The law is silent on the matter. There is so much around laws or rule systems that forms as conventions around the rules that are often just as important as the text of the rules themselves.
See also solid white lines (do not cross) in the US. People cross them for pragmatic reasons in many locations all the time, often in full view of law enforcement.
There are also road users for whom laws are rarely enforced or more frequently disobeyed, such as cyclists. In many places cyclists are bound to obey the same traffic lights as motor vehicles. This plainly doesn't happen 100 percent of the time, and again is often tolerated by law enforcement. A car with an absolutist interpretation of driving regulations would therefore still have to deal with the law enforcement and others may not share in this view.
How many seconds is one supposed to stop at a stop sign? The law is silent on the matter. There is so much around laws or rule systems that forms as conventions around the rules that are often just as important as the text of the rules themselves.
See also solid white lines (do not cross) in the US. People cross them for pragmatic reasons in many locations all the time, often in full view of law enforcement.
There are also road users for whom laws are rarely enforced or more frequently disobeyed, such as cyclists. In many places cyclists are bound to obey the same traffic lights as motor vehicles. This plainly doesn't happen 100 percent of the time, and again is often tolerated by law enforcement. A car with an absolutist interpretation of driving regulations would therefore still have to deal with the law enforcement and others may not share in this view.
This is one of the fundamental pain points of AI. All those moral dilemmas and trolley scenarios we've philosophized about since ancient history? Well it's come time to choose.
Computers do exactly what they are told to do and never do what you did not tell them to do. They do not have any common sense that you did not give them. As you build more and more capable computers and give them more influence and control, you damn well better have an exhaustive, complete and comprehensive set of values for it to follow.
Computers do exactly what they are told to do and never do what you did not tell them to do. They do not have any common sense that you did not give them. As you build more and more capable computers and give them more influence and control, you damn well better have an exhaustive, complete and comprehensive set of values for it to follow.
Mmmm. That's the case with programs, but it's definitely not the case with neural nets. Much like you or I, NNs do things based on training, experience, and data.
They do, but that doesn't mean our data has the answer for every possible scenario. This is important because AI will run into ethical scenarios humans rarely or never have to think about.
At first this will be due to AI's one current superhuman ability: speed; a classic is the do you drive into the old couple or the little kid car scenario. It's not really an issue for humans because the situation is over before we can make more than a reflexive response, but the AI can take its time to make a concrete decision. This isn't a thing we spend a lot of time thinking or blogging about and a societal consensus probably has not formed and wouldn't show up in the data. Presumably the AI would pick at more or less random as we are forced to do, but I can see an entity capable of superspeed choosing to perform random actions as causing other as yet unforseen problems.
The big pinch is if and when we create transformational AI. We will be presented with options we had never imagined being possible. Do you keep Mars as a memory of the old days, or do you use its crust to construct interstellar spaceships and build better lives for Earth humans? Do you allow humans to live lives as they do now, or do you modify their biology and neurology to live longer lives and never engage in antisocial behavior?
These are lazy obvious examples that I came up with but there will be many less-cosmic decisions we will be presented with that we absolutely do not have an existing moral template to guide us.
At first this will be due to AI's one current superhuman ability: speed; a classic is the do you drive into the old couple or the little kid car scenario. It's not really an issue for humans because the situation is over before we can make more than a reflexive response, but the AI can take its time to make a concrete decision. This isn't a thing we spend a lot of time thinking or blogging about and a societal consensus probably has not formed and wouldn't show up in the data. Presumably the AI would pick at more or less random as we are forced to do, but I can see an entity capable of superspeed choosing to perform random actions as causing other as yet unforseen problems.
The big pinch is if and when we create transformational AI. We will be presented with options we had never imagined being possible. Do you keep Mars as a memory of the old days, or do you use its crust to construct interstellar spaceships and build better lives for Earth humans? Do you allow humans to live lives as they do now, or do you modify their biology and neurology to live longer lives and never engage in antisocial behavior?
These are lazy obvious examples that I came up with but there will be many less-cosmic decisions we will be presented with that we absolutely do not have an existing moral template to guide us.
> How many seconds is one supposed to stop at a stop sign?
I don't see a need for the law to specify a time. You have to stop and you proceed as soon as you have the right of way and it is safe to do so.
I don't see a need for the law to specify a time. You have to stop and you proceed as soon as you have the right of way and it is safe to do so.
My point still stands regardless. Rule systems are more than just their printed texts.
A strict interpretation by your car may mean that it yields to a person standing just off the curb is in the crosswalk with no intention of crossing. Your car won't go though because the law says they have the right of way.
Not moving when given the chance is indicative of non-intent.
For a working AI, it can still stop safely if the pedestrian starts moving into the street in the few seconds the car is crossing.
For a working AI, it can still stop safely if the pedestrian starts moving into the street in the few seconds the car is crossing.
What if someone stands in the cross walk, or is intentionally slowly moving to cause an issue?
I'm not sure what else I would do when driving except to wait for them to move. I suppose if it became clear that someone was intentionally trying to obstruct me, I might try to slowly move around them if it was safe to do so.
> absolutist interpretation
What's absolutist - the driver blew right through the crosswalk while someone was literally in it...
What's absolutist - the driver blew right through the crosswalk while someone was literally in it...
[deleted]
That's the law in my state here in the States as well. The pedestrian has the absolute right-of-way in the crosswalk and you are to stop for a pedestrian waiting to enter the crosswalk. And I live in a car-centric state - as none of our cities have much in the way of public transportation other than a bus. Maybe we need to adopt your squiggly lines so it's easier for FSD to recognize when it needs to be paying attention?
Britain goes further than many countries: once you've stopped, you aren't allowed to start again until the whole crossing is clear of pedestrians.
> If Tesla exports this self driving code to the UK
I'm guessing the vast majority of training data is collected in countries with right-hand traffic flow. I suspect it might not be so straightforward to adapt to countries with left-hand traffic.
I'm guessing the vast majority of training data is collected in countries with right-hand traffic flow. I suspect it might not be so straightforward to adapt to countries with left-hand traffic.
It's just a sign flip. Mirror the incoming video and the outgoing commands.
If the incoming video is mirrored, how can signs be read?
Tell that to the neural network.
In my state, the law says the Tesla's action here is fine, since the pedestrian wasn't close to the car's lane.
> Vehicles are required to stop when you are crossing their lane or when you are close to entering their lane.[1]
For California, I couldn't find an independent reference for the situation here but did find one saying a stopped driver doesn't have to wait until the pedestrian fully exits the crosswalk.[2]
[1] https://leddylaw.com/what-are-my-rights-as-a-pedestrian-in-s...
[2] https://www.karlaw.com/blog/do-drivers-have-to-wait-for-pede...
> Vehicles are required to stop when you are crossing their lane or when you are close to entering their lane.[1]
For California, I couldn't find an independent reference for the situation here but did find one saying a stopped driver doesn't have to wait until the pedestrian fully exits the crosswalk.[2]
[1] https://leddylaw.com/what-are-my-rights-as-a-pedestrian-in-s...
[2] https://www.karlaw.com/blog/do-drivers-have-to-wait-for-pede...
For California the law is here: https://law.justia.com/codes/california/2022/code-veh/divisi...
The Tesla vehicle violated this. subdivision (3) kinda implicitly confirms that a driver doesn't need to wait for a pedestrian to fully exit (what allows for that though is just the definition of and requirements around "right of way").
The Tesla vehicle violated this. subdivision (3) kinda implicitly confirms that a driver doesn't need to wait for a pedestrian to fully exit (what allows for that though is just the definition of and requirements around "right of way").
It’s not clear cut IMO. It doesn’t say you have to stop because a pedestrian is in the crosswalk, just that cars have to yield the right of way to pedestrians. You could argue no yielding was required because there was no conflict, the car and the pedestrian just proceeded at pace and did not interfere with each other.
Also, since we’re being pedantic, the vehicle can’t violate the law because it isn’t a person. The driver would be in violation.
Also, since we’re being pedantic, the vehicle can’t violate the law because it isn’t a person. The driver would be in violation.
> You could argue no yielding was required because there was no conflict, the car and the pedestrian just proceeded at pace and did not interfere with each other.
Sure, you could argue that. Were it true it'd be fine. But the pedestrian clearly slowed nearly to a stop (and shot the Tesla a disparaging look).
Sure, you could argue that. Were it true it'd be fine. But the pedestrian clearly slowed nearly to a stop (and shot the Tesla a disparaging look).
> But the pedestrian clearly slowed nearly to a stop (and shot the Tesla a disparaging look).
Unless we’re looking at different videos I don’t see either of those things happening. They don’t appear to change their pace at all after entering the crosswalk and they don’t seem disturbed in the slightest.
Unless we’re looking at different videos I don’t see either of those things happening. They don’t appear to change their pace at all after entering the crosswalk and they don’t seem disturbed in the slightest.
On a 2 lane road, you are about two seconds away from entering the other lane the moment you step on one. How damn close does it have to be ? Also, even taking law out of account: stop when there are goddamn pedestrians. At all times. Even if the law is with you. Heaven is full of people who had the right of way, and I'd rather not add each individual's perception of "but I had the right to speed up as the deceased was crossing!".
It may be necessary to drive a little more „human“ to improve the overall FSD experience. But once you start blurring the line of lawful/unlawful you get yourself into muddy waters. Seems as if Tesla is pulling out of their robotaxi/FSD plans without admitting so. Not good. Nonetheless i think they are doing some things right. a) they make quite an effort to visulize the mind of vehicle and b) they do not try to pretend safety by stacking up all sorts of limitations (geofencing, highways, top speed). If oly they marketed the system as ADAS and designed the UX as such and would not have made these silly robotaxi claims…
Reminds me of this masterpiece:
https://www.youtube.com/watch?v=9S75Rfva9O8
Maybe the car knew something, eh? :)
https://www.youtube.com/watch?v=9S75Rfva9O8
Maybe the car knew something, eh? :)
T̵o̵ ̵g̵e̵t̵ ̵t̵h̵e̵ ̵c̵l̵i̵c̵k̵b̵a̵i̵t̵ ̵h̵e̵a̵d̵l̵i̵n̵e̵ ̵o̵u̵t̵ ̵o̵f̵ ̵t̵h̵e̵ ̵w̵a̵y̵:̵ ̵T̵h̵e̵ ̵p̵e̵d̵e̵s̵t̵r̵i̵a̵n̵ ̵w̵a̵s̵ ̵o̵n̵ ̵t̵h̵e̵ ̵s̵i̵d̵e̵ ̵o̵f̵ ̵t̵h̵e̵ ̵r̵o̵a̵d̵.̵ ̵N̵o̵t̵ ̵i̵n̵ ̵t̵h̵e̵ ̵p̵a̵t̵h̵ ̵o̵f̵ ̵t̵h̵e̵ ̵v̵e̵h̵i̵c̵l̵e̵.̵
Edit: I'm embarrassed to say that I did not watch the video. It's much worse than the description implies. The pedestrian was in the crosswalk, and crossing. FSD absolutely should have stopped.
However, as the article points out, "California law requires drivers to come to a complete stop for pedestrians at crosswalks".
I see this as a failure of the FSD development process, not of FSD. Tesla's software devs simply aren't familiar with driving laws. They may not even be drivers - it's not a requirement.
You'd expect Tesla to simply teach the algorithm to follow driving laws, but they haven't done that.
Edit: I'm embarrassed to say that I did not watch the video. It's much worse than the description implies. The pedestrian was in the crosswalk, and crossing. FSD absolutely should have stopped.
However, as the article points out, "California law requires drivers to come to a complete stop for pedestrians at crosswalks".
I see this as a failure of the FSD development process, not of FSD. Tesla's software devs simply aren't familiar with driving laws. They may not even be drivers - it's not a requirement.
You'd expect Tesla to simply teach the algorithm to follow driving laws, but they haven't done that.
Tesla purposefully programs its FSD to drive "human-like" rather than strictly following the laws. For example, it will go with the flow of traffic rather than strictly following the speed limit.
Ah yes, but that's under the control of the driver. I can set it to obey the speed limit easily.
Where's the option for "stop for pedestrians at crosswalks"?
Where's the option for "stop for pedestrians at crosswalks"?
> Where's the option for "stop for pedestrians at crosswalks"?
I really hope this doesn't happen. Configurable driving behaviors might make sense to a degree, but this feels dangerous. I would hope that the Department of Transportation (or equivalent in other countries) would regulate such things.
I really hope this doesn't happen. Configurable driving behaviors might make sense to a degree, but this feels dangerous. I would hope that the Department of Transportation (or equivalent in other countries) would regulate such things.
Worth pointing out that this behavior is in the release notes:
> - Improved ego's assertiveness for crossing pedestrians in cases where ego can easily and safely cross before the pedestrian.
> - Improved ego's assertiveness for crossing pedestrians in cases where ego can easily and safely cross before the pedestrian.
Human drivers in my city routinely do the same thing, so from my perspective the car behaved in a very-human-like manner.[a]
A longer video by the same person showing FSD Beta 11.4.1 driving his car is here (40+ minutes):
https://www.youtube.com/watch?v=XAFyPqSu8Vo
---
[a] What's more, when there's a line of pedestrians about to cross an intersection, which will block it, I've seen human drivers accelerate to avoid getting stuck behind the line. Actually, I see that all the time when there are a lot of pedestrians downtown.
A longer video by the same person showing FSD Beta 11.4.1 driving his car is here (40+ minutes):
https://www.youtube.com/watch?v=XAFyPqSu8Vo
---
[a] What's more, when there's a line of pedestrians about to cross an intersection, which will block it, I've seen human drivers accelerate to avoid getting stuck behind the line. Actually, I see that all the time when there are a lot of pedestrians downtown.
I noticed EV drivers are much more willing to break and wait for people crossing.
I think it is because of regenerative breaking and quick acceleration, which makes the cost of stopping much smaller. Or negative if you get a kick out of acceleration.
I think it is because of regenerative breaking and quick acceleration, which makes the cost of stopping much smaller. Or negative if you get a kick out of acceleration.
Fair enough, but better than a human is supposed to be the point of all this. Who needs self driving cars if they're going to be mowing people down while they check their Instagram?
Back in Austin, it was pretty common for pedestrians to have to stop and wait in the crosswalk while cars continued to pass. I've also seen cars pulled over by police for doing so.
[deleted]
Very human.
This is what you want if you want AI to be Artificial Human Intelligence.
I don't want to sound pedantic. If you would put a camera in a 1000 cars fully driven by humans and film each for a day, how many of such recordings could be found? I guess a handful.
The real question is... how far off is this technology from practical human driving and when (not whether) it will surpass.
The real question is... how far off is this technology from practical human driving and when (not whether) it will surpass.
You realize "it drives like a human being" and "it will surpass [humans in terms of safety?]" are contradictory statements?
I'm confused. English is not my native language but did I imply anywhere "like a human being"?
I was reasonig about the distance between machine and humans in their driving capabilities. It is clear that there us a gap in favor of humans and the gap is narrowing. Moreso, in the future there will be a surpassing. I don't get the contradiction you see.
I was reasonig about the distance between machine and humans in their driving capabilities. It is clear that there us a gap in favor of humans and the gap is narrowing. Moreso, in the future there will be a surpassing. I don't get the contradiction you see.
As written, they are not given that the first is present tense and the second is future tense. There is also some ambiguity surrounding what "like" a human being means in this context, but I could imagine a car driving "like" a human, but safer.
It's not exactly contradictory. It can "surpass humans" in the sense of being significantly better than average. If FSD drove like only the best human drivers, free of mistakes, you could say it surpassed us in terms of safety.
If you would put a camera in a 1000 cars fully driven by humans and film each for a day, how many of such recordings could be found?
Already with the whataboutism? Automated driving is supposed to promise safer driving than humans, not bug-for-bug compatibility.
Already with the whataboutism? Automated driving is supposed to promise safer driving than humans, not bug-for-bug compatibility.
But if humans do this 10 out of 1000 and automated cars do it 1 out of 1000 times isn’t the automated driving safer than humans then?
The video didn't look unsafe to me. The pedestrian was walking in from the left side of the road and never got close to the car. In my state that's perfectly legal.
> never got close to the car
Yes, because the pedestrian slowed down when it saw the Tesla was not giving them right of way (as the Tesla is required to in the state of california).
> In my state that's perfectly legal.
In most states its illegal to force a vehicle or pedestrian or whatever that has right of way to slow to a near stop to avoid a collision with you that you could have easily stopped and avoided. I don't know of a state where a pedestrian in a crosswalk would not have right of way in this situation.
Now, there are many states where a pedestrian in a crosswalk doesn't have like right of way over everything (though, technically, not california), but for the most part this applies to vehicles that could cross the crosswalk without affecting the pedestrian.
Yes, because the pedestrian slowed down when it saw the Tesla was not giving them right of way (as the Tesla is required to in the state of california).
> In my state that's perfectly legal.
In most states its illegal to force a vehicle or pedestrian or whatever that has right of way to slow to a near stop to avoid a collision with you that you could have easily stopped and avoided. I don't know of a state where a pedestrian in a crosswalk would not have right of way in this situation.
Now, there are many states where a pedestrian in a crosswalk doesn't have like right of way over everything (though, technically, not california), but for the most part this applies to vehicles that could cross the crosswalk without affecting the pedestrian.
I don't see any particular slowing by the walker, and neither did another HN commenter who replied to you.
This is an asinine argument. It’s not necessarily safer right out of the gate! Maybe they just need to fix the bug.
Every bug needs solving.
But also: every bug will _never_ be solved. Why would you think any system or even component should be perfect? Name one perfect system. More simple systems like traffic lights, headlights, tires, brakes,... each of them can fail. Trains can derail, so we should not trust trains?
The “bug” involves a breathing, thinking human being who didn’t opt into being a lab rat.
I think in this respect it is already safer than human driving, my point being.
Okay, I see what you're saying. But wouldn't an automated system be consistent in its behavior? IOW, if it does not respect crosswalk laws in one case, should we not expect it to do the same in all, or at least most cases? At least some (and hopefully most) human drivers are staying out of pedestrian-occupied crosswalks.
That's what happens when your leadership has third world carbrain (especially as a mode of exclusion) expectations.
What does “third world carbrain” mean?
Lol people are still falling for this scam? There is a fool born every minute I guess.
The benchmark for this situation shouldn’t be “it was as safe/safer than a human” when we have to append “while clearly breaking the law”.
This is a non-story. It's pretty clear to me the car should have stopped (and I question the original tweeter's road judgement for suggesting otherwise) but plenty of bad drivers do the same thing in the city.
"FSD doesn't have to be perfect, it only has to be better than humans." That's what I hear from the pro-AI guys all the time.
So you can have that, or "well humans do that too", but not both.
So you can have that, or "well humans do that too", but not both.
That isn't what I'm saying. I agree that this behavior needs to be fixed. I'm saying that "Tesla's self-driving algorithm makes an error at a single intersection in SF" doesn't meet my threshold for news any more than a human doing that would.
A story called "Teslas systemically drive unsafely at intersections", with an investigation into this kind of behavior at mass scale? That's actual journalism, not just a glorified quote retweet.
A story called "Teslas systemically drive unsafely at intersections", with an investigation into this kind of behavior at mass scale? That's actual journalism, not just a glorified quote retweet.
FSD already introduces new failure modes, if it's going to hardcode human mistakes as well what's even the point?
This is also assuming that FSD runs through crosswalks at the same rate as human drivers, but it might be much more frequent. If Tesla actually backed up their safety claims with data we could check and verify, but they are famously reticent to provide evidence of how their cars perform that hasn't been cherry picked.
This is also assuming that FSD runs through crosswalks at the same rate as human drivers, but it might be much more frequent. If Tesla actually backed up their safety claims with data we could check and verify, but they are famously reticent to provide evidence of how their cars perform that hasn't been cherry picked.
I'm city person, lived in NYC most of my life and continue to do so today, and it's pretty clear to me that the pedestrian entered the crosswalk (on the other side) before the tesla made it to the crosswalk... like... it's on video...