George Hotz is on a hacker crusade against the ‘scam’ of self-driving cars(theverge.com)
theverge.com
George Hotz is on a hacker crusade against the ‘scam’ of self-driving cars
https://www.theverge.com/2018/7/13/17561484/george-hotz-comma-ai-self-driving-car-scam-diy-kit
146 comments
No safety relevant code is written in Python. All the safety relevant code runs real-time on a STM32 micro (inside the Panda), it's written in C and it's placed at the interface between the car and the Eon. This code ensures the satisfaction of the 2 main safety principles that a Level 2 driver assistance system must have: 1- the driver needs to be able to easily disengage the system at any time; 2- the vehicle must not alter its trajectory too quickly for the driver to safely react. See https://github.com/commaai/openpilot/blob/devel/SAFETY.md
Among the processes that runs on the Eon, you can find algorithms for perception, planning and controls. Most of it is actually autogenerated code in C++ (see model predictive controls). Python code is used mainly as a wrapper and for non-computational expensive parts. To use functional safety terminology, the Eon functionality is considered QM (Quality Management). This means that any failure in delivering the desired output at the right time is perceived as bad quality and has no safety implications. So, how often those algorithms deliver the wrong output because some parts are written in Python? How often because RT isn’t enforced? Negligible. Pretty much all the mistakes of a level 2 driver assistance system are due to the quality of the algorithms, the models, the policies etc… There is a long way to go before changing the coding language will be the lowest hanging fruit to improve the system. Until then, using the simplest and most agile coding language (given performance constraints) is probably the best way to maximize quality.
Among the processes that runs on the Eon, you can find algorithms for perception, planning and controls. Most of it is actually autogenerated code in C++ (see model predictive controls). Python code is used mainly as a wrapper and for non-computational expensive parts. To use functional safety terminology, the Eon functionality is considered QM (Quality Management). This means that any failure in delivering the desired output at the right time is perceived as bad quality and has no safety implications. So, how often those algorithms deliver the wrong output because some parts are written in Python? How often because RT isn’t enforced? Negligible. Pretty much all the mistakes of a level 2 driver assistance system are due to the quality of the algorithms, the models, the policies etc… There is a long way to go before changing the coding language will be the lowest hanging fruit to improve the system. Until then, using the simplest and most agile coding language (given performance constraints) is probably the best way to maximize quality.
As someone who has had a bit to do with safety critical software in agricultural machinery, and I am sure to any one in any other regulated industry this sounds crazy. The regulation in many industries for software that can kill is onerous, python on Android for any safety related code in other industries would be the punch line of a joke.
To clarify (as a safety hobbyist) why this is a problem:
* consumer hardware does not normally fulfil automotive safety requirements. It could for instance go into thermal shutdown or into a degraded mode if the temperature is too high. Additionally, there is no HW redundancy, I assume that if any of the HW components of the smartphone fail, the system cannot continue to maintain its safety properties.
* Android is a consumer OS, designed for consumer workloads. A real-time, safety certified OS like INTEGRITY, ThreadX, Nucleus etc should be typically used for such workloads.
* The safety-relevant software running on top of the OS is developed with specific toolchains and using specific programming languages. Some requirements [1] for a language used in safety-critical context are defined behaviour, explicit dependability support (e.g. design by contract), predictable timing, suitability for static verification, significant field use, strong typing (not necessarily static typing!), feasibility to restrict the language to a subset (e.g. MISRA, JSF, etc).
One of the most popular languages for such software is C, which is not safe and doesn't fulfil several of the above criteria. This is mitigated through tooling, processes, code generation, design validation & verification and so on.
[1]: Taken from Embedded Software Development for Safety-Critical Systems, Hobbs. Interestingly the author would personally choose D or Rust for safety-critical development with the condition of having enough confidence in the compilers.
* consumer hardware does not normally fulfil automotive safety requirements. It could for instance go into thermal shutdown or into a degraded mode if the temperature is too high. Additionally, there is no HW redundancy, I assume that if any of the HW components of the smartphone fail, the system cannot continue to maintain its safety properties.
* Android is a consumer OS, designed for consumer workloads. A real-time, safety certified OS like INTEGRITY, ThreadX, Nucleus etc should be typically used for such workloads.
* The safety-relevant software running on top of the OS is developed with specific toolchains and using specific programming languages. Some requirements [1] for a language used in safety-critical context are defined behaviour, explicit dependability support (e.g. design by contract), predictable timing, suitability for static verification, significant field use, strong typing (not necessarily static typing!), feasibility to restrict the language to a subset (e.g. MISRA, JSF, etc).
One of the most popular languages for such software is C, which is not safe and doesn't fulfil several of the above criteria. This is mitigated through tooling, processes, code generation, design validation & verification and so on.
[1]: Taken from Embedded Software Development for Safety-Critical Systems, Hobbs. Interestingly the author would personally choose D or Rust for safety-critical development with the condition of having enough confidence in the compilers.
> [...] to any one in any other regulated industry this sounds crazy.
This sounds crazy even to a person who makes living from running software on commodity hardware (x86/x86_64). Android does not sound like a hard real-time OS. I've seen it hang up in a fsckin' coffee machine! I don't want it at the center of two tons of steel that move 100km per hour.
This sounds crazy even to a person who makes living from running software on commodity hardware (x86/x86_64). Android does not sound like a hard real-time OS. I've seen it hang up in a fsckin' coffee machine! I don't want it at the center of two tons of steel that move 100km per hour.
Ummm... If you look in the actual lib/ directory where the work happens, you will see a bunch of high-performance C written in and generated by Acado, which is a very efficient optimization framework. It does look like there is a bunch of python logic for managing that controller, but a significant amount of work seems to have gone into performance optimization.
I am curious how they prevent the Android GC from causing problematically long pauses, though.
There's no Android GC, there's a Java GC. Python on Android isn't written in Java, it's CPython, written in C.
Python has a GC as well, but we turn it off for the control loop processes. https://github.com/commaai/openpilot/blob/devel/selfdrive/co...
Haters love to bring up the Python, but they never stick around long enough to explain exactly why it's a problem.
Python has a GC as well, but we turn it off for the control loop processes. https://github.com/commaai/openpilot/blob/devel/selfdrive/co...
Haters love to bring up the Python, but they never stick around long enough to explain exactly why it's a problem.
Because it's not a statically typed language for the most part, which brings in an entire class of bugs of its own. It's also an extremely mutable language.
Great for scripts, but there is a reason why most large companies start bolting on types on whatever dynamic language they started with.
Great for scripts, but there is a reason why most large companies start bolting on types on whatever dynamic language they started with.
Most production languages that nominally do some static typing aren't statically very safe - certainly Java, Go, C/C++ are rife with runtime errors. After the required testing to eliminate those, it's not clear Python is significantly different.
Edit: also, Python does eagerly signal type errors, unlike say Javascript or C, so you don't get silently wrong answers. C is the default language in auto industry. .
Yeah, this is a bit of whataboutism, certainly it would be nice if the state of the art in production languages was closer to the ideal of statically verified... Haskell and Rust are in the right direction, and would be clearly superior in this regard
Edit: also, Python does eagerly signal type errors, unlike say Javascript or C, so you don't get silently wrong answers. C is the default language in auto industry. .
Yeah, this is a bit of whataboutism, certainly it would be nice if the state of the art in production languages was closer to the ideal of statically verified... Haskell and Rust are in the right direction, and would be clearly superior in this regard
I wasn't implying that static types solve everything, they just make things n+1 better and remove a class of bugs.
Statically typed languages are mature and you have no excuse not using them if your doing anything that approaching a need for reliability. Cars do, social cat pictures, not so much.
Statically typed languages are mature and you have no excuse not using them if your doing anything that approaching a need for reliability. Cars do, social cat pictures, not so much.
What about Erlang?
Android is not typically a realtime OS. The root of the point, about uncontrolled pauses, is valid IMO.
The safety-relevant code runs on a microcontroller in the CAN dongle and is written in C.
Why does everyone assume the phone is doing all the work?
Why does everyone assume the phone is doing all the work?
[deleted]
Lack of type safety and static analysis tools. You can't apply formal verification to it. I wouldn't sit in a car in which any safety critical component was driven by python. Hopefully you would also not be able to get it certified for road use. I happen to know people that work on these problems for German automotive companies. This wouldn't fly there. I truly hope that solid engineering wins out over these approaches. That being said I admire the audacity.
Because German automotive companies are known for making systems that are robust and not just faking results.
I think it's great that your code is open source. The other self driving cars are all based on secret code that the public is not allowed to inspect or audit. This is extremely worrying.
Having your code open source means that outsiders will notice flaws in it, try not to "push back" too much against them. Sometimes they will point out serious flaws that allow you to improve your code significantly, sometimes they will point out non-issues or simply be wrong about things. So instead you should embrace it and take the time to consider the feedback. The crowd is a valuable resource that you have, that closed source projects don't.
Having your code open source means that outsiders will notice flaws in it, try not to "push back" too much against them. Sometimes they will point out serious flaws that allow you to improve your code significantly, sometimes they will point out non-issues or simply be wrong about things. So instead you should embrace it and take the time to consider the feedback. The crowd is a valuable resource that you have, that closed source projects don't.
> But the software that enables the semi-autonomous driving is free to download. Hotz says this allows him to sidestep the regulatory issue, though it’s unclear whether NHTSA would agree. “We aren’t selling any products that control a car,” he says. “We are giving away free software, and software is speech.” (A spokesperson for NHTSA did not respond to a request for comment.)
Sounds like they're just using open source to avoid liability and side step regulators. I love open source, but I do not think it's being used for benevolent reasons here.
Sounds like they're just using open source to avoid liability and side step regulators. I love open source, but I do not think it's being used for benevolent reasons here.
Yow! Shows how naive I am. Thanks for the reply..
The Android ART GC doesn't cause "long pauses".
https://www.youtube.com/watch?v=iFE2Utbv1Oo
https://www.youtube.com/watch?v=iFE2Utbv1Oo
[deleted]
As much as I love Python, you need realtime processing for this kind of things. You can't have a slow language with gc pauses and no way to guaranty execution time for a given operation.
You should also mention the GIL here (Global Interpreter Lock).
https://realpython.com/python-gil/
https://realpython.com/python-gil/
CPython spent actually need the GC. If you make sure you have no cycles, you can disable it, since the rest uses reference counting (which AFAIK is predictable).
If you want to replace the always attentive driver who never blinks their eyes?
It's not necessarily about speed of reaction, although that certainly plays a part.
Such software needs to react in real time though, if the task that's turning the steering wheel gets preempted in the middle of taking a curve on a cliff your self-driving car will become a self-flying car.
Such a system would be the equivalent of a driver that suddenly starts texting at all sorts of poorly chosen times.
Such software needs to react in real time though, if the task that's turning the steering wheel gets preempted in the middle of taking a curve on a cliff your self-driving car will become a self-flying car.
Such a system would be the equivalent of a driver that suddenly starts texting at all sorts of poorly chosen times.
It's more a matter of concurrency and blocking behavior. Gc blocks. A task you can't bound to an execution time blocks.
In the 1980's the backup flight control software for landing the space shuttle ran on a HP-41 calculator, though they never had to use it AFAIK, with the multiple redundant onboard computers.
That's not what I found online: the devices running customized SW were apparently used as personal calculators by the crew and they would have also used them for manual calculation in case their flight computer had a problem.
They weren't connected to other shuttle computers, were they?
They weren't connected to other shuttle computers, were they?
I found multiple stories/blogs with recollections like this one from Smithsonian including the use of calculators on early missions before laptops were used:
https://airandspace.si.edu/collection-objects/calculator-han...
I remembered reading it in the newspapers at the time which would have probably make the reports generally pre internet (or pre newspapers on the internet) in the 1980's. Of course I could be remembering it wrong or it could have been a HP calculator ad.
https://airandspace.si.edu/collection-objects/calculator-han...
I remembered reading it in the newspapers at the time which would have probably make the reports generally pre internet (or pre newspapers on the internet) in the 1980's. Of course I could be remembering it wrong or it could have been a HP calculator ad.
Love to study those projects in more details - Not just the code, but also the design, testing, validation processes.
What? thats enough redundancy for a space station
It sounds to me like you don't understand how little performance you actually need for this kind of stuff. People have been writing effective autopilot software for decades using hardware that wasn't even particularly fast at the time and may as well be a pile of sand today. What exactly do you think that that code needs to do? I promise that Python on a new smartphone can do way more now than we could do 20 years ago on a Pentium in C or C++. You're lucky if any of your sensors emit at even 100Hz.
you can't do timing sensitive stuff without an RTOS, because resources could be preempted by the os or other applications.
it used to be that certain chipsets could do motor control by bitbanging the parallel port, but modern pcs/phones can no longer do this due to latency.
it used to be that certain chipsets could do motor control by bitbanging the parallel port, but modern pcs/phones can no longer do this due to latency.
I think OP's point is about the latency from all the layers. At 70 mph, where 100ms is 3m, this is a problem for a control system.
Human latency is about 250ms.
I think that's not a fair comparison - a digital control system needs much finer time slices to compete with human reaction time. A control system that worked off of webcam images at 4hz would be a jittery mess, or just very slow if using a Kalman filter.
I agree it's not a fair comparison.
Your analogy, though, confuses latency and throughput.
[deleted]
When George demonstrated his first model I showed a friend who designs safety critical systems. To paraphrase he said, that of course you can hack something together in a weekend that does this.
However a analysing a million edge cases, making sure all failures fail in a safe way takes a lot of time and knowledge.
It made me think that this is a great demonstration of the difference between hobbyist hacking and professional engineering.
However a analysing a million edge cases, making sure all failures fail in a safe way takes a lot of time and knowledge.
It made me think that this is a great demonstration of the difference between hobbyist hacking and professional engineering.
> that of course you can hack something together in a weekend that does this. However a analysing a million edge cases, making sure all failures fail in a safe way takes a lot of time and knowledge.
Thats true for a lot of things. Most developers can whip up a Slack clone in a weekend, create an uber clone in a weekend or create a Netflix clone in a weekend. But dealing with all the edge cases, the details, making it scalable, etc. that's were the pain is.
Thats true for a lot of things. Most developers can whip up a Slack clone in a weekend, create an uber clone in a weekend or create a Netflix clone in a weekend. But dealing with all the edge cases, the details, making it scalable, etc. that's were the pain is.
> California’s DMV requires all companies testing self-driving cars to report how many times safety operators were forced to take control of its vehicles.
You make what you measure. This encourages companies to have their safety operators take over less often: if you tell the operators to err on the side of safety and take over more often, your car will look less safe.
I don't have a solution here, just pointing out how this actually can benefit companies that care less about safety.
You make what you measure. This encourages companies to have their safety operators take over less often: if you tell the operators to err on the side of safety and take over more often, your car will look less safe.
I don't have a solution here, just pointing out how this actually can benefit companies that care less about safety.
Sorry if this is a stupid question but what does California do with these numbers? If Wayne has 10 engagements per mile driven with 10M miles driven and Tesla has 2 per mile with 500k miles driven and Uber has 1 per mile with 1M miles driven (all made up numbers), will there be some sort of penalty for below average scorers? Is it possible to categorically say who is the best? How does this work?
It just makes the data public.
What are the consequences of not reporting those accurately, and how would anyone know?
If you are interested in learning more about George, you should listen to him talking at Mapbox's Locate conference with the Mapbox CEO, Eric Gundersen [1]. Its another example of George being George. I just happened to be called out during the talk, but overall rather enjoyed it.
[1] https://www.youtube.com/watch?v=N9WxlR1ZTZc
[1] https://www.youtube.com/watch?v=N9WxlR1ZTZc
I found his description of his company, “ghost riding for the masses”, to be particularly amusing
I prefer this one -> https://www.youtube.com/watch?v=9iUvuaChDEg
The original rap diss of Sony who was suing him, actually badass :)
The original rap diss of Sony who was suing him, actually badass :)
> If the driver looks down or away from the road for two seconds, a visual warning will appear on the Eon’s screen. After four seconds, the driver will hear an audible alert. And after six seconds, the system will disengage and begin to slow down.
The biggest use of some kind of driver assist system is something that can keep me going straight and at a constant speed on a relatively open road while I twist open a bottle cap and take a sip of a drink. Making me have a death stare out of the windshield to avoid disengagement is about as useful as standard cruise control. Except standard cruise control isn't going to be prone to making a mistake about whether or not I'm paying attention, and then stopping my car because it isn't satisfied.
The biggest use of some kind of driver assist system is something that can keep me going straight and at a constant speed on a relatively open road while I twist open a bottle cap and take a sip of a drink. Making me have a death stare out of the windshield to avoid disengagement is about as useful as standard cruise control. Except standard cruise control isn't going to be prone to making a mistake about whether or not I'm paying attention, and then stopping my car because it isn't satisfied.
Right. Keeping in lane for 2 secs when you spill your coffee or the kid screams or you do something else that makes you lose your concentration for a short while is great.
I don’t need to sleep or work for an hour during a commute. Not now not tomorrow. I can drive the damn thing or at least watch it drive itself. And like mr Hotz I believe that’s what it will be like for decades to come. My prediction is flying cars before completely self driving ones. And not some novelty one person quadcopter. I mean the proper Blade Runner ones.
I don’t need to sleep or work for an hour during a commute. Not now not tomorrow. I can drive the damn thing or at least watch it drive itself. And like mr Hotz I believe that’s what it will be like for decades to come. My prediction is flying cars before completely self driving ones. And not some novelty one person quadcopter. I mean the proper Blade Runner ones.
Now that it’s open source you can remove that part.
precisely what i was thinking.
What would the point of current self-driving car systems be? If you can't take your attention off the road anyway, it's basically a glorified, expensive cruise control.
What’s wrong with good cruise control? Any modern car is already loaded with expensive cruise control, often with distance sensing, some times with lane assist etc.
There is a huge market for this and a few players (such as Bosch Mobility) own the market. I don’t see why it’s a bad idea to try to enter this market as a startup. You don’t have to revolutionize or disrupt a market to have a viable product.
I think (like Hotz) that very good assist tech is the future, and dodgy self driving tech is a temporary fad.
Complete autonomy on geofenced highways may well be good enough soon that we can sleep or work for 90% of the commute because the autonomous system can give us 20 sec to take over when confused, rather than 1. And that can probably be solved with very cheap hardware (1-2 cameras, a front radar) compared to the huge expensive lidar and camera arrays you need for driving in complex scenarios.
So if someone can make a system that offers fantastic assist and limited but safe autonomy for 10% of the roads but 90% of the driving time - that’s not a bad thing.
I think (like Hotz) that very good assist tech is the future, and dodgy self driving tech is a temporary fad.
Complete autonomy on geofenced highways may well be good enough soon that we can sleep or work for 90% of the commute because the autonomous system can give us 20 sec to take over when confused, rather than 1. And that can probably be solved with very cheap hardware (1-2 cameras, a front radar) compared to the huge expensive lidar and camera arrays you need for driving in complex scenarios.
So if someone can make a system that offers fantastic assist and limited but safe autonomy for 10% of the roads but 90% of the driving time - that’s not a bad thing.
Ever drive 8-12 hour days for 2 days straight? Used to do it 4-5, but sometimes up to 10, times per year. I'd pay many thousands for lane-keeping on rural interstates in a heartbeat.
Yep. I love the headspace of focusing on something so simple for so long. If I weren’t going to have the experience of driving, I would just fly.
> If I weren’t going to have the experience of driving, I would just fly.
I mean, today, ditto. But for like at least 100M of the 300M+ Americans, that's not an option for various reasons...
I mean, today, ditto. But for like at least 100M of the 300M+ Americans, that's not an option for various reasons...
The difference is that glorified cruise control actually exists.
Real self driving cars, on the other hand, only exist in inspirational videos.
Real self driving cars, on the other hand, only exist in inspirational videos.
Or on dedicated tracks.
I really don’t mind dedicated tracks. Then we just have self driving taxis/buses that service predefined points. Nothing wrong with that.
A 24/7 transport system that gets you to the last mile and is cheap is an impressive achievement.
A 24/7 transport system that gets you to the last mile and is cheap is an impressive achievement.
> I really don’t mind dedicated tracks. Then we just have self driving taxis/buses that service predefined points
That's an interesting way to describe trains
That's an interesting way to describe trains
That's the hurdle that will or will not kill this wave of SDV.
You can't give bits and piece of automation for a vehicle. Mainstream tech is all about making it fool proof. Not giving warnings and calling about weighted death stats.
You can't give bits and piece of automation for a vehicle. Mainstream tech is all about making it fool proof. Not giving warnings and calling about weighted death stats.
> What would the point of current self-driving car systems be?
You’re teaching the AI in Tesla’s case. Theoretically, your drive is safer with driver augmentation safety systems in other manufacturers’ vehicles, even if they’re not contributing data back for algo refinement.
You’re teaching the AI in Tesla’s case. Theoretically, your drive is safer with driver augmentation safety systems in other manufacturers’ vehicles, even if they’re not contributing data back for algo refinement.
We are training our AI using user data as well.
https://medium.com/@comma_ai/the-half-way-point-55662cef04f2
https://medium.com/@comma_ai/the-half-way-point-55662cef04f2
You shouldn't really consider anything that isn't level 4 or above self-driving.
No level 1-3 system can get you from A to B unassisted.
No level 1-3 system can get you from A to B unassisted.
The point would be to save lives. With most accidents coming from drivers being drunk/distracted, a vastly improved cruise control could prevent many deaths.
> What would the point of current self-driving car systems be?
Baby steps, man... baby steps
Baby steps, man... baby steps
nah, if the android board fails, you get the plus of loosing the control of your vehicle...
I think that this being marketed as anything other than the next generation of cruise control is a lie. Don’t get me wrong, for that it’s absolutely great. I would pay for this kit (if my car was new enough to actually have software for it) just for the ability to be able to eat a sandwich on the way to work or cruise down the boring parts of E6 or I-10 in a more relaxed, reclined position.
I think Hotz is wrong about the predictions of self driving cars, but he doesn’t have to be right to make this business succeed in the medium term because he is filling an entirely different market niche here.
I think Hotz is wrong about the predictions of self driving cars, but he doesn’t have to be right to make this business succeed in the medium term because he is filling an entirely different market niche here.
Truly self driving car is incredibly hard. He’s right: waymo, cruise, Uber, Tesla are over-selling it. People have died in Teslas because drivers were over confident in their technology. Uber has run over people because their software couldn’t see humans in dark.
A reliable and smarter cruise control can save so many lives. I think we’re trying to jump the gun sometimes.
The number of highway deaths is insane. If software just augmented humans better and saved those people/animals we’re coming out ahead as a civilization.
A reliable and smarter cruise control can save so many lives. I think we’re trying to jump the gun sometimes.
The number of highway deaths is insane. If software just augmented humans better and saved those people/animals we’re coming out ahead as a civilization.
I'm personally very happy that homemade self-driving tech would be illegal to use on the road in my country.
How about a small startup selling that tech? Would it be legal to use?
Curious, what kind of regulations does your country have in place?
Curious, what kind of regulations does your country have in place?
I'm in Australia.
Cars need to be registered to legally be on the road, and you couldn't register a car with some homemade self-driving tech attached to it.
If you had a small startup selling the tech, I'm sure there's some painful and lengthy process to prove it's safe. But certainly if the code is user-modifiable, it wouldn't be approved.
How do other countries operate without such regulations? Certainly I've seen some crazy car mods on the road in US TV shows that would never be allowed without masses of red tape here in Australia. Even to import a normal car from Japan to Australia you need to get child restraint attachments installed/engineer approval/etc.
I don't know how a car could possibly be considered safe otherwise. Some of those "Pimp my Ride"-type show mods just don't seem safe.
Cars need to be registered to legally be on the road, and you couldn't register a car with some homemade self-driving tech attached to it.
If you had a small startup selling the tech, I'm sure there's some painful and lengthy process to prove it's safe. But certainly if the code is user-modifiable, it wouldn't be approved.
How do other countries operate without such regulations? Certainly I've seen some crazy car mods on the road in US TV shows that would never be allowed without masses of red tape here in Australia. Even to import a normal car from Japan to Australia you need to get child restraint attachments installed/engineer approval/etc.
I don't know how a car could possibly be considered safe otherwise. Some of those "Pimp my Ride"-type show mods just don't seem safe.
Sour grapes? I recall when push button elevators were installed. The union insisted an operator was still needed - who asked you 'what floor?' and then pushed that button.
Without operators, western, and indeed all global civilization would come to an end...
"Thank you for choosing Liftatron 4000. These multi-occupant vertical travel devices are almost fully autonomous, and typically require no manual control. Simply state your floor, and enjoy your journey.
Trips passing floors 3, 7, or any floor in which any resident has recently taken a shower, may revert this Fully Autonomous Lifting Device to human operated mode immediately following a n assertive audio tone. Failure to correctly operate the pop-up controls during this unlikely occurrence may result in death or serious injury to you and any other passengers. Please take the time to familiarise yourself with all operating controls that will rarely if ever be required."
Trips passing floors 3, 7, or any floor in which any resident has recently taken a shower, may revert this Fully Autonomous Lifting Device to human operated mode immediately following a n assertive audio tone. Failure to correctly operate the pop-up controls during this unlikely occurrence may result in death or serious injury to you and any other passengers. Please take the time to familiarise yourself with all operating controls that will rarely if ever be required."
[deleted]
...and then we have the issues of “non-standard” accents. https://m.youtube.com/watch?v=sAz_UvnUeuU
Point taken, but it doesn't mean that the scope and requirements of elevator travel are directly comparable to road travel. Car driving has not only more axes of allowed movement, but several magnitudes more variables. The skill floor for operating an elevator has more or less been the same, and easy enough for someone without any training. But driving a car today still requires weeks of education, as it did decades ago.
> I recall when push button elevators were installed.
The designs for some of the first push button elevator circuitry was really cool tech.
It was a complicated problem because elevators could appear and disappear in a given building at will. Some would appear and operate within a building for only a few minutes. Otherwise could leave seconds after they arrived without signaling their departure. The circuitry had to account for this unreliability within the "network" of all possible elevators within a given city block.
The circuity in each elevator car also had to handle collision avoidance as new elevators appeared within the same shaft as other elevator cars that were already moving in that shaft.
It's fascinating to look at the designs:
www.archive.org/apples-and-oranges.html
Self-driving cars seem like a cake walk by comparison.
The designs for some of the first push button elevator circuitry was really cool tech.
It was a complicated problem because elevators could appear and disappear in a given building at will. Some would appear and operate within a building for only a few minutes. Otherwise could leave seconds after they arrived without signaling their departure. The circuitry had to account for this unreliability within the "network" of all possible elevators within a given city block.
The circuity in each elevator car also had to handle collision avoidance as new elevators appeared within the same shaft as other elevator cars that were already moving in that shaft.
It's fascinating to look at the designs:
www.archive.org/apples-and-oranges.html
Self-driving cars seem like a cake walk by comparison.
Elevators gaining sentience is a little to Hitchiker's Guide to me. Would be frightening.
What would the elevator operator do if the elevator got stuck between floors, or kept going past the selected floor? Was there a hand brake for the elevator that only they could use?
They probably couldn't do any more than they could do when the manually operated elevator got stuck between floors.
I think the real reason that elevator operators stayed on for a period of time is the same reason that some older people sometimes reject new technology, such as smartphones or in-home digital assistants, even when it ostensibly makes a lot of things easier. The same thing probably happened with push-button elevators. People didn't want to have to think about pushing buttons when they were used to just stepping into the elevator and telling the operator what floor they needed, or more likely the operator already knew.
I think the real reason that elevator operators stayed on for a period of time is the same reason that some older people sometimes reject new technology, such as smartphones or in-home digital assistants, even when it ostensibly makes a lot of things easier. The same thing probably happened with push-button elevators. People didn't want to have to think about pushing buttons when they were used to just stepping into the elevator and telling the operator what floor they needed, or more likely the operator already knew.
When push-button elevators were installed, they worked. Can you imagine if the pitch had been, “sometimes this elevator will kill the odd person, thanks for helping us work out the bugs.” As a bonus, the actual working elevator might not emerge until decades after installation began, and the early models only worked on alternate Wednesday’s.
They’re still working out the kinks in the elevator in my old office; it spent 6 months of the year out of order every day being repaired, and numerous people got trapped in it. Eventually, people just stopped using it, even when it was “fixed”.. because it never really was.
It behaved like a chronic dud. A bad self driving car would probably develop the same reputation.
It behaved like a chronic dud. A bad self driving car would probably develop the same reputation.
This might be a valid point if the current system of driving wasn’t already killing millions of people per year.
"Millions" is an overstatement. Annual deaths worldwide are around 1.25 million. And deaths in the U.S. have declined significantly from their peak in the '70s.
All of that begs the question of whether autonomous cars will save lives. It is always asserted that it will "because humans are such bad drivers compared to computers," but there's no actual, real-world data to back up that claim. Oh yes, some of these vehicles have been tested with millions of road miles, but that is far from the scale of billions or trillions of road miles on every kind of road under all conditions where people travel currently.
All of that begs the question of whether autonomous cars will save lives. It is always asserted that it will "because humans are such bad drivers compared to computers," but there's no actual, real-world data to back up that claim. Oh yes, some of these vehicles have been tested with millions of road miles, but that is far from the scale of billions or trillions of road miles on every kind of road under all conditions where people travel currently.
Of course, it's not the elevator buttons that kill, it's the entrance.
I personally do not want a smartphone with a 3D printed case and heatsink controlling my car. Do you hear that? It's the sound of a critical flaw in your product, no matter what you do technically you are never going to have a mass appeal product that becomes more than the nerds who hang out in your Slack channel.
But it's fine for a lifestyle business I suppose. You can market to them and sell them over-priced odb2 fit-bit-for-your-car dongles. Until Honda or whoever comes out with a "good enough" consumer grade "auto-pilot" (whatever that means these days because you still have to watch the car like a teenage driver), you'll loose a huge swatch of users right there who are only spending time on it to be on the cutting edge of tech and to look cool for their friends. Once it becomes a blender, those users will leave and go on to the next thing that grabs their attention in a banner ad. Or else people will get bored, loose interest after the first few joy rides with their friends. I'd imagine retention would be difficult in a business like that which takes a lot of sustained effort and engagement from the users.
The only possible path forward in this market is to partner with an OEM (car manufacture vroom vroom), have access to users, or make some kind of must-have tech that you can then con one of the real big money backed self-driving companies into buying. I've seen a bunch of web apps popping up lately that offer to annotate your training data, those seem like a lot better pump and dump tech companies than this dashcam that plays Pandora and runs some cool python PID controller.
But it's fine for a lifestyle business I suppose. You can market to them and sell them over-priced odb2 fit-bit-for-your-car dongles. Until Honda or whoever comes out with a "good enough" consumer grade "auto-pilot" (whatever that means these days because you still have to watch the car like a teenage driver), you'll loose a huge swatch of users right there who are only spending time on it to be on the cutting edge of tech and to look cool for their friends. Once it becomes a blender, those users will leave and go on to the next thing that grabs their attention in a banner ad. Or else people will get bored, loose interest after the first few joy rides with their friends. I'd imagine retention would be difficult in a business like that which takes a lot of sustained effort and engagement from the users.
The only possible path forward in this market is to partner with an OEM (car manufacture vroom vroom), have access to users, or make some kind of must-have tech that you can then con one of the real big money backed self-driving companies into buying. I've seen a bunch of web apps popping up lately that offer to annotate your training data, those seem like a lot better pump and dump tech companies than this dashcam that plays Pandora and runs some cool python PID controller.
geohot has always been equal parts brains and bombast.
Driver assist is a good way to get people comfortable with automated driving. The only way autonomous vehicles will ever really show any benefit is if communities, municipalities and eventually, governments start rolling them out en masse and incentivising their purchase; SDVs driving together is the only way to finally eliminate traffic related deaths and make travel smooth and fast. As long as you have computers trying to predict and/or respond to erratic human behavior, you're fighting a losing battle; it doesn't matter how many times you rehearse it you'll hit a certain plateau where the correct decisions simply can't be determined.
[deleted]
It seems likely that autonomy will not be first adopted by the US but instead in the 'wild west' of BRICs - Brazil, Russia, India or China.
Also maybe in a country small, rich, authoritarian, and organized enough to build infrastructure around it and limit liability like Singapore or Luxembourg.
And oh look, a cheap and open source solution that gets you started in CommaAI. I think they (or a fork of them) are quite good candidates for the first system to reach one trillion miles driven. There will no doubt be carnage. But the benefits of self driving are going to be first on display outside of US markets.
Also maybe in a country small, rich, authoritarian, and organized enough to build infrastructure around it and limit liability like Singapore or Luxembourg.
And oh look, a cheap and open source solution that gets you started in CommaAI. I think they (or a fork of them) are quite good candidates for the first system to reach one trillion miles driven. There will no doubt be carnage. But the benefits of self driving are going to be first on display outside of US markets.
Isn't a key part of openpilot, the vision related code, still closed source?
This guy is all talk and represents one problem with Silicon Valley. There are a lot of people now who can talk the talk and get the attention needed to get money but with no ability to deliver. Stop giving him attention until he actually shows something that matches even a little bit of what he is promising.
People who know what they're doing technically are deeply aware of the issues with their technology. Anyone in this area knows how hard it is to make these vehicles work. He is finding out the reality of what he promised two years late and is spinning it to get more attention.
People who know what they're doing technically are deeply aware of the issues with their technology. Anyone in this area knows how hard it is to make these vehicles work. He is finding out the reality of what he promised two years late and is spinning it to get more attention.
You are aware we are giving away open source software that will control your car, right? It's online today! And it's on par with the best systems money can buy, Tesla Autopilot and GM Super Cruise.
https://github.com/commaai/openpilot
No ability to deliver?
https://github.com/commaai/openpilot
No ability to deliver?
Not disputing your claim, but how did you determine the comma Ai system was on par with that of autopilot and super cruise?
In general, are there any standardized tests for self driving cars, like car safety ratings but for the driving part? Basically, is there even a way to measure how good a driving system is?
In general, are there any standardized tests for self driving cars, like car safety ratings but for the driving part? Basically, is there even a way to measure how good a driving system is?
Not really. It's mostly subjective, but a good metric is time between user action needed. We did 2 hours of press demos on highways without a single one. (usually it's about an hour)
In addition to us, we have a community of users on slack with thousands of miles of experience with each system. I think most would generally agree with the quality assessment. You'll also find a bunch of YouTube videos comparing them.
In addition to us, we have a community of users on slack with thousands of miles of experience with each system. I think most would generally agree with the quality assessment. You'll also find a bunch of YouTube videos comparing them.
So you test the system with human drivers for a few miles, and if none of them dies then it's secure ? You actually rely on the subjectivity of uninformed users to assess the reliabilty of a safety-critical system ?
>> Not really. It's mostly subjective.
No. The quality of a safety-critical system is not subjective.
>> I think most would generally agree with the quality assessment.
You are not qualified to make that claim, and neither are your users.
>> Not really. It's mostly subjective.
No. The quality of a safety-critical system is not subjective.
>> I think most would generally agree with the quality assessment.
You are not qualified to make that claim, and neither are your users.
Self driving systems is in its very early days and commaai is one of the open ones i.e you can take the software and verify under the hood. commaai isn’t also lying that it’s a glorified cruise control.
Unlike crash safety, crash and burn, topple etc, there are no open tests or third party certifications that can verify that car model X is subjectively this safe and passes this scenarios.
The big cos are notoriously secretive and throwing a lot of marketing money to create hype.
I’m just glad that certain states have allowed such cars to be tested on the streets.
We have to start somewhere.
Unlike crash safety, crash and burn, topple etc, there are no open tests or third party certifications that can verify that car model X is subjectively this safe and passes this scenarios.
The big cos are notoriously secretive and throwing a lot of marketing money to create hype.
I’m just glad that certain states have allowed such cars to be tested on the streets.
We have to start somewhere.
A good metric would be collision avoidance in a multitude of realistic tests, simulating all kinds of dynamic obstacles and vehicle error modes.
Tesla and Uber did seriously disappoint. It's even below that but still. I'm dubious about the capabilities of comma.ai system and calling it on par with Tesla after dissing it is a bit disingenuous.
I still consider geohot too egotistical in his approach, and that it will not bear fruits. That big commercial companies are pulling bad tricks is one thing but that doesn't mean one lone motivated (and talented) wolf can reach the goal.
Try to poach some waymo guys or some robotics to team up.
I still consider geohot too egotistical in his approach, and that it will not bear fruits. That big commercial companies are pulling bad tricks is one thing but that doesn't mean one lone motivated (and talented) wolf can reach the goal.
Try to poach some waymo guys or some robotics to team up.
comma.ai poached an engineer from Tesla[0]
I think they are very capable. Maybe their tech is not ready now, but who knows in 2-5 years where they will be.
[0] https://www.theverge.com/2016/4/18/11454858/george-hotz-self...
I think they are very capable. Maybe their tech is not ready now, but who knows in 2-5 years where they will be.
[0] https://www.theverge.com/2016/4/18/11454858/george-hotz-self...
I meant waymo because they have a less commercial approach compared to Tesla (which might taint this engineer view [or not])
I believe the only way out of the security pit is to double down on rigorous physics and not rapid ML
I believe the only way out of the security pit is to double down on rigorous physics and not rapid ML
Waymo is the pinnacle of commercialized FSD. Have you seen their ads with actors? Tesla does ZERO marketing.
Waymo is taking their sweet time, Tesla is aggressively talking about autopilot. Far from Zero.
Send me a link to a Tesla Autopilot video or graphic brochure that is produced by the Tesla marketing team. I'll wait..
Meanwhile there are a few of these: https://www.youtube.com/watch?v=3HrN12WG-2Q
Meanwhile there are a few of these: https://www.youtube.com/watch?v=3HrN12WG-2Q
what about musk tweets about it and vehicle presentations about autopilot 2 ?
One example that I like to give is a downed power line. This is something that happens often. Try solving it with rigorous physics.
A purely physical approach is just a design paradigm in how to use other sensors (with or without ML), having this means uncertainty would be correlated with momentum and anything causing doubt (a power line down) would mean reducing the internal energy of the system to give ability to either stop to a halt, enter a safe configuration or assess the situation deeper/differently to decide.
So far what we see about Tesla and the likes is that they jumped early on the ML fad as very naive feedback loops on basic car controls (estimate lanes -> center the car. estimate obstacle distance -> adapt speed). It's not physics first.
So far what we see about Tesla and the likes is that they jumped early on the ML fad as very naive feedback loops on basic car controls (estimate lanes -> center the car. estimate obstacle distance -> adapt speed). It's not physics first.
The problem is, you'd be hitting all these pesky corner cases, with some of them requiring active avoidance - downed power lines, water on the road, vehicles on fire, other traffic participants. My guess, it is hard to engineer a system that could handle the long tail of making decisions under uncertainty with "purely physical approach". You'll run out of complexity budget.
it's certainly full of dimensions. I may be a ideal-extremist here, and business probably doesn't like being overly cautious, but at least that would be a vehicle I'd trust no to endanger anybody (in or out), even if it looks underperforming compared to autopilot~.
Well, usually driving over one with rubber tires causes no problem. One in the air would be detectable, no?
It is hard to even recognize it. On the road it looks exactly like a crack. For a case of a low hanging one - it is invisible by LIDAR or radar, on the camera the difference between a regular one and a low hanging one is very subtle.
And the requirement is not just braking action, like with a railroad crossing it is active avoidance.
https://www.pge.com/en_US/safety/electrical-safety/what-to-d...
My point is - corner cases like these tend to create "IF" statements in a "pure physics model" that is used for decision making under uncertainty. And engineering such models without "ML fad" is difficult.
And the requirement is not just braking action, like with a railroad crossing it is active avoidance.
https://www.pge.com/en_US/safety/electrical-safety/what-to-d...
My point is - corner cases like these tend to create "IF" statements in a "pure physics model" that is used for decision making under uncertainty. And engineering such models without "ML fad" is difficult.
Software that can't pass automotive safety standards.
> Hotz has been teasing a $1,000 after-market kit called Comma One that would let customers transform their dumb cars into smart, self-driving ones. But after a sternly worded letter from the US National Highway Traffic Safety Administration last year, Hotz abandoned those plans
http://www.reuters.com/article/us-selfdriving-safety-idUSKCN...
Sounds like you have a nice demo in a controlled known environment. It's a far way off the promise made previously about building a self driving car, a claim which is now being retracted. Best of luck turning it into a product. Watch out for those edge cases.
> Hotz has been teasing a $1,000 after-market kit called Comma One that would let customers transform their dumb cars into smart, self-driving ones. But after a sternly worded letter from the US National Highway Traffic Safety Administration last year, Hotz abandoned those plans
http://www.reuters.com/article/us-selfdriving-safety-idUSKCN...
Sounds like you have a nice demo in a controlled known environment. It's a far way off the promise made previously about building a self driving car, a claim which is now being retracted. Best of luck turning it into a product. Watch out for those edge cases.
It is an impressive hack. But you are not building something that is useful. Build a small component that follows safety standards - this will be useful.
Ask our users if they feel similarly about it not being useful. https://slack.comma.ai
Yes, I understand that. And my perspective is different from your users. It is that of someone who is seeing (and attempting to help) big automotive companies. Companies in the shadow of Waymo, that had recently reached $135B valuation. It is also that of someone, who had been hacking code for medical equipment at 13, some 25 years back.
Do as you wish.
edit: correction. out of your project we are probably having more hackers that understand at least some aspect of the problem - you still might be doing something very useful.
Do as you wish.
edit: correction. out of your project we are probably having more hackers that understand at least some aspect of the problem - you still might be doing something very useful.
Help automotive companies? They have been sitting idle for decades and would rather spend their resources on flashy marketing, rather than innovate.
Car companies, even luxury brands like MB, BMW, and Audi are akin to cell phone companies running Symbian, Windows Mobile, Palm OS, and Symbian before the iPhone came along.
Now we have innovators like Tesla and open source DIY stuff from comma.ai which can churn out great tech in a very short amount of time, without the need for getting a new year model vehicle.
Car companies, even luxury brands like MB, BMW, and Audi are akin to cell phone companies running Symbian, Windows Mobile, Palm OS, and Symbian before the iPhone came along.
Now we have innovators like Tesla and open source DIY stuff from comma.ai which can churn out great tech in a very short amount of time, without the need for getting a new year model vehicle.
Waymo's valuation is based on hype. They have no revenue.
We aren't trying to help big automotive companies, they are our competition. We are trying to disrupt them.
We aren't trying to help big automotive companies, they are our competition. We are trying to disrupt them.
So now we'll have weekend "hackers" downloading this open source software, tweaking it, and hurling their cars down the road at high speed. Great.
The system also seems to have a single front facing camera. How will this detect for example a cyclist riding along the car and prevent the car from turning into him?
The system also seems to have a single front facing camera. How will this detect for example a cyclist riding along the car and prevent the car from turning into him?
You seem to have pretty strong opinions without even actually taking the time to look into comma.ai and how it works.
All it takes is 5 minutes to know that it actually uses the built-in safety systems in cars like ADAS, LKAS, ACC. Hence, they only support only specific make/models. It's not just the "single front facing camera".
All it takes is 5 minutes to know that it actually uses the built-in safety systems in cars like ADAS, LKAS, ACC. Hence, they only support only specific make/models. It's not just the "single front facing camera".
None of these built-in safety systems is design to work without a human driver having the actual responsibility. This is why e.g. ACC is a "comfort" system but not a "safety" system. Even systems like AEB don't come with a guarantee. They just might help.
>None of these built-in safety systems is design to work without a human driver having the actual responsibility
You are exactly right! which is why NEITHER comma.ai, Tesla's Autopilot, Nissan's Propilot, and GM Supercruise are being marketed as such i.e. only as L2/Partial Automation at the moment.
You are exactly right! which is why NEITHER comma.ai, Tesla's Autopilot, Nissan's Propilot, and GM Supercruise are being marketed as such i.e. only as L2/Partial Automation at the moment.
But cars that have any cameras likely already have a front facing one. So if they support only cars that have other cameras it can use (and assuming now that autonomous driving requires camera/or lidar view) why do they even need the camera? Did he drive that civic autonomously without lidars and without it knowing what’s behind it?
First, lidar is not a silver bullet and are still very cost prohibitive. Look at Mobileye and Supercruise which are both pretty good and only use radar, cameras and HD maps.
Second the camera is for the vision system and comma.ai hooks in the car via CANBUS to interpret the OEM radar signals.
Second the camera is for the vision system and comma.ai hooks in the car via CANBUS to interpret the OEM radar signals.
Previous discussion: https://news.ycombinator.com/item?id=17522766
Thanks. Since that submission didn't get too much attention, we've moved those comments here.
Brilliant! Pure AI is a dead end street. Turing machines cannot increase their minimal Kolmogorov sufficient statistic, so will always be unable to learn more information than is put into them originally. Humans are the only known information creators, and so only hybrid human/AI systems can work in the real world.
>Humans are the only known information creators
What do you think whale-sounds and bird-songs are?
What do you think whale-sounds and bird-songs are?
If whale-sounds and bird-songs are information, and human intelligence is the only thing that can create information, then the conclusion is whales and birds were created by some kind of intelligence.
So it is your belief that whales and birds and all other animals were created by humans?
And if so, then why would a human not be able to create an AI that is capable of creating information?
And if so, then why would a human not be able to create an AI that is capable of creating information?
I think they were created by God. God is a meta-intelligence, an intelligent being that can create other intelligent beings. Our intelligence is only capable of creating mechanical beings, which are necessarily not intelligent.
The second question does not follow. If intelligence is defined by the ability to do something non-algorithmic (i.e. create information), then AI can never be intelligent.
The second question does not follow. If intelligence is defined by the ability to do something non-algorithmic (i.e. create information), then AI can never be intelligent.
Obviously entropy!
> Humans are the only known information creators
No. Evolution created RNA, DNA, all kinds of animal languages, both sound- and chemical-based.
No. Evolution created RNA, DNA, all kinds of animal languages, both sound- and chemical-based.
Any proof this is true? Sounds like conjecture to me.
Interesting tidbit: Hotz actually means thief in a few languages.
I actually don’t believe this guy. He may be brilliant, but there are quite a few brilliant people working on this problem - so no.
I actually don’t believe this guy. He may be brilliant, but there are quite a few brilliant people working on this problem - so no.
> Interesting tidbit: Hotz actually means thief in a few languages.
You've piqued my inner language geek, but cursory Googling / Google Translate hasn't given me any luck. What language(s) would that be?
You've piqued my inner language geek, but cursory Googling / Google Translate hasn't given me any luck. What language(s) would that be?
hoţ (pronounced hotz) is Romanian for thief. Not sure what the etymology is but it could be a Slavic loan word.
Edit: looked it up... Unknown etymology. I guess if you count Moldovan as a separate language then it means thief in at least two languages...
Edit: looked it up... Unknown etymology. I guess if you count Moldovan as a separate language then it means thief in at least two languages...
I found https://blogulblog.wordpress.com/z-postari-pe-alte-teme-ling... which lists it as coming from Romani. However, I find that a bit weird, as it's čor – hoț. The first letter being https://en.wikipedia.org/wiki/Voiceless_postalveolar_affrica...
So, yes, cioara could come from there, but not sure how you get from there to hot. And, DEX lists cioara as coming from the Albanian sorrë.
So, yes, cioara could come from there, but not sure how you get from there to hot. And, DEX lists cioara as coming from the Albanian sorrë.
Romanian
I think the parent maybe have been attempting a joke. I also could not find any languages where Hotz means thief.
Interestingly it means cold in Basque and hotspot in Estonian.
Interestingly it means cold in Basque and hotspot in Estonian.
If you google it, it comes up as the first result :) :)
https://www.google.com/search?q=hotz+thief
https://www.google.com/search?q=hotz+thief
Your comment here comes up as the first result for me. The second and third results are other comments in this thread. None of the other results on page 1 or 2 have any relevance, so I'd say it's pretty obscure.
What languages are you referring to?
What languages are you referring to?
> If you google it, it comes up as the first result :) :)
As another user has mentioned, the only results in your query are links to this thread. Perhaps your results are different than mine?
As another user has mentioned, the only results in your query are links to this thread. Perhaps your results are different than mine?
Hotz always reminds me of those kids using Adderall in the TV series Silicon Valley.
Their low level control is done in Python, on top of Android, on a smartphone. No further comment, your honor.
https://github.com/commaai/openpilot/tree/devel/selfdrive/co...