Exploiting Wi-Fi Stack on Tesla Model S(keenlab.tencent.com)
keenlab.tencent.com
Exploiting Wi-Fi Stack on Tesla Model S
https://keenlab.tencent.com/en/2020/01/02/exploiting-wifi-stack-on-tesla-model-s/
66 comments
So this exploit is not specific to Tesla but rather any Linux 2.6.36 (and maybe up) that uses the 88W8688 (and maybe others), which could be a pretty common combination. And all that's required is an AP running the exploit, everything is automatic. Very impressive.
Definitely a common combination. The OLPC XO-1.5 was using this hardware 10+ years ago. It sounds like NX protections defeat the attack, and it looks like they went in soon after this kernel, around 2.6.38.
Q: How come Teslas are running kernels from 2008?
It appears that: Tesla bought this all-in-one ARM board from Parrot, and Parrot gave them a 2.6.36 kernel with thousands of custom patches in, and Tesla's not in much of a position to port it to a newer kernel, and then Parrot presumably went on to building other stuff and ignored keeping this kernel up to date and now we're here.
They (probably Parrot) might have updated it in the last year since the vulnerability was discovered though, it's hard to tell.
They (probably Parrot) might have updated it in the last year since the vulnerability was discovered though, it's hard to tell.
If it's not in mainline kernel - it's abandonware.
To be fair, that is a 10 year old kernel at this point.
> As Linux Kernel 2.6.36 does not support NX
Does the Model S really run 2.6.36, first released ten years ago? I'm sure they're using a stable branch and keeping up with security patches in general [1], but that's still shocking to me.
[1]: although not 100% of security patches are going to end up in the stable@ queue or have their effect on 2.6.36 considered.
Does the Model S really run 2.6.36, first released ten years ago? I'm sure they're using a stable branch and keeping up with security patches in general [1], but that's still shocking to me.
[1]: although not 100% of security patches are going to end up in the stable@ queue or have their effect on 2.6.36 considered.
It looks like a car would need to be over 2 years behind Tesla's OTA updates for it to still have version 2.6.36 and be vulnerable to this exploit.
https://electrek.co/2017/06/30/tesla-new-linux-kernel-update...
EDIT: See nils2014's comments below. The WiFi chip is connected to the entertainment system which received an updated kernel, but it is seems like the kernel on the WiFi chip itself may not have been updated.
https://electrek.co/2017/06/30/tesla-new-linux-kernel-update...
EDIT: See nils2014's comments below. The WiFi chip is connected to the entertainment system which received an updated kernel, but it is seems like the kernel on the WiFi chip itself may not have been updated.
The kernel version of the information center is irrelevant.
The Wi-Fi stack is running on a separate module (the Parrot module, as mentioned in the article), which runs its own obsolete linux version.
The Wi-Fi stack is running on a separate module (the Parrot module, as mentioned in the article), which runs its own obsolete linux version.
I wonder if there's a way to find out which kernel version Parrot runs today.
Do they even still exist? It's impossible to find anything on these modules beyond FCC filings; some pixelated datasheets show the modules with "www.parrot.com" printed on them but going as far back as 2017 in the WWW archive, that site belongs to some drone startup.
What do you mean by "still exist"? Obviously a vuln was found in one that was inside a Model S, reported to Tesla and fixed.
Marvel fixed the bug in their firmware and apparently Tesla shipped some sort of update, I don't see anything that necessarily requires Parrot, the obscure maker of these modules, to do anything.
Oh! You misunderstood my question: I meant to ask whether Tesla's latest software update continues to ship a 2.6.36 kernel on Parrot, or a newer kernel instead, as they do for their other modules.
(Tesla uses the name Parrot to refer to this system in the car.)
A reason for them not to still be on 2.6.36 is that it's deeply exploitable.
(Tesla uses the name Parrot to refer to this system in the car.)
A reason for them not to still be on 2.6.36 is that it's deeply exploitable.
I think the root of the confusion here is that you think Parrot is just a Tesla nickname. But Parrot is the name of the manufacturer of the fully-integrated WiFi+Bluetooth module so Tesla likely have nothing to do with the firmware that runs on that module with the ancient kernel.
As far as I can tell, Parrot-the-module-maker split from Parrot-the-drone-company and is now http://parrot-faurecia-automotive.com
As far as I can tell, Parrot-the-module-maker split from Parrot-the-drone-company and is now http://parrot-faurecia-automotive.com
Oh, I didn't get that impression. Keen's previous Black Hat paper made it sound like Tesla is dual-maintaining 2.6.36 and 4.x kernels, because they're unable/unwilling to upgrade from 2.6.36 on the Parrot board but are still issuing kernel fixes for it.
Looking at the datasheet some more, it does seem like the intention is for Tesla (or some other customer) to develop userland software and integrate that with Parrots SDK into a final firmware image for the module. But in that case, their situation is much the same as it is with the nVidia Tegra entertainment board: you are totally dependent on nVidia and Parrot to provide the basic infrastructure from bootloader to a Linux kernel with drivers to interface all the functionality they are selling you (a BSP). None of their hardware has any part upstreamed.
In Teslas position, as the ultimate vendor that is shipping this stuff on thousands of cars, you might decide that your supplier isn't forthcoming with security updates and decide to backport some isolated fixes to the kernel, but an upgrade from 2.6 to say even 4.4 is entirely out of the question; there are thousands of lines of hacked-together proprietary vendor code in those kernels where just going up a minor version will break the build. For a 2.6 based system in particular, you would essentially need to convert the entire board support to a device tree based system.
In Teslas position, as the ultimate vendor that is shipping this stuff on thousands of cars, you might decide that your supplier isn't forthcoming with security updates and decide to backport some isolated fixes to the kernel, but an upgrade from 2.6 to say even 4.4 is entirely out of the question; there are thousands of lines of hacked-together proprietary vendor code in those kernels where just going up a minor version will break the build. For a 2.6 based system in particular, you would essentially need to convert the entire board support to a device tree based system.
This is fine, but not an answer to the actual question of whether Tesla is still shipping 2.6.36 on a Model S's Parrot system today. Parrot-the-company could have given an updated kernel in the year or so since the vulnerability was reported, or like you say Tesla could have taken on the mammoth task of maintaining an out of tree arch port themselves. It would be useful to know whether this happened, since if it's still running 2.6.36 it would be reasonable to consider the devices as extremely vulnerable in the wild today.
> but an upgrade from 2.6 to say even 4.4 is entirely out of the question
We did this kind of transition at OLPC ten or so years ago, helping Marvell upstream enough of their arch support code for a new ARM SoC to move us from their private 2.6 branch to latest public upstream (including the move to device tree), and with only a few kernel engineers working on it. Tesla surely has many times more resources; it's not entirely out of the question, but it's true that it's probably not economical for a minor support chip. Although, it's the minor support chip that's connected to the outside world..
> but an upgrade from 2.6 to say even 4.4 is entirely out of the question
We did this kind of transition at OLPC ten or so years ago, helping Marvell upstream enough of their arch support code for a new ARM SoC to move us from their private 2.6 branch to latest public upstream (including the move to device tree), and with only a few kernel engineers working on it. Tesla surely has many times more resources; it's not entirely out of the question, but it's true that it's probably not economical for a minor support chip. Although, it's the minor support chip that's connected to the outside world..
> there are thousands of lines of hacked-together proprietary vendor code in those kernels where just going up a minor version will break the build. For a 2.6 based system in particular, you would essentially need to convert the entire board support to a device tree based system.
What I don't get with these cases at all: by the GPL manufacturers are forced to provide the full (!) Linux kernel source code, same goes for u-boot. But while Tesla at least seems to provide source code (https://github.com/teslamotors), why the fuck and how can Android phone and other embedded device makers get away with not publishing anything?
Let's just take two examples of hardware that I know run Linux because I managed to root them:
- Sony A7S2 camera: the firmware is at https://www.sony.com/electronics/support/downloads/00016077, but no mention at all about contained open-source code, licenses, or build instructions
- HP Z2100 24-inch plotter: firmware is at https://support.hp.com/sg-en/drivers/selfservice/hp-designje... (you have to select Windows XP 32-bit to see the firmware download, even though the firmware can be installed via anything capable of running a browser), and again, no mention of anything regarding Linux.
What I don't get with these cases at all: by the GPL manufacturers are forced to provide the full (!) Linux kernel source code, same goes for u-boot. But while Tesla at least seems to provide source code (https://github.com/teslamotors), why the fuck and how can Android phone and other embedded device makers get away with not publishing anything?
Let's just take two examples of hardware that I know run Linux because I managed to root them:
- Sony A7S2 camera: the firmware is at https://www.sony.com/electronics/support/downloads/00016077, but no mention at all about contained open-source code, licenses, or build instructions
- HP Z2100 24-inch plotter: firmware is at https://support.hp.com/sg-en/drivers/selfservice/hp-designje... (you have to select Windows XP 32-bit to see the firmware download, even though the firmware can be installed via anything capable of running a browser), and again, no mention of anything regarding Linux.
(Note that this article is only talking about the dashboard entertainment machine, which probably isn't the one the wifi chip's connected to. Perhaps that one got slower updates.)
> I'm sure they're using a stable branch and keeping up with security patches
Surely you must be joking... The last kernel 2.6.x LTS version to be maintained is 2.6.32.71, and its maintenance ended in March 2016 [1].
No Security patches were released for 2.6.x since then.
[1] https://en.wikipedia.org/wiki/Linux_kernel_version_history#R...
Surely you must be joking... The last kernel 2.6.x LTS version to be maintained is 2.6.32.71, and its maintenance ended in March 2016 [1].
No Security patches were released for 2.6.x since then.
[1] https://en.wikipedia.org/wiki/Linux_kernel_version_history#R...
You are right, but AFAIK they are running CentOS and Redhat have been back-porting security and fixes to major bugs at least until recently. CentOS 6 is in extended support phase 2 [1] Hopefully they move to CentOS 8 soon. I have no idea how this is implemented or if that parrot module has a flashable firmware. It is interesting that they say there is no support for NX. CentOS 6 which is based on the 2.6 kernel does support it. [2] That is not to say that things were compiled with the right options.
[1] - https://access.redhat.com/support/policy/updates/errata
[2] - https://fedoraproject.or/wiki/Security_Features_Matrix
[ EDIT ] I was advised it is not running CentOS and that may be another system in the vehicle.
[1] - https://access.redhat.com/support/policy/updates/errata
[2] - https://fedoraproject.or/wiki/Security_Features_Matrix
[ EDIT ] I was advised it is not running CentOS and that may be another system in the vehicle.
That's incorrect. The Parrot module has only 32MB RAM and is running a custom marvell buildroot-style linux distribution [1].
You're confusing the linux running on the information center, with the linux running on the Parrot board.
[1] https://fccid.io/RKXFC6050W/Users-Manual/user-manual-1707044
You're confusing the linux running on the information center, with the linux running on the Parrot board.
[1] https://fccid.io/RKXFC6050W/Users-Manual/user-manual-1707044
And the information center runs Ubuntu, not CentOS.
If Parrot's still running 2.6.36, that's a really really bad sign for its security. You would have to predict more remote vulns on the way.
If Parrot's still running 2.6.36, that's a really really bad sign for its security. You would have to predict more remote vulns on the way.
Agreed. Having all these systems with all the different distros / builds of linux sounds very difficult to maintain. I am not comfortable with cars that are remotely accessible as it is, especially if they have any self driving abilities. Elon also wants to make brain implants for people that will be remotely accessible.
I am probably thinking of the system that controls the driving, which I am 99% sure is CentOS 6. Reaching out to someone I know there.
I am probably thinking of the system that controls the driving, which I am 99% sure is CentOS 6. Reaching out to someone I know there.
Thankyou for the correction.
There are many 10+ year old linux kernels in embedded devices all over the place, and much less secure systems and services on top of that. It's often called the internet of shitty things.
Right, but it's surprising that Tesla would be one of those vendors since they have an ongoing obligation to the security of their devices.
It looks like their open source disclosure of the kernel source is a 4.4 kernel, so something weird is going on there. Maybe they got off 2.6 since the time the vulnerability was discovered?
I see something saying they moved to 4.4 in the 8.1 update in June 2017, which is confusing because it suggests the vulnerability was never live.
But I also see they have multiple CPUs running Linux and don't update the kernel on all of them at the same time, so perhaps the CPU running the 8686 driver wasn't updated?
It looks like their open source disclosure of the kernel source is a 4.4 kernel, so something weird is going on there. Maybe they got off 2.6 since the time the vulnerability was discovered?
I see something saying they moved to 4.4 in the 8.1 update in June 2017, which is confusing because it suggests the vulnerability was never live.
But I also see they have multiple CPUs running Linux and don't update the kernel on all of them at the same time, so perhaps the CPU running the 8686 driver wasn't updated?
A former Tesla software engineer posted a thread on twitter a few months ago describing their development/release process and it was a total mess of hacky bad practices that barely worked at all.
It does not seem like software is much of a priority for them.
It does not seem like software is much of a priority for them.
Obviously delivering cars is their priority. I’m not sure what else you would expect?
Well they are pretending to be a tech company to wall street and consumers, so someone is lying.
Many (major) tech companies have just as bad internal engineering infrastructure.
What's worse is they are very resistant to change.
Think Banks and Windows XP. Except without Microsoft drawing a line in the sand and saying that they're no longer offering even paid support for it.
If the change doesn't bring in value for the shareholders (or business) there will be no change.
Think Banks and Windows XP. Except without Microsoft drawing a line in the sand and saying that they're no longer offering even paid support for it.
If the change doesn't bring in value for the shareholders (or business) there will be no change.
[deleted]
Most car manufacturers are reactive, like the Jeep hack a few years ago. Cyber Security should be on the forefront of their radar, but takes a back seat. So having outdated software is not that big of a surprise. I have a 3 year old phone that functions fine, but i might need to get a new one since it is full of security holes and no updates.
I find myself conflicted since i love the technology behind the new cars, but my paranoia about the lack of security and the inability of myself to do anything about it.
If only they had cameras in the cars so while they are applying the brakes when you are on the interstate they can collect that classic picture of horror on your face....
I find myself conflicted since i love the technology behind the new cars, but my paranoia about the lack of security and the inability of myself to do anything about it.
If only they had cameras in the cars so while they are applying the brakes when you are on the interstate they can collect that classic picture of horror on your face....
>If only they had cameras in the cars so while they are applying the brakes when you are on the interstate they can collect that classic picture of horror on your face....
They already do. IIRC Model 3 has about 6 external cameras and one internal one. However, the internal one is currently disabled and isn't used for anything, which is a subject to change for when they decide what to do with it and push it in an update.
If you are looking for some wtf road videos, we are bound to see more of them soon due to the most recent update Tesla pushed. It added the "save on honk" option, which (if enabled) saves some footage right before the honk, as well as about 5 seconds after the honk. With how much spicy footage the sentry mode update has provided in less than a year, I expect nothing less from the "save on honk" update.
As a sidenote, it isn't as bad as it seems in terms of privacy, because all videos (on-honk ones, or manual ones, or the sentry mode accidents) only get saved locally on your HDD/SSD/usb-drive/etc (whatever you have inserted into one of the front USB ports).
They already do. IIRC Model 3 has about 6 external cameras and one internal one. However, the internal one is currently disabled and isn't used for anything, which is a subject to change for when they decide what to do with it and push it in an update.
If you are looking for some wtf road videos, we are bound to see more of them soon due to the most recent update Tesla pushed. It added the "save on honk" option, which (if enabled) saves some footage right before the honk, as well as about 5 seconds after the honk. With how much spicy footage the sentry mode update has provided in less than a year, I expect nothing less from the "save on honk" update.
As a sidenote, it isn't as bad as it seems in terms of privacy, because all videos (on-honk ones, or manual ones, or the sentry mode accidents) only get saved locally on your HDD/SSD/usb-drive/etc (whatever you have inserted into one of the front USB ports).
If you find WiFi-based attacks on devices to be interesting, you may be interested in the following research from Google Project Zero on attacking the iPhone’s WiFi stack (and linked are articles on Android):
https://googleprojectzero.blogspot.com/2017/04/over-air-expl...
https://googleprojectzero.blogspot.com/2017/04/over-air-expl...
Technically interesting and also interesting that this comes from Tencent.
> Tencent Holdings Limited is a Chinese multinational conglomerate holding company founded in 1998, whose subsidiaries specialise in various Internet-related services and products, entertainment, artificial intelligence and technology both in China and globally.
> Tencent Holdings Limited is a Chinese multinational conglomerate holding company founded in 1998, whose subsidiaries specialise in various Internet-related services and products, entertainment, artificial intelligence and technology both in China and globally.
If you read the Apple product security announcements you'll find that they (as well as Qihoo 360) has been finding security vulnerabilities in unaffiliated companies' products for a long time. It's not unusual. They probably got the inspiration from Project Zero though.
Tencent has a number of security teams in addition to Keen Lab that have been around for a while: Xuanwu for example. I believe Keen Lab itself is some sort of acquisition of Keen Team, which did independent security research.
I'd say it's no different to Google Project Zero.
Of course since it's China, there'd be worries. I'd guess the Chinese government also employs crack hackers, like the NSA probably does?
Of course since it's China, there'd be worries. I'd guess the Chinese government also employs crack hackers, like the NSA probably does?
> Of course since it's China, there'd be worries.
I could imagine the government siphoning off the more valuable exploits.
I could imagine the government siphoning off the more valuable exploits.
Yeah, but they don't share 0 days like this, just like the NSA doesn't.
From the article:
> Responsible disclosure
> All the two vulnerabilities we presented above are reported to Tesla in March 2019. Tesla already fixed them in version 2019.36.2, and the Marvell also has deployed a fix and published a security advisory[4] to the issue. The disclosure of the vulnerability research report had been communicated to Tesla, and Tesla is aware of our release.
> Responsible disclosure
> All the two vulnerabilities we presented above are reported to Tesla in March 2019. Tesla already fixed them in version 2019.36.2, and the Marvell also has deployed a fix and published a security advisory[4] to the issue. The disclosure of the vulnerability research report had been communicated to Tesla, and Tesla is aware of our release.
Right... They shared the 0 day.
NSA and Comment Crew just sit on them like a dragon hording gold.
NSA and Comment Crew just sit on them like a dragon hording gold.
I believe it’s less black and white than that - all of these organizations have both offensive and defensive priorities and sometimes choose to disclose exploits publicly after calculating their remaining value. See e.g.
https://foreignpolicy.com/2017/09/25/is-the-nsa-doing-more-h...
https://www.npr.org/sections/alltechconsidered/2017/11/17/56...
https://foreignpolicy.com/2017/09/25/is-the-nsa-doing-more-h...
https://www.npr.org/sections/alltechconsidered/2017/11/17/56...
[deleted]
By only connecting Tesla to AP with PMF (encrypted management frames) enabled, this exploit would not work? Because then you can not force a reconnect to AP using Deauthenticate frames.
The problem is more explained here in a recent HN post: https://news.ycombinator.com/item?id=21889837
The problem is more explained here in a recent HN post: https://news.ycombinator.com/item?id=21889837
The DEAUTH thing is entirely incidental to the fully remote kernel code execution exploit chain.
I can agree on the Deauth, though it would maybe make it less receiptable to automated attack.
I see that Action frames also is encrypted and verified when enable PMF, so it is still viable to avoid the exploit?
I'm not sure why firefox thinks I need DRM enabled to play the video on there.
Would having a Rust based embedded OS help with attacks such as this?
Good to see at end all were reported and acted upon before release.
Was I the only one to notice the scrolling phallus doing things on the the status line for "waiting for Telsa to join our AP" on the article's attached video?
With this "wardriving" gets a whole new meaning ;)
The MCU is pretty well decoupled from any driving controls other than helping to display a bit of info like the speedometer. I mean it can even be rebooted anytime while driving. So I suspect the larger effect would be on things like entertainment features. Not really driving. But yes harhar, nice play on words.
There's a CAN bus gateway service listening on a UDP port for commands. MCU can write to CAN in addition to read from it (albeit not all operations).
Well.. that and GPS and all the worries about logging long term GPS location data, user info and mics and so on. Not a crash risk but could totally be exploited for nasty means. Plus you could falsify data too.
Patched already but good to know if you're restoring a write off.