Facebook scans system libraries on Android and uploads them to their server(twitter.com)
twitter.com
Facebook scans system libraries on Android and uploads them to their server
https://twitter.com/wongmjane/status/1167463054709334017?s=12
67 comments
As someone who’s built my company’s mobile crash reporting solution, I have a guess why they might do this.
It’s is extremely difficult to diagnose Android native code crashes. Unlike iOS where it is both straightforward to unwind on the phone, and where Apple makes the iOS system symbols available for symbolizing system frames in a stack trace, neither of these things are true on Android.
My first approach for my company’s Android crash manager SDK was to use Google Breakpad. This works by capturing a snapshot of stack memory at the time of the crash. Unwinding then occurs on a backend server. But to unwind successfully, absent a frame pointer register, you need unwind info to provide to the unwinder. This simply isn’t available except for Nexus devices for which you can download the system images from Google. And even on devices where the code was compiled with a frame pointer, you still need symbols so you know what each frame’s function was.
Another approach is to unwind on the device. In my experience, using libunwind, this is successful about 50% of the time. It also risks hanging the app, which looks even worse to the user than just crashing.
Years ago, I briefly considered having our crash SDK, optionally and with user consent, extract the symbols and unwind data from the libraries on the device and upload them to our backend. I dismissed it as too expensive to do on a user’s phone.
Instead, we crowd source as much as we can from our employee phones.
Android native code crashes remain a bear to diagnose. Especially annoying since Android itself collects a ton of diagnostic data about your app when it crashes - it just doesn’t make it easily, or in some cases at all, accessible to the app itself.
It’s is extremely difficult to diagnose Android native code crashes. Unlike iOS where it is both straightforward to unwind on the phone, and where Apple makes the iOS system symbols available for symbolizing system frames in a stack trace, neither of these things are true on Android.
My first approach for my company’s Android crash manager SDK was to use Google Breakpad. This works by capturing a snapshot of stack memory at the time of the crash. Unwinding then occurs on a backend server. But to unwind successfully, absent a frame pointer register, you need unwind info to provide to the unwinder. This simply isn’t available except for Nexus devices for which you can download the system images from Google. And even on devices where the code was compiled with a frame pointer, you still need symbols so you know what each frame’s function was.
Another approach is to unwind on the device. In my experience, using libunwind, this is successful about 50% of the time. It also risks hanging the app, which looks even worse to the user than just crashing.
Years ago, I briefly considered having our crash SDK, optionally and with user consent, extract the symbols and unwind data from the libraries on the device and upload them to our backend. I dismissed it as too expensive to do on a user’s phone.
Instead, we crowd source as much as we can from our employee phones.
Android native code crashes remain a bear to diagnose. Especially annoying since Android itself collects a ton of diagnostic data about your app when it crashes - it just doesn’t make it easily, or in some cases at all, accessible to the app itself.
[deleted]
This is clearly not for crash reporting. They're sucking up libs to figure out what apps their users use.
Could be a few reasons, could be boring metrics, could be anticompetitive identification of acquisition targets, could be oppo research, could be user profiling.
None of these things I'm ok with Facebook getting off my phone.
Could be a few reasons, could be boring metrics, could be anticompetitive identification of acquisition targets, could be oppo research, could be user profiling.
None of these things I'm ok with Facebook getting off my phone.
It sounds like you don't understand what the term "system libraries" means. It's impossible to tell what apps people are using by uploading system libraries. You could probably tell what type of phone they are using by doing it, but that would be dumb since Android already tells you what kind of phone it is.
> They're sucking up libs to figure out what apps their users use.
Hypothesis 1, debugging: requires full copies of system libraries
Hypothesis 2, fingerprinting: requires hashes of application libraries
Evidence: full copies of system libraries are being uploaded
How are you using that evidence to be so confident in hypothesis 2 and confidently against hypothesis 1?
Hypothesis 1, debugging: requires full copies of system libraries
Hypothesis 2, fingerprinting: requires hashes of application libraries
Evidence: full copies of system libraries are being uploaded
How are you using that evidence to be so confident in hypothesis 2 and confidently against hypothesis 1?
Hypothesis 3: they do both
[deleted]
If the actual files are uploaded(?), doesn't this constitute reverse software piracy? Under what license are the uploaded files covered by? Can facebook require the end users to supply the source code for GPL-licensed library binaries uploaded through this system? What about proprietary oem blobs that are often covered by a no-redistribution license or EULA? What if I am an AOSP or vendor developer working on unreleased R&D next-version Android?
It likely constitutes copyright infringement, carrying a $100K or so civil payment without need to show any damages, if FB is subject to the law.
If you hold the copyright to a library deployed on android, you might want to talk to a lawyer.
If you hold the copyright to a library deployed on android, you might want to talk to a lawyer.
Not that many people do...
This only includes system libraries which a phone OEM shipped. It doesn't include libraries which are bundled with an app.
This only includes system libraries which a phone OEM shipped. It doesn't include libraries which are bundled with an app.
If you build a custom AOSP from source and add a new system library or modify an existing system library that adds a proprietary algorithm or maybe even a poem/artwork that you wrote yourself and put a proprietary license/copyright on, maybe you could go after fb for illegally pirating/stealing your own copyrighted work? Probably wouldn't work if you do this just to poke fb knowing about this feature, but if you happened to work on custom AOSP roms for personal use before this feature became widely known, then it sounds like you might possibly have a good cause?
The OEM isn't the copyright holder of the libraries they ship though.
Very few of the libraries will be owned by an individual though - I would bet >99% are owned by an OEM or software house. That OEM probably isn't interested in suing facebook for a small amount, because if facebook blocked that OEM's phones, it would probably spell the end of the OEM.
It's not really reverse software piracy, it's just software piracy. Facebook is pirating software off of users' phones.
Nice angle but I guess the defense is that the user explicitly gave Facebook permission to read those files.
If I buy a book, I'm allowed to read that book, but I am not allowed to copy it.
You are allowed to copy it, you just can’t sell copies of it.
You are not allowed to copy it, or sell copies of it
Under fair-use laws (which vary country to country), you can usually copy a small portion of the work for non-profit educational use
https://en.wikipedia.org/wiki/Fair_use
Under fair-use laws (which vary country to country), you can usually copy a small portion of the work for non-profit educational use
https://en.wikipedia.org/wiki/Fair_use
The first line of wikipedia: "Copyright is the exclusive right, given to the creator of a creative work, to reproduce the work"
You probably won't be in trouble until you start trying to distribute your copies, but the prohibited thing is copying. Hence the term copy-right.
You probably won't be in trouble until you start trying to distribute your copies, but the prohibited thing is copying. Hence the term copy-right.
Who says Facebook is copying it, not just reading it?
It's not an angle it's literally copyright infringement.
Users aren't given the right to distribute their copies.
Users aren't given the right to distribute their copies.
So are they allowed to make point in time backups / snapshots of their phone or is that also “copyright infringement”? Usually copyright infringement focuses on distribution for piracy and/or fraudulent sales — this is neither.
Yes, users can copy this data for backup purposes, that is covered by fair use.
> Usually copyright infringement focuses on distribution for piracy and/or fraudulent sales — this is neither.
I'm not sure what you mean. Copyright infringement is the act of infringing on one's copyright. It's not a question of focus.
Copyright law does focus on certain aspects of it but it's not up to Facebook to decide when it's copyright and when it's not. The law is pretty clear here.
This is clearly distribution and clearly not licensed copying.
The purpose here is largely irrelevant.
Further, this is clearly unauthorised access and removal of user data. This is quite likely a criminal act under US hacking laws.
> Usually copyright infringement focuses on distribution for piracy and/or fraudulent sales — this is neither.
I'm not sure what you mean. Copyright infringement is the act of infringing on one's copyright. It's not a question of focus.
Copyright law does focus on certain aspects of it but it's not up to Facebook to decide when it's copyright and when it's not. The law is pretty clear here.
This is clearly distribution and clearly not licensed copying.
The purpose here is largely irrelevant.
Further, this is clearly unauthorised access and removal of user data. This is quite likely a criminal act under US hacking laws.
FWIW the sibling comments are correct under USA's Fair Use but in the UK there is only a very restrictive Fair Dealing which last I looked did not allow backups without permission unless there is no digital protection in place. There is a right under S.50A to make a backup of a computer program but when paired with the rest of the act it's unworkable.
In the UK you can't (you could for a while) rip a CD/DVD, apps like iTunes are contributory infringers.
In the UK you can't (you could for a while) rip a CD/DVD, apps like iTunes are contributory infringers.
Backups are a special case exempted from copyright. What is happening here is clearly not a backup for personal use but redistribution to another party for commercial use.
Permission? They're system libraries. If it can run, it can access those.
Okay what is the purpose of this even?
Sure, everyone is going to talk about fingerprinting, but let's face it, there are way easier and more reliable methods of doing that than system libraries that mostly match between same devices.
Must be for some sort of debugging? Still seems insane...
Sure, everyone is going to talk about fingerprinting, but let's face it, there are way easier and more reliable methods of doing that than system libraries that mostly match between same devices.
Must be for some sort of debugging? Still seems insane...
There are hundreds of thousands of variants of android phones.
Facebook wants their app to work on all of them, but cant track down all of those physical devices.
Instead, I bet they load all the libraries into a big test bench and check all features of the app work with all possible hardware.
It wouldn't be perfect, since I bet many of those libraries rely on custom system services, kernel interfaces, etc, but I bet it helps them track down a bunch of issues before they impact real users.
Facebook wants their app to work on all of them, but cant track down all of those physical devices.
Instead, I bet they load all the libraries into a big test bench and check all features of the app work with all possible hardware.
It wouldn't be perfect, since I bet many of those libraries rely on custom system services, kernel interfaces, etc, but I bet it helps them track down a bunch of issues before they impact real users.
My guess is that it helps to detect spamming tools, flash game cheats, malware that affects Facebook apps, etc.
What about the Instagram and WhatsApp apps? Do they behave the same way?
I'm really interested in an answer for this question, since WhatsApp is the only Facebook app I have installed/keep an account.
Unfortunately, everyone uses WhatsApp in Brazil, and very few people uses Telegram, for example. This makes it kinda impossible to be Facebook-free here.
Unfortunately, everyone uses WhatsApp in Brazil, and very few people uses Telegram, for example. This makes it kinda impossible to be Facebook-free here.
Apps should be statically linked and the kernel should not allow any access to the hdd or fs. Where you have to opt in to what folders the app is able to read or write to. Same for network access and any other hardware, camera, mic, etc.
Frequently those static libraries provide device-specific functionality. For example, the
Jpeg decompressor on my phone uses custom silicon, and is a systemwide shared library.
Is there any way to make memory pages executable, but not readable?
Kernel modules are executable, but not readable (by userspace processes). Or just have it in a different process and talk over IPC.
Wouldn't they just hash the libraries, surely they can get copies by legal means, like buy "a few" phones?
How does this pass through their legal team?
I suspect they don't run anything through their legal team, only ask them to help after they've fucked up.
This has the advantage of getting away with things the legal team would advise against, which I think they do a lot.
This has the advantage of getting away with things the legal team would advise against, which I think they do a lot.
If that is true...This is a culture inbreeding disaster...
Another reason I'm glad I quit
Both. No point using an operating system that is a malware and privacy nightmare, just so I can fiddle with it more. It's not a computer, it's just a phone.
And no point using Facebook, really. Still with Whatsapp (and passively Instagram) as my friends are massively on those.
And no point using Facebook, really. Still with Whatsapp (and passively Instagram) as my friends are massively on those.
Whatsapp and Instagram are Facebook. The same company owns and runs all three.
I am well aware of that. Thus I mentioned I am still with those apps (involuntarily). So still with Facebook Inc, but not Facebook.com/App
Android or Facebook?
Both for me. But my drug of choice is Instagram, so FB still has me...
How is this not utterly illegal in the EU, per GDPR? (Which was drafted to stop indiscriminate data acquisition on human subjects: I'm assuming that metadata about the core libraries on your phone, in conjunction with FB's user metadata, are trivially de-anonymizable.)
In general, I don't think this is going to collect any more identifiable data than just logging the phone model and Android build ID would; under almost all circumstances every build should correspond to exactly one set of system libraries, and they could match across upgrades just as easily with just the build metadata. Generally, every phone of the same model sold by the same carrier in the same country will have the exact same Android build with the exact same system libraries.
The main benefit that Facebook likely get out of this is that it helps them debug crashes on devices they don't have themselves.
The main benefit that Facebook likely get out of this is that it helps them debug crashes on devices they don't have themselves.
How does this provide any more data for fingerprinting than just checking the model of the phone?
My previous phone got an OS patch every few months. You would get most of this by reading the phone model and Android version, but there's probably a smidge more information in the library versions
Harder to spoof, less likely to be faked, plus additional meta information.
For starters, I'm not convinced it would be harder to spoof that than the library information (which also seems pretty easy to spoof if not easier).
But even if that were the case, why would they spend this level of engineering effort just to be able to fingerprint people in that extremely rare case of having a spoofed phone model? Do you think that kind of customer would even be receptive to targeted ads in the first place? It just doesn't make sense to me.
But even if that were the case, why would they spend this level of engineering effort just to be able to fingerprint people in that extremely rare case of having a spoofed phone model? Do you think that kind of customer would even be receptive to targeted ads in the first place? It just doesn't make sense to me.
Tracking value increases with rarity. Data is packaged and sold, not simply stored and empty cells can go for crazy money if the target is hot enough.
How much would you pay for Elons verified personal number?
How much would you pay for Elons verified personal number?
Under which provision would this be illegal though? It's the operating system image Facebook can get by just buying the exact same model of phone the user owns. I'm having trouble finding under which definition of GDPR managed personal data does this fall over.
The only angle I see is copyright infringement for copying libraries owned by the phone manufacturer... but even that I'm not sure if it's really illegal in this case. Worth filing a complaint anyway I guess.
The only angle I see is copyright infringement for copying libraries owned by the phone manufacturer... but even that I'm not sure if it's really illegal in this case. Worth filing a complaint anyway I guess.
I would assume system libraries don't count as personally identifiable data (after all, millions of people have those same libraries), so the gdpr doesn't apply.
It's not metadata, is IS the binaries.
I'm pretty sure you can fingerprint a device by gathering data about the binaries (and versions thereof) installed on it. I'm pretty sure that Facebook also know whose instance of the FB app is running on the device in question. Ergo, the data can be deanonymized and gives them more insights into what their users are doing … including, oh, competing platforms and apps?
The traditional model of computer security assumes that there's one device (the computer) which may have multiple users, so the emphasis is on identifying the user to the device. But today, one user may have one or more computers (smartphones/tablets/laptops), so the emphasis is on linking devices to users and thereby tracking usage patterns across devices. Which lands it straight in GDPR territory.
The traditional model of computer security assumes that there's one device (the computer) which may have multiple users, so the emphasis is on identifying the user to the device. But today, one user may have one or more computers (smartphones/tablets/laptops), so the emphasis is on linking devices to users and thereby tracking usage patterns across devices. Which lands it straight in GDPR territory.
> I'm pretty sure you can fingerprint a device by gathering data about the binaries
Actually, probably not. These libraries are the base system image, which is read-only, and typically will only identify which model of phone it is. It might identify you if you have a custom android build you've done yourself though.
Actually, probably not. These libraries are the base system image, which is read-only, and typically will only identify which model of phone it is. It might identify you if you have a custom android build you've done yourself though.
Shit, there are dozens of us. Dozens
No I meant they're outright uploading binaries, not just metadata.
Fingerprinting on steroids?
It's copying the actual libraries, so it's more like cutting off actual fingers rather than just taking the prints.
Maybe more like making a copy of your fingers and keeping them while leaving your actual fingers intact.
This is pretty bad.
What about the breaking of GDPR as well?
https://news.ycombinator.com/item?id=20839689