Non-Consensual Intimate Image Pilot(newsroom.fb.com)
newsroom.fb.com
Non-Consensual Intimate Image Pilot
https://newsroom.fb.com/news/h/non-consensual-intimate-image-pilot-the-facts/
183 comments
There was an excellent Twitter thread on why they didn't do this: https://twitter.com/alexstamos/status/928740488395608065
The salient bit:
> A quick note to everybody who says 'calculate the hash locally':
> A) Photo fingerprinting algorithms are usually not included in clients to prevent the development of circumvention techniques.
> B) Humans need to review to prevent adversarial reporting.
Further down the thread, they discuss further that PhotoDNA and similar algorithms are, in particular, not resistant to being tricked. Until they are, it's another layer to prevent adversaries from fully decomposing how they're hashing it, which is a good thing.
It's a trade-off, for sure, but this is a hard problem with no cut and dry answer. Facebook already has the images in question, because you have to be sending them to somebody in order to mark them as NCII. Asking yet another Facebook service to look at them without storing them seems reasonable to me. You may disagree.
The salient bit:
> A quick note to everybody who says 'calculate the hash locally':
> A) Photo fingerprinting algorithms are usually not included in clients to prevent the development of circumvention techniques.
> B) Humans need to review to prevent adversarial reporting.
Further down the thread, they discuss further that PhotoDNA and similar algorithms are, in particular, not resistant to being tricked. Until they are, it's another layer to prevent adversaries from fully decomposing how they're hashing it, which is a good thing.
It's a trade-off, for sure, but this is a hard problem with no cut and dry answer. Facebook already has the images in question, because you have to be sending them to somebody in order to mark them as NCII. Asking yet another Facebook service to look at them without storing them seems reasonable to me. You may disagree.
First, Facebook isn't the only place to share revenge porn, so they aren't really solving the problem, only reducing it slightly.
Second, security by obscurity is not a good thing. Not running the algorithm locally, not to mention producing the source to their algorithm is not going to help in the long run.
Third, regardless of whether the source of the algorithm is known or not, it would be a simple task to upload a test image of oneself, and then attempt to circumvent the algorithm using dummy accounts.
Fourth, if someone abuses the service by uploading the hashes of a benign images, it can be reported at that time. No need to pre-emptively guard against this.
Second, security by obscurity is not a good thing. Not running the algorithm locally, not to mention producing the source to their algorithm is not going to help in the long run.
Third, regardless of whether the source of the algorithm is known or not, it would be a simple task to upload a test image of oneself, and then attempt to circumvent the algorithm using dummy accounts.
Fourth, if someone abuses the service by uploading the hashes of a benign images, it can be reported at that time. No need to pre-emptively guard against this.
> Humans need to review to prevent adversarial reporting
Humans already need to review the takedown reports themselves to prevent adversarial reporting!
Humans already need to review the takedown reports themselves to prevent adversarial reporting!
>Humans need to review
uhm...this seems like a bit of a flaw in the plan
uhm...this seems like a bit of a flaw in the plan
valuearb(4)
The obvious reason why Facebook didn't go down this path is one of scale - It's easy (and altogether likely) for an attacker to pollute the hashes with innocuous pictures. But, even more importantly, once you reveal your hashing system, it's relatively trivial for people to work around them and modify the pictures in such a way as to render the hashing system useless. Has to be done server side for these reasons.
Could you elaborate why you think it's easy to pollute hashes/fingerprints in this scheme? It seems to me that it would be hard to do reliably, since each hash submitted is going to be linked to an individual account. If an account submitted a bunch of hashes that matched non-explicit images, all further hash submissions from that user account could be rejected. (In the GP's proposed scheme a human can still verify a sample of all positive matches, to validate if they are true or false positives).
I think the point about revealing your fingerprinting system is a good one. But I wonder if the equilibrium point would still be better than what we have now, i.e. unsophisticated posters would get caught, and sophisticated ones might not, vs. what we have now where there is no way to catch any of these posts.
I think the point about revealing your fingerprinting system is a good one. But I wonder if the equilibrium point would still be better than what we have now, i.e. unsophisticated posters would get caught, and sophisticated ones might not, vs. what we have now where there is no way to catch any of these posts.
If “scale” is the reason they insist that explicit images be uploaded directly to them instead of hashed beforehand, then they really don’t care about this program enough to do it right.
They can easily scale this by allowing hash uploads, using a trivial client side classifier to guess at whether an image is actually served well by this technology, and confirm positives with CV after a bad actor reposts their media on the site, with human intervention only on the fringes.
They can easily scale this by allowing hash uploads, using a trivial client side classifier to guess at whether an image is actually served well by this technology, and confirm positives with CV after a bad actor reposts their media on the site, with human intervention only on the fringes.
You'll have to guarantee the authenticity of the client side code, but yes.
Here's what you do: on the client side embed image into a semantic space (using NN or whatever), quantize, _then_ hash the representation. Afterwards you send the hash. If you had to change client side code, you can just ask the user to redo the process.
Here's what you do: on the client side embed image into a semantic space (using NN or whatever), quantize, _then_ hash the representation. Afterwards you send the hash. If you had to change client side code, you can just ask the user to redo the process.
Yes, I was hoping they would not simply `sha1()` the image data. I wouldn't worry much about trying to fake hashes since we are already allowing them to provide whatever image as input they want (hence the review).
Kind of weird that they didn't do things this way as they supposedly already use hashing to flag child porn cf. http://www.wired.co.uk/article/iwf-hash-lists-child-abuse-im...
That system works the same way as what they're implementing. Someone reviews the raw image and enters the hash into the system.
>Someone reviews the raw image and enters the hash into the system.
Now that is a job I really would not want to have. Talk about mentally scarring.
Now that is a job I really would not want to have. Talk about mentally scarring.
Isn't that one of the highest turnover jobs that exists? It's really hard to keep people coming into work when their work is to review twisted content all day.
I have a hypothesis that undiscovered child fanciers will seek jobs like this.
I also hypothesise (on the basis of news articles saying how low the reoffending rates are compared to other categories of offender) that the distribution of sadistic narcissism is the same or similar for them as it is everyone else, so when they sit down at the office desk and see a screaming child, most of them will hate themselves if they didn’t already.
I’m not so sure about the second hypothesis though. It’s based on what newspapers say, and I’ve learned to distrust them.
I also hypothesise (on the basis of news articles saying how low the reoffending rates are compared to other categories of offender) that the distribution of sadistic narcissism is the same or similar for them as it is everyone else, so when they sit down at the office desk and see a screaming child, most of them will hate themselves if they didn’t already.
I’m not so sure about the second hypothesis though. It’s based on what newspapers say, and I’ve learned to distrust them.
As other has pointed out, this is extremely creepy.
The easiest solution would have been this one:
- The user uses a JS solution to hash the images on the client, without the image being uploaded
- She compiles a form with additional information (e.g: capture her account, reasons for uploading, suspect person sharing the picture)
- The picture is saved in the DB as un-verified revenge porn.
- The first time someone uploads a picture that matches the hash, the pic is quarantined and the specially trained individual manually check them
- A scoring system could be used to check the reliability of the submission. If multiple photos marked revenge porn get rejected, the control becomes ex-post. For even more violations, the user get banned from using the tool and should directly contact Facebook. Submitting the same hash that has been rejected, will count as a "red mark"
Now, I understand this system is very complex, what Facebook has done is an MVP and as a product manager, this is what I prefer. But considering the issue (revenge porn, not something I necessarily want to test the impact on retention :-) ). Also, yes, it requires resources, but Facebook has a problem with trust lately, better to do the best...
[edited for formatting]
- The user uses a JS solution to hash the images on the client, without the image being uploaded
- She compiles a form with additional information (e.g: capture her account, reasons for uploading, suspect person sharing the picture)
- The picture is saved in the DB as un-verified revenge porn.
- The first time someone uploads a picture that matches the hash, the pic is quarantined and the specially trained individual manually check them
- A scoring system could be used to check the reliability of the submission. If multiple photos marked revenge porn get rejected, the control becomes ex-post. For even more violations, the user get banned from using the tool and should directly contact Facebook. Submitting the same hash that has been rejected, will count as a "red mark"
Now, I understand this system is very complex, what Facebook has done is an MVP and as a product manager, this is what I prefer. But considering the issue (revenge porn, not something I necessarily want to test the impact on retention :-) ). Also, yes, it requires resources, but Facebook has a problem with trust lately, better to do the best...
[edited for formatting]
Now figure out a way to do step #1 (user does a hash client-side) without making it trivial for someone else to create a filter that adds enough noise to invalidate step #4 (uploaded pics that match a hash are quarantined.)
PhotoDNA would be an option, I imagine. Or at least something similar!
https://en.wikipedia.org/wiki/PhotoDNA
https://en.wikipedia.org/wiki/PhotoDNA
Honestly, I doubt that most of these sorts of algorithms would survive concerted attacks – that's why they tend to be closely guarded.
Alex Stamos (Facebook's CISO) implies this is why they can't do it client-side:
https://twitter.com/alexstamos/status/928646228472078336
Alex Stamos (Facebook's CISO) implies this is why they can't do it client-side:
https://twitter.com/alexstamos/status/928646228472078336
The attacker isn’t a national state adversary. The attacker is a vengeful ex. The chances of such attacker even been aware of what hashing is, is close to zero.
Whoever designed this system is probably heavy on security, but low on product.
Whoever designed this system is probably heavy on security, but low on product.
No, the attacker is a bored hacker who wants to create problems for FB, just because they feel like it.
And when the attacker Googles "how to upload images banned by Facebook" won't they find that article from that one vengeful ex who understood the technology well enough to build a tool?
From when security through obscurity is a good idea?
This is in general an unsolvable problem, and the only possible security is through obscurity. Images are on a spectrum, and so in order to hash an image, you must quantize the spectrum. Between two images, there is a continuum of mixed images. Using bisection, you can identify the point where it goes from matching to not matching. You can then upload the non-matching variant. Because of the way hashing works, this will be indistinguishable from any other non-matching image. Therefore, this creates an easy method to bypass facebook's filter while sacrificing minimal quality. As long as facebook matches against hashes, there is way to prevent bypassing of this form. All they can do is make it as difficult as possible.
You should go put a $100 bill on the dashboard of your car along with your laptop on the front seat. Report back the results.
snark-mode off: We use obscurity every day and its a completely valid layer of security.
snark-mode off: We use obscurity every day and its a completely valid layer of security.
> We use obscurity every day and its a completely valid layer of security.
Not sure I agree with that, most of the time when we do that is because we don't want to spend the time to have better security. And then we get burned. To your example of the 100$ bill: at my parents home with the car parked in the garage? No problem to do that at all. Out on the street in SF? No. I don't trust my glass enough as a security measure. But I don't leave money at all, is not security through obscurity.
But we are going OT. The problem that is raised is that they need necessarily security through obscurity. And we have two problems:
- How really robust are these algorithms? How long before we will see people abusing them?
- Have you thought hard enough about how this system could work? E.g: have a partial hashing made client-side and the final one on the server? Or a situation where the server code is open-sourced without the model to calculate the hash? That would allow for external review without disclosing the hash. Yes, you still need warranties that Facebook is using that code, but you could have a trusted third party certifying the program.
My point is, the person who designed this program didn't really understand the problem. The problem is not revenge porn. The problem is Facebook reputation. And this solution is totally deaf.
Not sure I agree with that, most of the time when we do that is because we don't want to spend the time to have better security. And then we get burned. To your example of the 100$ bill: at my parents home with the car parked in the garage? No problem to do that at all. Out on the street in SF? No. I don't trust my glass enough as a security measure. But I don't leave money at all, is not security through obscurity.
But we are going OT. The problem that is raised is that they need necessarily security through obscurity. And we have two problems:
- How really robust are these algorithms? How long before we will see people abusing them?
- Have you thought hard enough about how this system could work? E.g: have a partial hashing made client-side and the final one on the server? Or a situation where the server code is open-sourced without the model to calculate the hash? That would allow for external review without disclosing the hash. Yes, you still need warranties that Facebook is using that code, but you could have a trusted third party certifying the program.
My point is, the person who designed this program didn't really understand the problem. The problem is not revenge porn. The problem is Facebook reputation. And this solution is totally deaf.
Facebook is pretty low on companies I would trust, on the other hand we know that NSA employees who have the highest level of security clearance were using their ability to intercept everything to stalk their exes and pass around their exes nude photos according to Snowden. Clearly nobody can be trusted, and as you write, a solution for this must incorporate this fact.
Your comment hit the nail on the head
Why would you need to? We are talking about vengeful ex-es here. It’s possible to bypass this trivially by uploading it to another image sharing service, so the goal should be defection in the general case; not try and protect against the 0.1% that even know what a perceptual hash is.
Agree, that's a problem but there are options to solve it. Look at PhotoDNA by Microsoft [0]. But it is a second step. First, you need the reporting properly done.
[0] https://www.microsoft.com/en-us/photodna
[0] https://www.microsoft.com/en-us/photodna
[deleted]
Good point, that said, with the recent visual ML trend, I'm sure fb or the likes can engineer a solution for that.
How does giving Facebook the image solve that problem, in a way that can't be done client side?
This is already a solved problem at FB (PhotoDNA does this for them for CP images).
> - The user uses a JS solution to hash the images on the client, without the image being uploaded
You have to trust facebook in either case, each time you do it. Either to handle your nude pics properly or to serve you javascript that does what they claim it does, every single time.
On the other hand an open source desktop application only needs to be audited once and then can be validated based on a hash.
In browser crypto is not a solution if you want to minimize the needed trust.
You have to trust facebook in either case, each time you do it. Either to handle your nude pics properly or to serve you javascript that does what they claim it does, every single time.
On the other hand an open source desktop application only needs to be audited once and then can be validated based on a hash.
In browser crypto is not a solution if you want to minimize the needed trust.
Nobody wants to run a desktop application given to them from facebook
But it could force autoinstall from the website and autoscan your drives then preupload your nudie pics and tag them with your name. How convenient! If all your friends are using it then it must be safe right?
> only needs to be audited once and then can be validated based on a hash.
I thought I covered that concern, but I neglected to mention that it should be open source so everyone can audit it.
I thought I covered that concern, but I neglected to mention that it should be open source so everyone can audit it.
It’s better than sending them NSFW photos for human review
You could examine network traffic to confirm that not enough data is sent to reconstitute the picture
They could save it indexedb or localstorage and send it later. So now you have three things to inspect. They could even shunt it to a different domain within the browser through iframes and messaging. Now you have another thing to watch. There aren't even any devtools to watch that.
The cheese is full of holes.
The cheese is full of holes.
You'd have to check every single time, since the code can change at any time.
The discussion surrounding this made me think of a feature I'd love to have along these lines. I'd like to be able to blacklist people / certain people from uploading pictures that facebook detects my face in. Fb as a platform must already have this functionality, what with the auto face identification and privacy / review post settings.
I can imagine images of people that aren't "non-consensual intimate images" that they'd reasonably still like to be able to block being posted.
One use case I can imagine is doxxing prevention. Say there is a national news story involving some random person in some small town that exposes the persons face and name and rough location. Some group of enraged internet denizens start sharing the person's face and name all over the place as a means to spread their pitchfork & torches mob against them.
Seems like a reasonable feature to me. Recently a professor in my state made some remarks about white privilege and the doxxing of him was so violent and pervasive that he and his family had to leave the state for months, and he still gets threats and needs protection on campus.
I can imagine images of people that aren't "non-consensual intimate images" that they'd reasonably still like to be able to block being posted.
One use case I can imagine is doxxing prevention. Say there is a national news story involving some random person in some small town that exposes the persons face and name and rough location. Some group of enraged internet denizens start sharing the person's face and name all over the place as a means to spread their pitchfork & torches mob against them.
Seems like a reasonable feature to me. Recently a professor in my state made some remarks about white privilege and the doxxing of him was so violent and pervasive that he and his family had to leave the state for months, and he still gets threats and needs protection on campus.
> I can imagine images of people that aren't "non-consensual intimate images" that they'd reasonably still like to be able to block being posted.
If I take a photo of a public landscape and you happen to be there, do you have the right to prevent me from publishing it unaltered?
Keep in mind: you're not the primary subject (and no reasonable person would believe that you were) nor owner of the photo. Your likeness is not reasonably intended to alter the value of the photo.
If I take a photo of a public landscape and you happen to be there, do you have the right to prevent me from publishing it unaltered?
Keep in mind: you're not the primary subject (and no reasonable person would believe that you were) nor owner of the photo. Your likeness is not reasonably intended to alter the value of the photo.
Excellent! Then it should be short work for you to photoshop my face out so it won't be blocked. Everybody wins.
Why is Facebook obligated to fulfill your desires for privacy by censoring content you have no legal claim to?
I think the claim is that I should have a legal claim to my likeness.
Note that I'm not personally convinced of this, I just felt that it was really weak to say that it was fine because the faces aren't an important part of the picture.
Note that I'm not personally convinced of this, I just felt that it was really weak to say that it was fine because the faces aren't an important part of the picture.
> I just felt that it was really weak to say that it was fine because the faces aren't an important part of the picture.
This was very intentional, because it is how courts look at the monetization of this kind of content today.
Your likeness cannot be commercialized without your consent (which is why things like photo/video releases exist) - that being said, the line determining your importance to the photograph's value is up to what a reasonable person (a judge) would believe.
This comparison, while not perfect, was used to preempt an argument from the "$ without permission" angle.
--
As a private company, Facebook is free to add this feature for its users.
As a citizen: I am free to publish my photo containing your likeness elsewhere until legislatively or judicially prohibited. And then I am free to challenge that censorship (as many have on the commercialization issue) on obvious First Amendment grounds.
Aside: I won't get into any European arguments about the "right to be forgotten" - we (USA) don't recognize this as a right.
This was very intentional, because it is how courts look at the monetization of this kind of content today.
Your likeness cannot be commercialized without your consent (which is why things like photo/video releases exist) - that being said, the line determining your importance to the photograph's value is up to what a reasonable person (a judge) would believe.
This comparison, while not perfect, was used to preempt an argument from the "$ without permission" angle.
--
As a private company, Facebook is free to add this feature for its users.
As a citizen: I am free to publish my photo containing your likeness elsewhere until legislatively or judicially prohibited. And then I am free to challenge that censorship (as many have on the commercialization issue) on obvious First Amendment grounds.
Aside: I won't get into any European arguments about the "right to be forgotten" - we (USA) don't recognize this as a right.
But Facebook operates in Europe with photos of EU citizens. So regardless of where the photographer is from, the photographee should be able to assert that right.
Consider that this situation could extend or apply to humans, animals, or even plants.
Does a rhinocerous have a right to be forgotten?
https://www.reddit.com/r/dredmorbius/comments/25ll1v/does_a_...
Does a rhinocerous have a right to be forgotten?
https://www.reddit.com/r/dredmorbius/comments/25ll1v/does_a_...
I think this is a separate philosophical debate: operating under current legal frameworks in the United States of America, you do not have the right to prevent me from publishing my photo (with no demonstrable harm being done.)
The purpose of a discussion is to determine whether or not those legal frameworks are appropriate to the situation.
The law is a tool, not an ends.
The law is a tool, not an ends.
I 100% prefer the status quo to the idea proposed here. I understand that some people would rather not be pictured, but I don't think that should override the right of others to make images in public spaces (or private spaces with suitable permission).
Or they can block display of the photo until all faces approve.
It’s interesting that the general consensus is strangers seeing your nudes is creepy (referring to the manual processing aspect.) I understand it on a fundamentally emotional level: I want absolute control of my private life / photos / etc. I am embarrassed at the idea of being seen naked without my active participation. On a logical level, though, what difference does it make if every person you will never meet has seen your dick?
It’s a philosophical question, to be sure.
Also, what happens with these manual reviews that turn up underage nudes? Teen sexting is a thing. For that matter, isn’t Snapchat a massive repository of child pornography?
It’s a philosophical question, to be sure.
Also, what happens with these manual reviews that turn up underage nudes? Teen sexting is a thing. For that matter, isn’t Snapchat a massive repository of child pornography?
What's really bizarre to me is that people here don't think using facebook in general is creepy...
I think it's the same deal: your data is being seen / used by algorithms, your information is just a bunch of 1s and 0s being copied and processed and re-copied and re-processed. It's all very impersonal, so it's easy to abstract it away.
Ok, I had to laugh. FB is literally the worst company to make this service with the worst privacy reputation ever.
Then the service makes the worst mistake that they actually upload nudes to their platform, PLUS they have a human look at it, which adds another layer of security leaks.
Did they do this intentionally or did they have such a huge lack of common sense when they decided how to design this and nobody in the project had the idea to address this.
Then the service makes the worst mistake that they actually upload nudes to their platform, PLUS they have a human look at it, which adds another layer of security leaks.
Did they do this intentionally or did they have such a huge lack of common sense when they decided how to design this and nobody in the project had the idea to address this.
I think they want to see how much can they get away with. Let's see if we can actually get people to send us all their private porn.
So they basically openly admit that a human will review nudes? How is this in any way a sane idea? If you were already scarred (for life, likely) by said revenge porn existing and leaking out somewhere non-facebook, likely the last thing you want is to be forced, YOURSELF, to send it to someone (facebook)
From the article: "people can already report if their intimate images have been shared on our platform without their consent".
If it already happened, they don't need to upload it again. Presumably this new process is for prevention when it didn't happen yet but they have good reason to suspect that it will.
If it already happened, they don't need to upload it again. Presumably this new process is for prevention when it didn't happen yet but they have good reason to suspect that it will.
"if it already happened on facebook" != "if it already happened"
There is, in fact, a whole world outside of www.thefacebook.com
There is, in fact, a whole world outside of www.thefacebook.com
Great idea, absolutely horrible implementation even as an MVP.
Never in a million years would I trust that kind of data residing on Facebooks servers even fleetingly. Years back, Facebook allowed you to have private profile photos. My then-girlfriend-now-wife has a picture of the two of us as hers which for cultural and religious reasons she had private since we hadn’t decided to tell her parents yet. Facebook decided without telling anyone one day that Facebook photos were now public. She hasn’t spoken to her father since that day. Facebook can and will do whatever they want with the data they have on you. Maybe permissions change on messenger (or there’s a bug) and data from this flow leaks. Maybe a year from now this flow still exists but messager makes all picture uploads a soft delete because they want to keep things as training data. Maybe the upload gets replicated to their CDNs in a way that is discoverable and recoverable. Maybe documentation for this manual process outlives the software, someone finds it, and uploads a picture only to find out they did it in the wrong place that now has no guarantees of even an effort for privacy.
That’s a lot of maybes for something so sensitive. Facebook doesn’t deserve your trust with this kind of data. They should come up with something where the hashing happens locally and the user never has to put faith in them to begin with.
Never in a million years would I trust that kind of data residing on Facebooks servers even fleetingly. Years back, Facebook allowed you to have private profile photos. My then-girlfriend-now-wife has a picture of the two of us as hers which for cultural and religious reasons she had private since we hadn’t decided to tell her parents yet. Facebook decided without telling anyone one day that Facebook photos were now public. She hasn’t spoken to her father since that day. Facebook can and will do whatever they want with the data they have on you. Maybe permissions change on messenger (or there’s a bug) and data from this flow leaks. Maybe a year from now this flow still exists but messager makes all picture uploads a soft delete because they want to keep things as training data. Maybe the upload gets replicated to their CDNs in a way that is discoverable and recoverable. Maybe documentation for this manual process outlives the software, someone finds it, and uploads a picture only to find out they did it in the wrong place that now has no guarantees of even an effort for privacy.
That’s a lot of maybes for something so sensitive. Facebook doesn’t deserve your trust with this kind of data. They should come up with something where the hashing happens locally and the user never has to put faith in them to begin with.
> Once we receive this notification, a specially trained representative from our Community Operations team reviews and hashes the image
At first I wondered "Why?!" Why does a "specially trained" person have to look at the very same intimate images that their users wish to prevent being shared.
Then it struck me that without review, any image could be blocked, simply by submitting it through this process. This would also explain why the offline hashing approaches being suggested aren't practical - they're just too prone to abuse by flagging images that are not of a sexual nature, but that you would like to see blocked for whatever reason.
An ML approach to masking the most sensitive portions of the submitted images would both protect the user, and reduce the burden on those tasked with reviewing the submitted images.
If this could be handled on the client, and sent with a verifiable hash of the original image, that would solve the majority of concerns raised in the discussions here.
At first I wondered "Why?!" Why does a "specially trained" person have to look at the very same intimate images that their users wish to prevent being shared.
Then it struck me that without review, any image could be blocked, simply by submitting it through this process. This would also explain why the offline hashing approaches being suggested aren't practical - they're just too prone to abuse by flagging images that are not of a sexual nature, but that you would like to see blocked for whatever reason.
An ML approach to masking the most sensitive portions of the submitted images would both protect the user, and reduce the burden on those tasked with reviewing the submitted images.
If this could be handled on the client, and sent with a verifiable hash of the original image, that would solve the majority of concerns raised in the discussions here.
It's a cool idea, but isn't this pretty easily defeated by an adversarial attack, or just altering the image in any simple way?
E.g. as the perpetrator, open the image of your ex in MS Paint, add a stripe across the bottom in such a manner as to not obscure the primary content at all, ta-dah, upload, image will not have a matching hash.
E.g. as the perpetrator, open the image of your ex in MS Paint, add a stripe across the bottom in such a manner as to not obscure the primary content at all, ta-dah, upload, image will not have a matching hash.
Already addressed in this other comment: https://news.ycombinator.com/threads?id=turc1656
It seems like HN comment pages might have a critical mass beyond which it's impractical to see if somebody's already said what you want to say so people repeat the same idea - ballooning the comments page even further.
It seems like HN comment pages might have a critical mass beyond which it's impractical to see if somebody's already said what you want to say so people repeat the same idea - ballooning the comments page even further.
What happens when the FB employee[1] finds a way to collect the images and personal data they are reviewing and at some later time distributes the images using some other outlet?
[1] "... a specially trained representative from our Community Operations team reviews and hashes the image, ..."
[1] "... a specially trained representative from our Community Operations team reviews and hashes the image, ..."
In this thread there are a lot of valid concerns about client-side fingerprinting and how this would allow sophisticated algorithms to be broken. Putting something like PhotoDNA on clients is probably not good.
However, I think a simpler approach would handle >95% of cases here, and that’s to just calculate the image sha and use that. It does nothing for resizes/obfuscations, but if Facebook users are dumb enough to volunteer their nudes to human review, they’re dumb enough to not modify images before reposting them as revenge porn.
Everything else with respect to modified images and elaborate fingerprinting and perceptual hashing can happen at the next tier, which I trust Facebook already has an intricate (and publicly tolerated) system to handle.
However, I think a simpler approach would handle >95% of cases here, and that’s to just calculate the image sha and use that. It does nothing for resizes/obfuscations, but if Facebook users are dumb enough to volunteer their nudes to human review, they’re dumb enough to not modify images before reposting them as revenge porn.
Everything else with respect to modified images and elaborate fingerprinting and perceptual hashing can happen at the next tier, which I trust Facebook already has an intricate (and publicly tolerated) system to handle.
Facebook: "Send Nudes"
Even if this worked as intended, what’s the point? The person wanting to spread such images could just use another platform to do so, or modify the image slightly... Would this really stop much abuse?
The same hubris that lets Facebook believe they can do this without it backfiring on them is the same hubris that lets them believe that theirs is the only platform that matters.
[deleted]
Is there anyone else out here scratching their head as to why anyone would think it's a good idea to actually take (or otherwise) send (let's be honest, here) pornographic images of themselves to anyone else?
Back a long time ago, if you had a Polaroid you might have some reasonable expectation that picture was the only one... but now? Once it is digital you cannot guarantee a single thing that happens to it unless you do it offline.
Why would anyone want to take that risk?
Back a long time ago, if you had a Polaroid you might have some reasonable expectation that picture was the only one... but now? Once it is digital you cannot guarantee a single thing that happens to it unless you do it offline.
Why would anyone want to take that risk?
> Once we receive this notification, a specially trained representative from our Community Operations team reviews and hashes the image, which creates a human-unreadable, numerical fingerprint of it.
I'm wondering how long it will be before we start seeing cell phone pics of screens with peoples intimate images on them.
I'm wondering how long it will be before we start seeing cell phone pics of screens with peoples intimate images on them.
How long will it be before "nudes" are so normalized that people post them to their own profile? 10 years? 20 years? 100 years?
If you doubt this will ever happen, look up some images of "1920s beachwear" and ask yourself where the trend is going.
If you doubt this will ever happen, look up some images of "1920s beachwear" and ask yourself where the trend is going.
Lots of people caught up here about nitty gritty details of hash type client side vs server side and so on. The elephant in the room in my view is the fact that the person can just upload the images on another image/file sharing platform.
Is it legal for Facebook to solicit pornographic images from minors, or are they (nominally) limiting this feature to adults?
OMG the mad rush to get there first for libraries of revenge porn.
Twins with different opinions about this are going to have a hard time...
Fuck Facebook
Am I the only person reading this who thinks it won't work at all...?
All it takes is the image being sent via screenshot, or otherwise manipulated to change the hash
This is dumb. A hash? That's only going to deter the most incompetent internet users when it comes to a file type that is subject to alteration without losing it's information value (pictures, video, music, etc.).
All someone has to do is change a single bit/pixel in the image and the hash will be different. No one will notice that and it defeats the hash. Hell, you don't even have to do that. You can just rotate the image, save it, then rotate it back, and re-save it. The odds are that program that you used will most certainly not write the image data in the same exact way.
Hashes would be better for files that cannot be altered without breaking the contents. Although, these could be subject to compression, which would create a new hash.
Either way, I think they need to use something similar to Google's image search that actually examines the photo contents for similarity.
All someone has to do is change a single bit/pixel in the image and the hash will be different. No one will notice that and it defeats the hash. Hell, you don't even have to do that. You can just rotate the image, save it, then rotate it back, and re-save it. The odds are that program that you used will most certainly not write the image data in the same exact way.
Hashes would be better for files that cannot be altered without breaking the contents. Although, these could be subject to compression, which would create a new hash.
Either way, I think they need to use something similar to Google's image search that actually examines the photo contents for similarity.
In this case, 'hash' doesn't necessarily mean 'a cryptographic hash of the raw file contents'.
There are perceptual hash techniques which still match after changes to the file, and are tuned in a similar manner to lossy compression algorithms to prioritize comparing information relevant to human perceptions.
https://www.phash.org/ is one open-source example.
There are perceptual hash techniques which still match after changes to the file, and are tuned in a similar manner to lossy compression algorithms to prioritize comparing information relevant to human perceptions.
https://www.phash.org/ is one open-source example.
It's not a technical article. I think it's safe to assume that 1) The people implementing this aren't idiots, and 2) The "hash" is probably referring to a "perceptual hash", where very-similar images would show up as the same.
There are hashes that represent the content of photos, the Microsoft’s PhotoDNA and Cloudinary’s pHash. I’m pretty sure Facebook’s engineers are smart enough to that they can’t just run photos through MD5 and have everything work.
It could be a perceptual hash. Who knows? The article doesn't go into detail.
Tricky. Sending them all of your potentially leaked embarrassing photos so that they can store them and prevent them from leaking out. If they get hacked now someone has a lifetime supply of blackmail material. Not clear if the cure is better than the ailment.
From the fine article:
"We store the photo hash—not the photo—to prevent someone from uploading the photo in the future. If someone tries to upload the image to our platform, like all photos on Facebook, it is run through a database of these hashes and if it matches we do not allow it to be posted or shared."
"We store the photo hash—not the photo—to prevent someone from uploading the photo in the future. If someone tries to upload the image to our platform, like all photos on Facebook, it is run through a database of these hashes and if it matches we do not allow it to be posted or shared."
All you can do is take FB at their word that the image is destroyed irrevocably.
Given what we know about data retention policies at big tech firms, I'm not so sure I would feel confident taking the press release at face value. I'd really like to see a white paper or similar outlining the specifics of how this is being handled. I'd also have questions around what steps have been taken to prevent rogue Facebook employees trying to obtain the images.
I really wish they had found a way to generate the hashes they require client side and not receive the image at all, especially as this is something that presumably would be really great for revenge porn victims.
Given what we know about data retention policies at big tech firms, I'm not so sure I would feel confident taking the press release at face value. I'd really like to see a white paper or similar outlining the specifics of how this is being handled. I'd also have questions around what steps have been taken to prevent rogue Facebook employees trying to obtain the images.
I really wish they had found a way to generate the hashes they require client side and not receive the image at all, especially as this is something that presumably would be really great for revenge porn victims.
I don’t know. I think a whitepaper might just be a drier way to lie about what they’re really doing.
You have to read this as a lawyer would. Let me show you: We store the photo hash - not the photo. This says nothing about whether other parties we pass the data onto or through store the photo. Remember when prism was leaked and some tech companies said 'we do not give our data to the nsa'. give. (Of course we provided them with a backdoor login.)
Ah, missed that, thanks. Yeah I guess you have to trust that part. I guess I'm willing to believe that. Hopefully it's not like with Snapchat where everything get stored behind the scenes forever even though it "disappears".
Also, wonder if basing things on hash will make detecting slightly modified versions of that content almost impossible. E.g. IG / YT can detect if you upload content with a piece of music that's copyrighted regardless of how much you alter it, but this would be fairly primitive.
Also, wonder if basing things on hash will make detecting slightly modified versions of that content almost impossible. E.g. IG / YT can detect if you upload content with a piece of music that's copyrighted regardless of how much you alter it, but this would be fairly primitive.
Is there a way for me to audit that the image is never stored? (No) It seems like that's a huge honeypot; someone compromises the "upload your naughty photos here" endpoint and has an endless supply of "non-consensual intimate images".
Edit: Thinking about it a bit, I'd be way more comfortable with the idea of the image hashes being computed on-device and only the hashes being sent to the server. This opens a different door of abuse (by essentially permitting individuals to ban someone from posting an image by uploading the hash of the image), but it's definitely preferable to encouraging people to send all their selfies to the privacy commissioner.
Edit: Thinking about it a bit, I'd be way more comfortable with the idea of the image hashes being computed on-device and only the hashes being sent to the server. This opens a different door of abuse (by essentially permitting individuals to ban someone from posting an image by uploading the hash of the image), but it's definitely preferable to encouraging people to send all their selfies to the privacy commissioner.
While this (client-side hashing) would be better in terms of privacy protection for this specific case it is not going to happen because Facebook is not allowed to perform the specific photo hashing client-side as this would expose the hashing mechanism to analysis.
> this would expose the hashing mechanism to analysis
I mean, it worked pretty well for Google's ranking algorithm, so much so that it spawned an entire industry (SEO, which still doesn't let you get away with much in the way of black-hat type stuff).
Even just the rate-limiting aspect of keeping the algo on the server (in this case behind a human, but that's even not necessary) is useful: I can't sit offline with the algo and keep beating it with slightly different images until the hash changes enough to be considered unique.
Even just the rate-limiting aspect of keeping the algo on the server (in this case behind a human, but that's even not necessary) is useful: I can't sit offline with the algo and keep beating it with slightly different images until the hash changes enough to be considered unique.
maybe they shouldn't ask for nudes until they have designed an open source algorithm?
How would an open source fingerprinting algorithm work in this case?
There would be a browser extension to "refingerprint" images within 24 hours of release.
There would be a browser extension to "refingerprint" images within 24 hours of release.
Make the algorithms public, let them be broken, refine them etc. until the margin between identification and refingerprinting becomes so small that refingerprinting involves such significant changes that we are reasonably confident that we are no longer dealing with theseus' ship.
That's how we got good crypto. By not keeping it proprietary.
And a side benefit that we would get some amazingly alteration-resistant perceptual image hashes out of that.
That's how we got good crypto. By not keeping it proprietary.
And a side benefit that we would get some amazingly alteration-resistant perceptual image hashes out of that.
It would be interesting to see what people come up with, but I doubt any algorithm can stop someone dedicated to figure out a way to defeat it.
Having the algorithm open for public review will lead to better algorithms, but allow for direct feedback if you are attempting to circumvent it; whilst keeping it proprietary will prevent direct feedback, but might keep the algorithm less clever than it could be.
If Facebook is using just PhotoDNA, defeating it should be trivial¹:
> […] It works by converting the image to black and white, re-sizing it, breaking it into a grid, and looking at intensity gradients or edges […]
That's fairly generic, but effective enough for resizing, colour adjustments, minor changes, etc., but not good enough for porn and an adversary willing to do some experimenting with the photo.
If I were to guess at how Facebook uses this, I would expect that they use a preprocessing routine that strips out anything that doesn't resemble a nude or semi-nude person, to prevent alterations to the person's surroundings from changing the hash — which is what you would do if you wanted to bypass the filter whilst keeping the essence of the picture intact. Keeping that fact hidden would be integral to their strategy I suppose.
1: https://en.wikipedia.org/wiki/PhotoDNA
Having the algorithm open for public review will lead to better algorithms, but allow for direct feedback if you are attempting to circumvent it; whilst keeping it proprietary will prevent direct feedback, but might keep the algorithm less clever than it could be.
If Facebook is using just PhotoDNA, defeating it should be trivial¹:
> […] It works by converting the image to black and white, re-sizing it, breaking it into a grid, and looking at intensity gradients or edges […]
That's fairly generic, but effective enough for resizing, colour adjustments, minor changes, etc., but not good enough for porn and an adversary willing to do some experimenting with the photo.
If I were to guess at how Facebook uses this, I would expect that they use a preprocessing routine that strips out anything that doesn't resemble a nude or semi-nude person, to prevent alterations to the person's surroundings from changing the hash — which is what you would do if you wanted to bypass the filter whilst keeping the essence of the picture intact. Keeping that fact hidden would be integral to their strategy I suppose.
1: https://en.wikipedia.org/wiki/PhotoDNA
But you are sending an image to yourself in Messenger which DOES normally keep an image. It doesn't say here that they explicitly delete the message you sent via Messenger, does it? Perhaps you have to delete that for yourself? And do they take back-ups of Messenger data in which this photo may now be in?
Seems like a very awkward flow to forcefully repurpose Messenger to do something it shouldn't be doing.
Seems like a very awkward flow to forcefully repurpose Messenger to do something it shouldn't be doing.
This is insane. If they can fingerprint your images, why can't they provide a tool for processing the image and sending the fingerprint only? Can you imagine how big of a target a tailor made database of blackmail material linked to facebook accounts would be?
In addition to the reasons other have suggested, one reason that this is not done client side is that the PhotoDNA tech that is used for the fingerprinting is not something that Facebook can share or make available. It's primary use is in matching posted images with known databases of child porn images, so providing a mechanism for someone to reverse engineer the system and develop a means of bypassing it is not going to happen.
They could just send fingerprints, sure.
The problem is that you presumably want the image to be blocked immediately upon upload.
But facebook is worried that people will abuse the system, and they don't want images to be instantly wrongfully blocked until someone can review it.
Abuse-resistance. Instant blocking. Perfect privacy. Pick two. There's no perfect solution.
The problem is that you presumably want the image to be blocked immediately upon upload.
But facebook is worried that people will abuse the system, and they don't want images to be instantly wrongfully blocked until someone can review it.
Abuse-resistance. Instant blocking. Perfect privacy. Pick two. There's no perfect solution.
Why not suspend reported images for review. If the review indicates the reporter is abusing the system, blacklist the reporter from the automatic function. Also block new and recent accounts, and stop automatic blocking for images on frequently targeted accounts and major accounts (e.g. If someone is reporting Donald Trump, manually check, don't automatically remove).
Doesn't this system already exist? But reporting an image means it wasn't preemptively blocked, the benefit of the new system.
If you can send only the fingerprint, what stops you from uploading photos you are not in?
A better idea may be an uploading tool where you can obscure some parts for verification.
A better idea may be an uploading tool where you can obscure some parts for verification.
That sounds like a solution to me, it satisfies whatever the need is for human involvement while enabling the user to redact the weaponizable elements of the picture.
That's an interesting idea, but might break whatever hashing method is used?
The idea is to hash the original image, but to send the obscured one for human verification. However this requires a different hashing method (such a Merkle tree) to know the unobscured parts match.
Seems like anybody can upload any body.
isn't nudity generally banned on facebook anyway? I didn't realise this would/could be an issue. Unless they're talking about private communications in eg. messenger.
Revenge porn isn't all about nudity. And this does cover Messenger (and Instagram). The point isn't to get an image taken down after the fact, but to proactively block it from being posted or sent in the first place.
I thought facebook would run posted images through their AI nudity filter before allowing them to be posted? It's good that this covers messenger as I've read stories in the past of people being blackmailed over it, but surely there must be a better way than asking people to upload their private images to facebook. Plus, what's the case with younger facebook members? Do they upload the images and are therefore sharing child pornography with facebook, or not upload and don't get to be protected from revenge porn postings, when they're one of the most at-risk demographics?
I didn't know Facebook had an AI nudity filter, I thought it was all based on reported images. They softened their stance on some things, especially after censoring a Pulizter-prize winning photo from Vietnam, and eventually changed their policy on breastfeeding too https://www.facebook.com/help/340974655932193/
I'm sure they don't have a legal exception for underage photos, but those laws do vary from state to state.
I'm sure they don't have a legal exception for underage photos, but those laws do vary from state to state.
It's probably infeasible to run that type of filter in-path during uploads without adding too much latency to posting, wheras the hashing approach is much less expensive. AI filters likely take down images after they've already been posted
Why do they need a human representative to hash the image? I don't understand why this can't all be done automatically. Creeps me out that a real human would see intimate pictures before they block them.
I'm _assuming_ the only reason a human representative is doing this is so the system doesn't get abused. Like say I upload a photo claiming it is of myself but it's actually the advertisement a competitor of mine is using. If they didn't verify it was something that SHOULD be removed they'd automatically remove my competitor's posting of their advertisement every time they posted it.
Granted I really wish this was all done on the client so Facebook didn't gain access to the images themselves but I'm not sure of a good way around it to verify the image is something they should remove and not an abuse of the system.
Granted I really wish this was all done on the client so Facebook didn't gain access to the images themselves but I'm not sure of a good way around it to verify the image is something they should remove and not an abuse of the system.
they'd automatically remove my competitor's posting of their advertisement every time they posted it
Since that will be very rare case why not make that the human-required step?
Since that will be very rare case why not make that the human-required step?
Why would that be a "very rare" case? Facebook and Twitter have (or have had) hundreds of thousands of bots and constant abuses, why would this feature be rarely abused?
I'd assume that would be dealt with by an appeals process on the other end; if you post an image and it's flagged, you should be able to say "this is obviously not porn" and get it posted and the original false claimer should lose trust points.
Problem is it's significantly easier and faster to create noise than it is to clean up the noise. Dropping the up front review in favor of an appeals only system sounds impossible to handle at Facebook scale, IMO. Every time you shut someone's account down for losing too much trust, 10 others would have already replaced it.
Because otherwise the platform will not be able to store or display images at all.
Report, report, report, report, report - you know someone will do it, or make a bot to.
Or they would need to put actual effort into developing a novel solution to a new-ish problem. But they're a large incumbent, so...no.
Report, report, report, report, report - you know someone will do it, or make a bot to.
Or they would need to put actual effort into developing a novel solution to a new-ish problem. But they're a large incumbent, so...no.
[deleted]
Because otherwise people will just submit random picture for the lulz
Ok, so now all the harassers out there are just going to edit a few pixels on the images before uploading, so the hash isn't caught. All the same tricks use don YouTube to avoid the copyright bot, will work here as well.
This is likely the same image hashing they use to detect child pornography, and there's been a lot of work invested into making sure that those types of filters don't prevent the detection. You can very effectively require that the "real" image is visually entirely dissimilar.
Now, obviously you can't prevent the actual data from being distributed. For example, you could base64 encode the image and send it as a series of messages instead.
The motivation for revenge porn is to hurt the other person. One vector for this is to post it to all of your mutual friends so that the victim is shamed. This adds a barrier so that the mutual friends have to invest work into getting and distributing the porn, and won't accidentally see it flowing through their feed.
Now, obviously you can't prevent the actual data from being distributed. For example, you could base64 encode the image and send it as a series of messages instead.
The motivation for revenge porn is to hurt the other person. One vector for this is to post it to all of your mutual friends so that the victim is shamed. This adds a barrier so that the mutual friends have to invest work into getting and distributing the porn, and won't accidentally see it flowing through their feed.
I’m not saying that people won’t be able to fool it, but I’m sure it would require way more than just changing a few pixels. I’ve seen YouTube videos that were clearly edited to circumvent the copyright bot, and the video transformations were so significant that the videos were essentially unwatchable.
Yeah I'm curious how their hashing mechanism works. Like, is it a straight up sha hash of the image or something a little more sophisticated that wouldn't be fooled by minor edits? Though in the latter case can that even technically be considered hashing? I guess it all depends on how they're doing it which brings us back to my first question...
Hash by strictest definition is just a function that maps any input to fixed size output. There is a special subcategory of hashes that are locality sensitive and change a little when input changes a little, just like there is a special subcategory of crypto hashes. See: https://en.wikipedia.org/wiki/Locality-sensitive_hashing
Technology for searching for similar images (I have no idea how it works interally) already exists and is very widely deployed. You can easily find an image similar (uncropped, cropped, with different filter, greyed, ungreyed, with a logo in corner, etc.) to your input image on https://tineye.com/ and https://images.google.com .
Technology for searching for similar images (I have no idea how it works interally) already exists and is very widely deployed. You can easily find an image similar (uncropped, cropped, with different filter, greyed, ungreyed, with a logo in corner, etc.) to your input image on https://tineye.com/ and https://images.google.com .
It is called PhotoDNA and was developed by Microsoft. This is the same tech that is used to find child porn images even when people go to various lengths to prevent simple hashing from being able to determine photo similarity. It works surprisingly well at the task from what I have seen/heard.
Photo dna is just one of a class of algorithms called perceptual image hashes (there are also perceptual audio and video hash algorithms). They by no means invented the technique, just one particular branded implementation of it.
They're (more than likely - almost certainly) using proprietary image fingerprinting techniques.
If you had access to the original implementation, you could probably defeat them permanently with minimal alteration via reverse engineering. Which is why you don't have access to the original software, its specifications, or implementation details.
(https://twitter.com/alexstamos/status/928646228472078336)
If you had access to the original implementation, you could probably defeat them permanently with minimal alteration via reverse engineering. Which is why you don't have access to the original software, its specifications, or implementation details.
(https://twitter.com/alexstamos/status/928646228472078336)
I have no knowledge of the implementation, but I think it's more than likely a safe bet that someone at Facebook probably thought of this and chose a slightly more sophisticated approach. The entire system would be completely pointless otherwise!
md5/sha isn't the only hashing method. There are perceptual hashes which are fairly resilient to simply editing few pixels, or rotating/resizing/cropping an image.
To everyone asking why they need a human to review to image before hashing it: it's currently the only way to prevent abuse of this system -- Without this, everyone would upload random images that would get taken down, effectively mass-trolling
At most, they would only need a human to review after the hash matches, not before.
Facebooks plan is awful.
Facebooks plan is awful.
The actual reason is probably more technical:
https://twitter.com/alexstamos/status/928646228472078336
(You'd reverse engineer the algorithm and permanently defeat it - rendering all fingerprints stored useless)
https://twitter.com/alexstamos/status/928646228472078336
(You'd reverse engineer the algorithm and permanently defeat it - rendering all fingerprints stored useless)
Something just doesn't add up here.
> To establish which image is of concern, people will be asked to send the image to themselves on Messenger.
> Once we receive this notification, a specially trained representative from our Community Operations team reviews and hashes the image, which creates a human-unreadable, numerical fingerprint of it.
This clearly implicates that Facebook has, almost without reserve, the ability to read messages from user to user (even though it might limited to messages to oneself), and exposes that ability to its employees.
While I stopped using Facebook a long time ago, Zuckerberg's quote, "they 'trust me'; dumb fucks" seems relevant here. If I were Facebook, I would do this in a different, much more privacy conscious way.
1. Obviously, Facebook is already hashing all images on its platforms and storing the hashes. Given that, let the user, using a frontend only platform, upload an image and generate the hash. Make sure this image is a real photo (not a meme or something heavily photoshopped) in addition to making sure:
1a) The image contains a human[0]. Fairly easily doable for most situations.
1b) If not, let the user know that "we can't automatically detect a human in this photo, do you want to still submit this photo for removal, this could get a strike against your account if (not in some list of reasons for removal)". Maybe the photo has a picture of a credit card or something.
2. Submit the hash to the backend. Given that hash is not used on the platform past a significant amount of views, automatically ban the photo, and allow other users to petition for reinstatement of the photo, knowing that a non-valid petition for reinstatement (the user doesn't have rights to the photo) COULD result in a strike also.
3. Have humans review the petitions, and in cases where the users explicitly allow, the images.
Doing this would limit another human seeing the intimate photo in well over 90% of the cases I'd bet. In addition to that, even if the photo is shared with another human, it would let the end user decide if they'd like a Facebook employee to view the image too. IE, basic user privacy. Come on Facebook. Dumb fucks.
[0] very doable, https://trackingjs.com/
> Once we receive this notification, a specially trained representative from our Community Operations team reviews and hashes the image, which creates a human-unreadable, numerical fingerprint of it.
This clearly implicates that Facebook has, almost without reserve, the ability to read messages from user to user (even though it might limited to messages to oneself), and exposes that ability to its employees.
While I stopped using Facebook a long time ago, Zuckerberg's quote, "they 'trust me'; dumb fucks" seems relevant here. If I were Facebook, I would do this in a different, much more privacy conscious way.
1. Obviously, Facebook is already hashing all images on its platforms and storing the hashes. Given that, let the user, using a frontend only platform, upload an image and generate the hash. Make sure this image is a real photo (not a meme or something heavily photoshopped) in addition to making sure:
1a) The image contains a human[0]. Fairly easily doable for most situations.
1b) If not, let the user know that "we can't automatically detect a human in this photo, do you want to still submit this photo for removal, this could get a strike against your account if (not in some list of reasons for removal)". Maybe the photo has a picture of a credit card or something.
2. Submit the hash to the backend. Given that hash is not used on the platform past a significant amount of views, automatically ban the photo, and allow other users to petition for reinstatement of the photo, knowing that a non-valid petition for reinstatement (the user doesn't have rights to the photo) COULD result in a strike also.
3. Have humans review the petitions, and in cases where the users explicitly allow, the images.
Doing this would limit another human seeing the intimate photo in well over 90% of the cases I'd bet. In addition to that, even if the photo is shared with another human, it would let the end user decide if they'd like a Facebook employee to view the image too. IE, basic user privacy. Come on Facebook. Dumb fucks.
[0] very doable, https://trackingjs.com/
This clearly implicates that Facebook has, almost without reserve, the ability to read messages from user to user (even though it might limited to messages to oneself), and exposes that ability to its employees.
You can view and search your message history on Facebook, so there is not really any question that they have and can read all messages. I never explicitly checked it but at least I never noticed any gaps, i.e. missing messages because I sent them using the Messenger app.
I believe to remember that Messenger is supposed to use Axolotl Ratchet for end-to-end encryption but that is hard to reconcile with the availability of your message history on facebook.com. So maybe it's - quote - end-to-end - unquote - between the phone and the server?
I never really thought about that. Or is it just not available or disabled in my Windows Phone version of the app?
You can view and search your message history on Facebook, so there is not really any question that they have and can read all messages. I never explicitly checked it but at least I never noticed any gaps, i.e. missing messages because I sent them using the Messenger app.
I believe to remember that Messenger is supposed to use Axolotl Ratchet for end-to-end encryption but that is hard to reconcile with the availability of your message history on facebook.com. So maybe it's - quote - end-to-end - unquote - between the phone and the server?
I never really thought about that. Or is it just not available or disabled in my Windows Phone version of the app?
You can download a complete copy of the data Facebook has on you, including all messages you've sent since your account was created [1]. I did this some years ago to graph changes in sentiment with friends.
1. https://www.facebook.com/help/131112897028467
1. https://www.facebook.com/help/131112897028467
Well, all data Facebook has on you that you uploaded, if other people upload your address, that's apparently their data.
It's not that infeasible to have a web platform that encrypts users messages, just each client gets an encryption key. Look at Telegram, or even FB Messenger end to end encryption. I'm not saying they ever said they didn't have the ability to do it, I'm saying it's weird that they would use "message snooping" as a form of user photo submission.
When I send messages using the Messenger app - which is supposed to use end-to-end encryption, isn't it? - I can see and search those messages on facebook.com. Something does not add up here. And if they have the keys on their servers, then it's rather pointless to use end-to-end encryption in the first place.
facebook only does end-to-end for "secret" conversations started in their mobile app.
[1] https://www.facebook.com/help/messenger-app/1084673321594605...
[1] https://www.facebook.com/help/messenger-app/1084673321594605...
That explains it and it seems unavailable in the Windows Phone version. And because I never owned or used an iPhone or Android device I was unaware of this difference.
Using Messenger you can start a secret conversation with someone which is encrypted end-to-end. Those conversations only exist in the current device. But regular conversations are not encrypted in any way.
They are sent over an encrypted transport which means law enforcement must subpoena them from Facebook, and cannot monitor them with a telephone intercept like calls and SMSes.
> This clearly implicates that Facebook has, almost without reserve, the ability to read messages from user to user (even though it might limited to messages to oneself), and exposes that ability to its employees.
Why is that surprising? In the absence of end-to-end encryption (not the default for Messenger, and can only be enabled on a conversation-by-conversation basis) where the keys never leave the endpoints, how do you design a system where literally no employee can access particular bits of data?
I'm sure FB doesn't let just any employee read people's messages, and likely such access is doled out on a case-by-case basis, and isn't a global can-read-anything permission.
Why is that surprising? In the absence of end-to-end encryption (not the default for Messenger, and can only be enabled on a conversation-by-conversation basis) where the keys never leave the endpoints, how do you design a system where literally no employee can access particular bits of data?
I'm sure FB doesn't let just any employee read people's messages, and likely such access is doled out on a case-by-case basis, and isn't a global can-read-anything permission.
> This clearly implicates that Facebook has, almost without reserve, the ability to read messages from user to user (even though it might limited to messages to oneself), and exposes that ability to its employees.
Why do you seem surprised by this? I can't fathom anything about Facebook's design that implies my posted content is hidden from employees.
Why do you seem surprised by this? I can't fathom anything about Facebook's design that implies my posted content is hidden from employees.
This clearly implicates that Facebook has, almost without reserve, the ability to read messages from user to user (even though it might limited to messages to oneself), and exposes that ability to its employees.
No, it implies that some Facebook employees have the ability to review messages which are reported as being abusive. Which is... well, kind of obvious. I mean, what did you think was going to happen when you report a message?
(I suspect that some employees have the ability to look at far more messages than just the ones which are reported, but that is not implied by what they're telling us here.)
No, it implies that some Facebook employees have the ability to review messages which are reported as being abusive. Which is... well, kind of obvious. I mean, what did you think was going to happen when you report a message?
(I suspect that some employees have the ability to look at far more messages than just the ones which are reported, but that is not implied by what they're telling us here.)
I really don't understand why anyone uses Facebook.
You have a phone, which can communicate with your friends. It's only remotely useful for its near-monopoly on event planning, and imo that's a job for the trustbusters.
You have a phone, which can communicate with your friends. It's only remotely useful for its near-monopoly on event planning, and imo that's a job for the trustbusters.
> I really don't understand why anyone uses Facebook.
You really cannot? Like, can you not imagine one person, who doesn't realize (or worse, and more likely IMO, does not care) about the privacy implications?
It's not hard to understand at all, I think.
You really cannot? Like, can you not imagine one person, who doesn't realize (or worse, and more likely IMO, does not care) about the privacy implications?
It's not hard to understand at all, I think.
Okay, fine, I don't want to understand; it jives against the charitable way that I want to think about people.
I guess there's a whole lot of cognitive dissonance between how fucking stupid People reliably are and how kind and admirable persons can be, though - I should probably just get used to it.
I guess there's a whole lot of cognitive dissonance between how fucking stupid People reliably are and how kind and admirable persons can be, though - I should probably just get used to it.
Compared to a phone (SMS) and other messaging apps (whatsapp, email, signal), fb messenger has two big benefits for me:
* Cross platform: I don't have to type on a tiny screen when I'm in front of a computer, and won't lose my account or my messages if I lose my phone or phone number.
* Discoverability: most people who I meet are on it, so I can reach out to people I meet at events as well as friends of friends.
I don't love messenger. I use Messenger Lite on my phone to avoid most of the snapchat/gif/events/birthdays features. But I'm not aware of any other cross-platform messaging app with that discoverability, let alone one free of privacy issues.
* Cross platform: I don't have to type on a tiny screen when I'm in front of a computer, and won't lose my account or my messages if I lose my phone or phone number.
* Discoverability: most people who I meet are on it, so I can reach out to people I meet at events as well as friends of friends.
I don't love messenger. I use Messenger Lite on my phone to avoid most of the snapchat/gif/events/birthdays features. But I'm not aware of any other cross-platform messaging app with that discoverability, let alone one free of privacy issues.
Which is sort of weird, isn't it? As little as 10-15 years ago there was a huge flourishing ecosystem of IM and chat clients, many of which worked together fairly seamlessly. Individual clients like Pidgin would happily amalgamate several platforms together. Now, at least in terms of usage, it's a blasted hellscape of a one-pony show - what happened?
[deleted]
>Cross platform: I don't have to type on a tiny screen when I'm in front of a computer, and won't lose my account or my messages if I lose my phone or phone number.
email isn't cross platform?
>Discoverability: most people who I meet are on it, so I can reach out to people I meet at events as well as friends of friends.
doesn't everyone have an email address?
email isn't cross platform?
>Discoverability: most people who I meet are on it, so I can reach out to people I meet at events as well as friends of friends.
doesn't everyone have an email address?
You can’t just type someone’s name into the To field of an email.
I don’t think that’s accurate. I believe for this specific feature you need to specifically report the message from within the Messenger UI, similarly to how you would report spam or other abuse. Only then is it sent to the human reps. I don’t think other messages are ever read.
I’m a FB employee and it’s definitely not possible for employees to easily snoop on random messages. Even if you found some way that isn’t properly protected, you’d get promptly fired if caught.
I’m a FB employee and it’s definitely not possible for employees to easily snoop on random messages. Even if you found some way that isn’t properly protected, you’d get promptly fired if caught.
No, you have to opt in to this feature, hence consenting to let an employee read your message. It does not imply that they can read messages without your consent.
[A]: human can review the image they already have
[B]: Facebook waits until someone uploads an image that matches and then reviews the image (as normal) but with a marker alerting the problem.
The benefit is that people NOT affected won't have to upload lots of images of themselves to facebook personal for "review" just to be sure.
The problem is that should facebook update it's hashing or find better ways to match images, having the original image would allow them to transition. This point is moot though since facebook claims not to save the image anyway.