WhatsApp moderators can read your messages if someone you talk to reports them(gizmodo.com)
gizmodo.com
WhatsApp moderators can read your messages if someone you talk to reports them
https://gizmodo.com/whatsapp-moderators-can-read-your-messages-1847629241
59 comments
Some people have started using Whatsapp for Telemarketing (message based, not voice). I frequently use the report option to report them.
So I feel like there are reasons to have that.
So I feel like there are reasons to have that.
2 billion people use Whatsapp, some of them for purposes that are not as pedestrian as yours.
Private Whatsapp groups can include thousands of people and the topics can be anything from retail discount alerts, to COVID updates, to political organizing. That absolutely merits some kind of reporting capability, particularly if criminal activity is being conducted or is about to be conducted.
Private Whatsapp groups can include thousands of people and the topics can be anything from retail discount alerts, to COVID updates, to political organizing. That absolutely merits some kind of reporting capability, particularly if criminal activity is being conducted or is about to be conducted.
Alright well I'm not a WhatsApp user, so I have a couple of questions:
In these large groups that obviously need moderation, are the moderators appointed in some way by WhatsApp, or are they like classical rooms in that the founders of the room pick the moderation team.
Also, do one to one rooms have reporting functionality and who picks the moderator that it is reported to?
In these large groups that obviously need moderation, are the moderators appointed in some way by WhatsApp, or are they like classical rooms in that the founders of the room pick the moderation team.
Also, do one to one rooms have reporting functionality and who picks the moderator that it is reported to?
Is your email private when recipient can take a screenshot and report it to the police? Is your snail mail private if recipient can take it to the police? The end to end encryption (or the envelope you put your snail mail into) only means that nobody but you and recipient(s) can read the content. It doesn’t mean that recipients can’t share said content with 3rd parties.
I don't think this is as complicated or hard to understand as this journalist is making out. Any content in a messaging system can be shared by either party with third parties, even if it's encrypted end to end. Hitting the "report" button is just one example of that, sharing the last few messages with WhatsApp moderators.
If you were to ask a (remote) contact "please delete this conversation as it is confidential" how can you be sure they have deleted it?
From the article "WhatsApp can read some of your messages if the recipient reports them."
Is this surprising? Any third party can read part of an e2e encrypted communication if one of the participants forwards it.
Is this surprising? Any third party can read part of an e2e encrypted communication if one of the participants forwards it.
"AI" running on every client can automatically flag messages and send them to moderators.
> Most can agree that violent imagery and CSAM should be monitored and reported; Facebook and Pornhub regularly generate media scandals for not moderating enough. But WhatsApp moderators told ProPublica that the app’s artificial intelligence program sends moderators an inordinate number of harmless posts, like children in bathtubs. Once the flagged content reaches them, ProPublica reports that moderators can see the last five messages in a thread.
> Most can agree that violent imagery and CSAM should be monitored and reported; Facebook and Pornhub regularly generate media scandals for not moderating enough. But WhatsApp moderators told ProPublica that the app’s artificial intelligence program sends moderators an inordinate number of harmless posts, like children in bathtubs. Once the flagged content reaches them, ProPublica reports that moderators can see the last five messages in a thread.
The problem here is that the third party controls the software on both ends of the communication. And that software can send the messages to this party without the participants knowingly triggering it.
The article says that by reporting a user, the software on the site of the reporting user silently sends data to WhatsApp. The reporting user does not know what data is sent.
The article says that by reporting a user, the software on the site of the reporting user silently sends data to WhatsApp. The reporting user does not know what data is sent.
its not silent, when you report messages there is a prompt that tells the reporter what is happening
take a look https://twitter.com/WABetaInfo/status/1435221936888483847
take a look https://twitter.com/WABetaInfo/status/1435221936888483847
Yeah it's not really too surprising, except that maybe the scale of what gets shared with Facebook is a bit unclear.
I'm not too sure at what point the artificial intelligence program gets involved though.
I'm not too sure at what point the artificial intelligence program gets involved though.
'a bit unclear' like in having got it half right like in simply unknown ?!
The article quotes the terms of service which say that they send:
>"the most recent messages”
and
>“information on your recent interactions with the reported user.”
which is unclear, but not unknown, and as far as this article claims they don't actually send anything else, though they do combine it with whatever metadata they have on the users involved.
>"the most recent messages”
and
>“information on your recent interactions with the reported user.”
which is unclear, but not unknown, and as far as this article claims they don't actually send anything else, though they do combine it with whatever metadata they have on the users involved.
There seem to be two things happening.
When a user reports a post it is (unsurprisingly) forwarded to the moderators.
Additionally, there is some kind of AI CSAM detector, which automatically forwards posts.
In both cases, it also forwards the previous five messages from the thread to the moderators.
When a user reports a post it is (unsurprisingly) forwarded to the moderators.
Additionally, there is some kind of AI CSAM detector, which automatically forwards posts.
In both cases, it also forwards the previous five messages from the thread to the moderators.
> Seated at computers in pods organized by work assignments, these hourly workers use special Facebook software to sift through streams of private messages, images and videos that have been reported by WhatsApp users as improper and then screened by the company’s artificial intelligence systems.
> Instead, WhatsApp reviewers gain access to private content when users hit the “report” button on the app, identifying a message as allegedly violating the platform’s terms of service. This forwards five messages — the allegedly offending one along with the four previous ones in the exchange, including any images or videos — to WhatsApp in unscrambled form, according to former WhatsApp engineers and moderators. Automated systems then feed these tickets into “reactive” queues for contract workers to assess.
From the actual ProPublica report. If their published understanding is correct, E2EE is not broken, but rather end users who are one of the ends of E2EE are sending the decrypted content to be moderated. The AI bit is a filter to reduce the amount of content passed on to human moderators.
From near that second quote:
> Artificial intelligence initiates a second set of queues — so-called proactive ones — by scanning unencrypted data that WhatsApp collects about its users and comparing it against suspicious account information and messaging patterns (a new account rapidly sending out a high volume of chats is evidence of spam), as well as terms and images that have previously been deemed abusive.
That part is AI driven, but my reading is that the moderators do not get access to the encrypted data (the actual messages) only the behavior patterns, and from that make a determination of what to do.
> Instead, WhatsApp reviewers gain access to private content when users hit the “report” button on the app, identifying a message as allegedly violating the platform’s terms of service. This forwards five messages — the allegedly offending one along with the four previous ones in the exchange, including any images or videos — to WhatsApp in unscrambled form, according to former WhatsApp engineers and moderators. Automated systems then feed these tickets into “reactive” queues for contract workers to assess.
From the actual ProPublica report. If their published understanding is correct, E2EE is not broken, but rather end users who are one of the ends of E2EE are sending the decrypted content to be moderated. The AI bit is a filter to reduce the amount of content passed on to human moderators.
From near that second quote:
> Artificial intelligence initiates a second set of queues — so-called proactive ones — by scanning unencrypted data that WhatsApp collects about its users and comparing it against suspicious account information and messaging patterns (a new account rapidly sending out a high volume of chats is evidence of spam), as well as terms and images that have previously been deemed abusive.
That part is AI driven, but my reading is that the moderators do not get access to the encrypted data (the actual messages) only the behavior patterns, and from that make a determination of what to do.
Correct me if I'm wrong but unless the "AI CSAM detector" is running on the client, it simply cannot be e2e encrypted.
It sounds like the only unencrypted data that the moderators see is sent from an endpoint (a user clicking "report"). After that an AI looks at the report and prioritizes ones that looks like it might be CSAM.
Yes, so I assumed it is running on the client, but for all I know they could be encrypting the message and sending an image hash to Facebook.
It looks like the AI stuff applies to the groups content which is not E2E.
> But WhatsApp moderators told ProPublica that the app’s artificial intelligence program sends moderators an inordinate number of harmless posts, like children in bathtubs. Once the flagged content reaches them, ProPublica reports that moderators can see the last five messages in a thread.
It's not just when a recipient reports them it seems but also when they have been flagged by their algorithm. If that were true, the claim that the conversation is e2e encrypted simply cannot be true, unless the algorithm runs on the client.
It's not just when a recipient reports them it seems but also when they have been flagged by their algorithm. If that were true, the claim that the conversation is e2e encrypted simply cannot be true, unless the algorithm runs on the client.
just think about the sheer number of people that report stuff.
given that facebook has less than 1k moderators, do you honestly think that they'd just let the moderators sift through everything manually?
obviously you'd classify stuff first, checking against known images is easy. Classifying new images is a lot harder, plus the ethics of training and labelling a dataset for accurate detection is pretty hard, also almost impossible to do legally.
I suspect the next best thing is detecting nudity and age of the subject, and taking the hit that you're going to prioritise a lot of malicious reports, rather than genuine.
given that facebook has less than 1k moderators, do you honestly think that they'd just let the moderators sift through everything manually?
obviously you'd classify stuff first, checking against known images is easy. Classifying new images is a lot harder, plus the ethics of training and labelling a dataset for accurate detection is pretty hard, also almost impossible to do legally.
I suspect the next best thing is detecting nudity and age of the subject, and taking the hit that you're going to prioritise a lot of malicious reports, rather than genuine.
It sounds to me that there’s actually an algorithm between the report and the moderator to control the volume of manual moderation.
What you’re describing doesn’t work with E2E encryption. I really doubt it works that way.
What you’re describing doesn’t work with E2E encryption. I really doubt it works that way.
>What you’re describing doesn’t work with E2E encryption
This smells like message franking, but I can’t be sure.
https://eprint.iacr.org/2017/664.pdf
This smells like message franking, but I can’t be sure.
https://eprint.iacr.org/2017/664.pdf
Is there any particular reason to believe its not running on the client?
because that would mean running a fairly large model on underpowered hardware. Also it would mean that you could never actually trust the output.
its far far more simple to run it server side on the reported message.
its far far more simple to run it server side on the reported message.
[deleted]
E2E is useless if the software is not opensource. Especially if you don't trust the vendor
> But WhatsApp moderators told ProPublica that the app’s artificial intelligence program sends moderators an inordinate number of harmless posts, like children in bathtubs. Once the flagged content reaches them, ProPublica reports that moderators can see the last five messages in a thread.
It seems like whatever the AI decides is questionable gets sent to moderators automatically, also revealing the thread history (at least partially) as well.
It seems like whatever the AI decides is questionable gets sent to moderators automatically, also revealing the thread history (at least partially) as well.
Its all about signal to noise.
THose who are claiming that they are running Machine learning at the edge, think about this:
how on earth can they trust the data coming from it?
We all know that ML is hard, we also know that to get accurate classification requires serious horsepower.
None of this can be run on phone hardware without people noticing.
If you were to design a system for handling reported data, who's hard constraint is _human_ eyeball time, would you run ML at the edge? No.
You'd make it so the "report this message" sent the last n lines and the attachment, and then run the ML optimised for accuracy (not speed or memory) on your hardware that you control and trust explicitly.
I would have thought all of this is obvious, given that facebook is all about avoiding spending money on human moderators. Edge ML is not going to help you do that.
THose who are claiming that they are running Machine learning at the edge, think about this:
how on earth can they trust the data coming from it?
We all know that ML is hard, we also know that to get accurate classification requires serious horsepower.
None of this can be run on phone hardware without people noticing.
If you were to design a system for handling reported data, who's hard constraint is _human_ eyeball time, would you run ML at the edge? No.
You'd make it so the "report this message" sent the last n lines and the attachment, and then run the ML optimised for accuracy (not speed or memory) on your hardware that you control and trust explicitly.
I would have thought all of this is obvious, given that facebook is all about avoiding spending money on human moderators. Edge ML is not going to help you do that.
I think the AI was applied against non-E2E encrypted data like groups.
this says that when you report a message, it sends over the last 5 messages
https://twitter.com/WABetaInfo/status/1435221936888483847
The pro publica article is pure click bait
https://twitter.com/WABetaInfo/status/1435221936888483847
The pro publica article is pure click bait
This doesn't at all imply that the same isn't happening upon automatic detection of "reportable" messages.
The sadness of my life is that I'm probably bound to be repeating this until I let out my final breath:
If you're not running the service yourself and taking care of end-to-end encryption, somebody can (and probably will) read your messages. Actual privacy of messaging does not exists today. Don't even bother looking for that.
EDIT: just to clarify, it might just be that somebody can access your data for a legitimate purpose (say an operator of the service) and then leak/sell the data.
This is the simplest thing that can happen and that DID happen. The united arab emirates paid a Twitter employee from UAE to leak data about anti-UAE-government from Twitter. People died because of this.
If you're not running the service yourself and taking care of end-to-end encryption, somebody can (and probably will) read your messages. Actual privacy of messaging does not exists today. Don't even bother looking for that.
EDIT: just to clarify, it might just be that somebody can access your data for a legitimate purpose (say an operator of the service) and then leak/sell the data.
This is the simplest thing that can happen and that DID happen. The united arab emirates paid a Twitter employee from UAE to leak data about anti-UAE-government from Twitter. People died because of this.
TL;DR: WhatsApp is E2E encrypted, but that doesn't mean there are only 2 ends.
No, that's not what the article is saying. There are two ends, but if one of the ends decides to forward the content to WhatsApp, then WhatsApp can read it. Which isn't shocking or breaking E2EE in any way.
How does one confirm how many “ends” are there in a conversation? If WhatsApp secretly set themselves (or another third party) up as an “end” in every conversation , that would also not break break E2EE.
Sure, but that's not what's being reported here.
My comment was not about the original article. It was directed at the comment above me which stated "There are two ends ...".
> There are two ends, but if one of the ends decides to forward the content to WhatsApp, then WhatsApp can read it.
Translation: there are two ends, but one of the ends can send it to yet another end. That sounds like there are more than two ends.
Translation: there are two ends, but one of the ends can send it to yet another end. That sounds like there are more than two ends.
Ok, but if we use this definition, then “somebody took a screenshot of my Signal chat and posted it on Twitter” is proof that “Twitter can read your Signal messages”. That doesn’t seem like a very practical definition.
A screenshot can be forged, meanwhile Facebook has reasonable certainty about the authenticity of the message. This third end is within the WhatsApp platform, so we're not really comparing the same thing.
It's essentially a forward to the party that supposedly can't see your messages, breaking what people understand by E2E.
Today this behavior is triggered when it's reported, tomorrow it'll be when the government, or worse, some AI, flags a user. You share a groupchat with the wrong person, or they have you in their contact list, boom, eavesdropped.
But some people will still claim it's only 2 ends just because it's not forwarded 100% of the time. I guess it's matter of semantics.
It's essentially a forward to the party that supposedly can't see your messages, breaking what people understand by E2E.
Today this behavior is triggered when it's reported, tomorrow it'll be when the government, or worse, some AI, flags a user. You share a groupchat with the wrong person, or they have you in their contact list, boom, eavesdropped.
But some people will still claim it's only 2 ends just because it's not forwarded 100% of the time. I guess it's matter of semantics.
Can those who claimed that WhatsApp was infallible because it was E2E-encrypted take a step back and see that incentives matters as much as technology?
Please?
Please?
Hm, and just recently WhatsApp management mocked Apple, bragged about their encryption, and said how they would never do anything like that.
https://twitter.com/wcathcart/status/1423701473624395784?s=2...
https://twitter.com/wcathcart/status/1423701473624395784?s=2...
They were critical of the client side scanning. So implementation as opposed to the goal. And also a marketing opportunity to take a pop at Apple due to their hubris over privacy and security.
I read this as a user, whom has received and decrypted an e2ee message, can report it. This will send it, e2ee, to WhatsApp HQ. no breakage here. E2ee is no panacea.
Boggles my mind that people see this as Facebook reading their encrypted messages. The person you sent the message to chose to forward the message to Facebook and are even told that this is happening when they report the message.
You can't stop someone who knows what a message says from sharing the contents of that message without physically restraining and gagging them. They can send a screenshot out, take a photo of the screen, write down the message on paper, or just memorize it and tell others.
You can't stop someone who knows what a message says from sharing the contents of that message without physically restraining and gagging them. They can send a screenshot out, take a photo of the screen, write down the message on paper, or just memorize it and tell others.
Boggles mine that people believe proprietary platforms blindly. It can't be proved that there's no backdoor added after the initial consulting for OpenWhisper.
Gee, seeing how they defended Whatsapp tooth and nail time and time again, I'm inclined to not be surprised if it was added _during_.
Gee, seeing how they defended Whatsapp tooth and nail time and time again, I'm inclined to not be surprised if it was added _during_.
Whether or not they're siphoning information some other way is not the point here. The accusations here are directed at how reporting content sends the decrypted messages to Facebook. The ridiculousness is that people are treating that reporting functionality as a secret data leakage channel.
If you think that they have backdoor access to messages, please investigate that, submit your findings as a different HN post, and then we can discuss it.
If you think that they have backdoor access to messages, please investigate that, submit your findings as a different HN post, and then we can discuss it.
Exactly. Your e2e message is encrypted from your end to the recipents end. They then decrypt the message and forward it to someone else.
Nothing new here. I really dont get what the expectation of these things are for people. Are they not aware, someone can forward a message to a thrid party, people can and do screenshot text and forward it. Writing a message, and sending it encrypted to someone else doesnt protect you from what they might do with the message they now have.
Nothing new here. I really dont get what the expectation of these things are for people. Are they not aware, someone can forward a message to a thrid party, people can and do screenshot text and forward it. Writing a message, and sending it encrypted to someone else doesnt protect you from what they might do with the message they now have.
The part that aggravates me is that alternatives have their own issues. I managed to get my mom from whatsapp to signal, but when ( I wanted to say if, but that is wishful thinking ) it turns out it is no longer viable I know I will have a hard time migrating her. And even if I do manage to move her, it will be a painful process of re-learning new system for her.
When you press the report button, a confirmation message is shown explicitly saying that the last 5 messages will be sent to WhatsApp without the other user knowing. You can then accept or cancel.
You can also copy the whole conversation and send it to the police, or post them on a forum, etc.
WhatsApp moderators can't read your messages, but they can read what the person you talk to send them. E2E encryption means that only the ends should be able to decrypt and read the contents, but what they do with them is another different topic.
You can also copy the whole conversation and send it to the police, or post them on a forum, etc.
WhatsApp moderators can't read your messages, but they can read what the person you talk to send them. E2E encryption means that only the ends should be able to decrypt and read the contents, but what they do with them is another different topic.
There is no proof their OpenWhisper implementation wasn't tampered with. And chances are against users, since it's proprietary software.
> WhatsApp moderators can't read your messages
How do you know?
How do you know?
Alex Stamos twitter thread on the original ProPublica story has some interesting points about the tradeoffs when building these kinds of systems.
https://twitter.com/alexstamos/status/1435285445336715265
I don't feel like it's unreasonable for a company to have a system where a user can say "Hey someone is sending me something unwanted using your service" and for that company to use technology to sort those complaints for humans to review and action appropriately based on their terms of service.
I don't feel like it's unreasonable for a company to have a system where a user can say "Hey someone is sending me something unwanted using your service" and for that company to use technology to sort those complaints for humans to review and action appropriately based on their terms of service.
If you and I are texting, privately, and you say something to me I don't like, I'll tell you., or I will stop talking to you. That's healthy social interaction. If I report it instead, that's not healthy social interaction, and it appears to me the tech companies designing these solutions to problems that were solved a decade ago are more than happy to oblige because it gives them an excuse to surveil and censor. They're encouraging unhealthy social interaction because it serves their ends to do so.
If an instant messaging application has moderators it's not an instant messaging application.