Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’(vice.com)
vice.com
Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’
https://www.vice.com/en/article/wx5yzq/apple-defends-its-anti-child-abuse-imagery-tech-after-claims-of-hash-collisions
460 comments
What's shocking to me is how little Apple management understood of what their actions looked like. Really stunning.
For a company that marketed itself as one of the few digital service providers that consumers could trust, I just don't understand how they acted this way at all.
Either there will be heads rolling at management, or Apple takes a permanent hit to consumer trust.
For a company that marketed itself as one of the few digital service providers that consumers could trust, I just don't understand how they acted this way at all.
Either there will be heads rolling at management, or Apple takes a permanent hit to consumer trust.
> I just don't understand how they acted this way at all.
There's a simple answer to this right? Despite everyone's reaction, Apple genuinely believe this is a novel and unique method to catch CSAM without invading people's privacy. And if you look at it from Apple's point of view that's correct: other major cloud providers catch CSAM content on their platform by inspecting every file uploaded, i.e. total invasion of privacy. Apple found a way to preserve that privacy but still catch the bad people doing very bad things to children.
There's a simple answer to this right? Despite everyone's reaction, Apple genuinely believe this is a novel and unique method to catch CSAM without invading people's privacy. And if you look at it from Apple's point of view that's correct: other major cloud providers catch CSAM content on their platform by inspecting every file uploaded, i.e. total invasion of privacy. Apple found a way to preserve that privacy but still catch the bad people doing very bad things to children.
It's really interesting to see the mental gymnastics people are willing to go through to defend their favorite trillion dollar corporations.
> other major cloud providers catch CSAM content on their platform by inspecting every file uploaded, i.e. total invasion of privacy.
> Apple found a way to preserve that privacy ...
So scanning for CSAM in a third-party cloud is "total invasion of privacy", while scanning your own personal device is "preserving privacy"?
The third-party clouds can only scan what one explicitly chooses to share with the third-party, while on-device scanning is a short slippery slope away from have its scope significantly expanded (to include non-shared and non-CSAM content).
> other major cloud providers catch CSAM content on their platform by inspecting every file uploaded, i.e. total invasion of privacy.
> Apple found a way to preserve that privacy ...
So scanning for CSAM in a third-party cloud is "total invasion of privacy", while scanning your own personal device is "preserving privacy"?
The third-party clouds can only scan what one explicitly chooses to share with the third-party, while on-device scanning is a short slippery slope away from have its scope significantly expanded (to include non-shared and non-CSAM content).
They are scanning files that are being uploaded. So, yes.
As I mentioned, it is (by definition) impossible for the cloud to have access to something that does not exist in the cloud - only the content that is explicitly uploaded/shared by the user can be scanned.
While local, on-device content access has no such guarantees - it's a question of policy rather than technical infeasibility.
And policies change.
> They are scanning files that are being uploaded. So, yes.
Even considering the situation as it is today, the cloud providers (as well as Apple) are scanning just the files that are being uploaded. So I'm still not sure how one is "total invasion of privacy" and the other "preserving privacy".
While local, on-device content access has no such guarantees - it's a question of policy rather than technical infeasibility.
And policies change.
> They are scanning files that are being uploaded. So, yes.
Even considering the situation as it is today, the cloud providers (as well as Apple) are scanning just the files that are being uploaded. So I'm still not sure how one is "total invasion of privacy" and the other "preserving privacy".
It has always been technically feasible for them to have software on the phone that scans everything on the phone. I don't see how this changes anything. Apple is trying to avoid scanning in the cloud so that the data can be E2EE and not subject to being stolen by someone that gets access to their servers somehow.
> It has always been technically feasible for them to have software on the phone that scans everything on the phone.
No, the OS can limit access to local content. E.g. Mac has https://support.apple.com/guide/mac-help/control-access-to-f..., and ios has its own permissions system.
It also requires installing third-party software on the device. While if Apple's policy regarding local content changes in the future, there is no way to avoid the OS itself (other than jailbreaking I suppose).
> Apple is trying to avoid scanning in the cloud so that the data can be E2EE
Citation needed.
No, the OS can limit access to local content. E.g. Mac has https://support.apple.com/guide/mac-help/control-access-to-f..., and ios has its own permissions system.
It also requires installing third-party software on the device. While if Apple's policy regarding local content changes in the future, there is no way to avoid the OS itself (other than jailbreaking I suppose).
> Apple is trying to avoid scanning in the cloud so that the data can be E2EE
Citation needed.
> defend their favorite trillion dollar corporations
> is a short slippery slope away
Obviously people who trust Apple aren't concerned about slippery slopes. What's the point of your post?
> is a short slippery slope away
Obviously people who trust Apple aren't concerned about slippery slopes. What's the point of your post?
He's just commenting on how ludicrous it is for anyone to believe that apple is being honest about this and about spying for governments on its users.
There is no way to scan people’s content while “respecting their privacy.” The goal should be to create a system where you couldn’t do so even of you wanted to (or the state demanded it).
> The goal should be to create a system where you couldn’t do so even of you wanted to (or the state demanded it).
There’s this bizarre notion that using end-to-end encryption can absolve you of responsibility, that the authorities will have to accept an answer of “we literally can’t access it”.
That’s just not the case for centralised things: you’re deliberately facilitating some service, government will find you liable for some things in its operation, and if you don’t comply, they’ll fine or shut you down. E2EE doesn’t absolve you from law; law is all about saying you’re not allowed to do things that are physically possible.
(Decentralised things, now they can be banned but not truly stopped because there’s no central party to shut down.)
There’s this bizarre notion that using end-to-end encryption can absolve you of responsibility, that the authorities will have to accept an answer of “we literally can’t access it”.
That’s just not the case for centralised things: you’re deliberately facilitating some service, government will find you liable for some things in its operation, and if you don’t comply, they’ll fine or shut you down. E2EE doesn’t absolve you from law; law is all about saying you’re not allowed to do things that are physically possible.
(Decentralised things, now they can be banned but not truly stopped because there’s no central party to shut down.)
When end-to-end encryption is done correctly, the answer is "we literally can't access it" as a matter of mathematics, whether the state accepts it or not.
A state that does not accept it might retaliate against the entity giving that answer or forbid future use of end-to-end encryption without backdoors, but the truth of the answer doesn't depend on anyone's acceptance.
A state that does not accept it might retaliate against the entity giving that answer or forbid future use of end-to-end encryption without backdoors, but the truth of the answer doesn't depend on anyone's acceptance.
Isn't that when the state prohibits your service? So then no body cares about your mathematical proof because it's a crime to use it.
This is what has happened in many countries already.
This is what has happened in many countries already.
Sure, but that's a case of "we're prohibited from providing end-to-end encryption and preserving user privacy so we can scan for prohibited content as mandated by authorities", not "we are keeping children safe while still preserving user privacy"
Legal terms such as "murder", "fraud" and "rape" do change as effect of regulatory changes. "Encryption" and "privacy" do not.
There's a limit to how much you can bend semantics in your PR before it breaks and you get backlash.
Legal terms such as "murder", "fraud" and "rape" do change as effect of regulatory changes. "Encryption" and "privacy" do not.
There's a limit to how much you can bend semantics in your PR before it breaks and you get backlash.
But if you're a company like apple, it'd be bad business to wait until large government bans your service/device before you respond to it. Much better to read the tea leaves and get a head of it.
Again, their double-speak and redefining words don't help with the reception.
They're deliberately misrepresenting what's happening, appearing surprised when people misunderstand, and bundling together legitimate criticism with misunderstandings.
I can draw some parallels to how Google went out with FLoC.
Honestly I can't tell where Hanlon's razor should cut here.
They're deliberately misrepresenting what's happening, appearing surprised when people misunderstand, and bundling together legitimate criticism with misunderstandings.
I can draw some parallels to how Google went out with FLoC.
Honestly I can't tell where Hanlon's razor should cut here.
What’s the double speak? It’s quite clear they are trying to maintain e2e encryption while also achieving CSAM scanning. Google/Facebook/Microsoft do this by scanning everything unencrypted. Apple do it while maintaining encryption and are being punished for it. It’s insane.
iCloud is not end-to-end encrypted, and Apple has not announced plans to add that option. Apple has the technical ability to scan images on the server.
iCloud (at least for photos) is (effectively) "end to end encrypted". All photos are encrypted or decrypted only on the device (or after being downloaded to a browser), they never exist as human viewable files on iCloud. No one, except the owner of the account, can decrypt the file.
This is why the checking for illegal file has to happen before they are upload. It cannot be done on the server, the encrypted file can never match the hash. What Apple has done that is real genius is that for each image identified as illegal, a part of the key is encrypted and sent with the file to Apple. After a threshold is met, the account is locked, the user notified of the appeals process, and Apple can decrypt and have the files verified. 1 or 2 false positives changes nothing.
Yes, they have the ability, but they found a way NOT to do that. Instead maintaining user privacy. And the world hates them for it, and would prefer they just unencrypted and scanned everything… doesn’t make sense.
There’s nothing stopping governments from banning E2EE, but in the absence of such bans, no one is under any obligation to build systems that empower them to spy on their users.
I wouldn’t be sure about that, the phone companies already have a legal obligation to allow wiretapping and the government is very happy to put gag orders on this stuff.
Shouldn't the E2EE apple walled-garden app equivalent of wiretapping be pushing an app update to the suspect's phone with a sidechannel added for law enforcement to snoop, with warrant in hand?
CALEA has a carve out for encryption. I’m sure when E2EE is about to be deployed to the masses the law will be updated to force key escrow.
Courts can order them to collect data on users.
The law may say that government isn't allowed to spy on people. So no, the state cannot just come and demand anything it wants.
Problem is that the law is self-contradictory and it is up to the judicative institutions to fix it as soon as possible.
Problem is that the law is self-contradictory and it is up to the judicative institutions to fix it as soon as possible.
> The goal should be to create a system where you couldn’t do so even of you wanted to (or the state demanded it).
You can still do secure backups of your phone without using iCloud, but there isn’t a way for Apple to do end to end encryption of backups transparently like you can with real time communication. The only way end to end encryption of backups works is to require people keep a separate secure key(s) to avoid losing their data, which means a universal implementation has real direct risk for users.
As long as Apple has access to these files the FBI can legally require them to do these searches. From a pure PR perspective they should have communicated what was already going on before releasing this system because people assume something significant changed.
You can still do secure backups of your phone without using iCloud, but there isn’t a way for Apple to do end to end encryption of backups transparently like you can with real time communication. The only way end to end encryption of backups works is to require people keep a separate secure key(s) to avoid losing their data, which means a universal implementation has real direct risk for users.
As long as Apple has access to these files the FBI can legally require them to do these searches. From a pure PR perspective they should have communicated what was already going on before releasing this system because people assume something significant changed.
Mega.io (from the same people as MegaUpload) has e2e file encryption with just usernames and passwords.
There is no reason the password can't be the encryption key, with backup keys stored with a trusted third party (eg: your credit union or bank) without notation as to what these backup keys are tied to.
There is no reason the password can't be the encryption key, with backup keys stored with a trusted third party (eg: your credit union or bank) without notation as to what these backup keys are tied to.
Standard passwords don’t provide enough entropy to provide secure encryption.
Trusting third parties with the password in unencrypted form is either systematic in which case the FBI now just needs collect data from 2 different organizations, or on a case by case basis in which case users will mess it up. Apple etc would have no way to verify users actually did something to back up their keys.
Apple’s current approach is to let users setup their own backups if they want security which allows for privacy just fine without providing a service with fundamental issues.
Trusting third parties with the password in unencrypted form is either systematic in which case the FBI now just needs collect data from 2 different organizations, or on a case by case basis in which case users will mess it up. Apple etc would have no way to verify users actually did something to back up their keys.
Apple’s current approach is to let users setup their own backups if they want security which allows for privacy just fine without providing a service with fundamental issues.
In this case, the state does demand it. Running a major cloud photo storage and sharing platform without checking for this material isn't an option in the US.
Apple inspects every file on the local device Before its uploaded. It’s just pinky promise only matched with the on device database when an upload is intended.
Apple controls the hardware, software, and cloud service. It was always a pinky promise that they wouldn't look at your files. I don't know why we should doubt that pinky promise less today than we did a month ago.
They don't control the database used, any country can thru legal means attach additional hashes for search and reporting.
Apple has already proven it will concede to China's demands.
They are building the worlds most pervasive surveillance system and when the worlds governments come knocking to use it ... they will throw their hands up and feed you the "Apple complies with all local laws etc.."
Apple has already proven it will concede to China's demands.
They are building the worlds most pervasive surveillance system and when the worlds governments come knocking to use it ... they will throw their hands up and feed you the "Apple complies with all local laws etc.."
Do any of the other companies control the database of hashes they use? A government could have already done what you’re suggesting but I can’t find a source where this has been the case.
They can upload any software to iPhones that they want. They may not create the database of hashes, but they can choose whether their software uses that database.
> They don't control the database used
They control what goes into the on-device database that is used.
>The on-device encrypted child abuse database contains only data independently submitted by two or more child safety organizations, located in separate jurisdictions, and thus not under the control of the same government
https://www.techwarrant.com/apple-will-only-scan-abuse-image...
They control what goes into the on-device database that is used.
>The on-device encrypted child abuse database contains only data independently submitted by two or more child safety organizations, located in separate jurisdictions, and thus not under the control of the same government
https://www.techwarrant.com/apple-will-only-scan-abuse-image...
For now. Wait until the FBI comes knocking with a warrant.
Then simply disable iCloud Photos sync.
It's bizarre to me that people are freaking out about governments adding client side hashes but no concern that they could be doing server side checks.
It's bizarre to me that people are freaking out about governments adding client side hashes but no concern that they could be doing server side checks.
Have you tried disabling various iCloud features? Disabling iCloud is incredibly buggy.
I tried disabling iCloud keychain and it just flips back on. Sometimes it asks for a login first. Sometimes it shows a cancel/continue modal. Either way, it magically flips back on. No error message.
I tried backing up my device to my hard drive (with Photos already on iCloud) and it kept complaining that there wasn’t enough space. It throws error message after warning message that your content will be deleted. It created additional copies of my photos each time my phone synced.
To properly back up, I had to copy the photos directory to an external hard drive, delete the original, mark the external hard drive one as the system one and then finally free up enough space to back up. iCloud and the device backup weren’t smart enough to free up space for my backup. In fact, I first backed up all my photos to iCloud first because they said that it would free up space on my hard drive as necessary. LOL.
BTW, iCloud keychain is still on for me. Fuck Apple.
I tried disabling iCloud keychain and it just flips back on. Sometimes it asks for a login first. Sometimes it shows a cancel/continue modal. Either way, it magically flips back on. No error message.
I tried backing up my device to my hard drive (with Photos already on iCloud) and it kept complaining that there wasn’t enough space. It throws error message after warning message that your content will be deleted. It created additional copies of my photos each time my phone synced.
To properly back up, I had to copy the photos directory to an external hard drive, delete the original, mark the external hard drive one as the system one and then finally free up enough space to back up. iCloud and the device backup weren’t smart enough to free up space for my backup. In fact, I first backed up all my photos to iCloud first because they said that it would free up space on my hard drive as necessary. LOL.
BTW, iCloud keychain is still on for me. Fuck Apple.
I just tried disabling iCloud for Game Center. It flips itself back on. No error message. Fuck Apple.
Should they even be doing that though? It seems like a matter of time before it's possible to SWAT somebody by sending them a series of hash colliding image files given how not cryptographically secure the hash algorithm is.
I think I'm not the only one who'd rather not have my devices call the cops on me in a country where the cops are already way too violent.
I think I'm not the only one who'd rather not have my devices call the cops on me in a country where the cops are already way too violent.
But this doesn't change any of that. It only changes whether the scanning happens on your device as part of the upload process instead of on the server after the upload.
Yeah that’s right and I don’t think either method is ethical.
Ethics aside, on-device scanning has the benefit of Constitutional protection, at least in the USA. Any attempt by the Government to compel Apple to expand the on-device searching of privately owned devices to find other things would be a clear-cut 4th Amendment violation.
Then don't ask other peole to hold your non encrypted files for you.
You could already just send them CSAM. That is a lot easier than finding a hash collision that also appears to be illegal content when downscaled and viewed by a human.
The human review phase is supposed to explicitly prevent that. Im not sure I would but my faith there, especially if its a flood of colisions and they are rated/paid on case clearance rate.
Further, this is step 1 of a process they have explicitly said they are looking to expand on [1], even going as far to state it in bold font with a standout color.
So theres no telling that they wont expand it by simply scanning everything, regardless of icloud usage, or pivot it to other combat "domestic terrorism" or "gun violence epidemics" or whatever else they feel like.
Its an erosion of trust, even if not a full stop erosion, its something they intend to expand upon and wont be taking back.
[1] https://www.apple.com/child-safety/pdf/Expanded_Protections_...
Further, this is step 1 of a process they have explicitly said they are looking to expand on [1], even going as far to state it in bold font with a standout color.
So theres no telling that they wont expand it by simply scanning everything, regardless of icloud usage, or pivot it to other combat "domestic terrorism" or "gun violence epidemics" or whatever else they feel like.
Its an erosion of trust, even if not a full stop erosion, its something they intend to expand upon and wont be taking back.
[1] https://www.apple.com/child-safety/pdf/Expanded_Protections_...
Because now Apple confirmed themselves that this promise is not kept.
What is "this promise"? Because I would consider it "we will only scan files that you upload to iCloud". That was true a month ago and that would be true under this new system. The only part that is changing is that the scanning happens on your device before upload rather than on an Apple server after upload. I don't view that as a material difference when Apple already controls the hardware and software on both ends. If we can't trust Apple to follow their promise, their products should already have been considered compromised before this change was announced.
Why is the scanning done on the device in the first place? Photos uploaded to Icloud are not encrypted (because the US government was apparently opposed to that idea), so why not do what Google and Facebook does and do the scanning once the image reaches their servers. What is the benefit in running the scan on the device?
> The only part that is changing is that the scanning happens on your device before upload
This is the key point.
1. What if I change my mind and decide not to upload the picture?
2. This is a new mechanism for scanning private pictures on the device. What could go wrong?
> If we can't trust Apple to follow their promise, their products should already have been considered compromised before this change was announced.
Many people did trust Apple to keep their files private until now.
This is the key point.
1. What if I change my mind and decide not to upload the picture?
2. This is a new mechanism for scanning private pictures on the device. What could go wrong?
> If we can't trust Apple to follow their promise, their products should already have been considered compromised before this change was announced.
Many people did trust Apple to keep their files private until now.
> This is a new mechanism for scanning private pictures on the device.
No it isn’t. It’s a mechanism for scanning pictures as they are uploaded to iCloud Photo Library.
Private pictures on the device are not scanned.
No it isn’t. It’s a mechanism for scanning pictures as they are uploaded to iCloud Photo Library.
Private pictures on the device are not scanned.
Pictures not uploaded yet are private.
If your device generates a safety token and it's never uploaded anywhere, that's a no-op.
How are those two examples different than before? You can't unupload a photo under either the old or new system. I don't know why we would expect that the scanning feature will be more prone to accidentally scan too many photos compared to the uploading feature accidentally uploading too many photos.
>Many people did trust Apple to keep their files private until now.
And that was my original point. If a pinky promise from Apple is not enough to trust them, then Apple should have never been trusted.
>Many people did trust Apple to keep their files private until now.
And that was my original point. If a pinky promise from Apple is not enough to trust them, then Apple should have never been trusted.
> You can't unupload a photo under either the old or new system.
You can choose to upload many pictures. They will start uploading. Then, you change your mind. Some pictures were not uploaded yet. But they were scanned by the new algorithm.
You can choose to upload many pictures. They will start uploading. Then, you change your mind. Some pictures were not uploaded yet. But they were scanned by the new algorithm.
I do wonder what happens in that case to the scan results for the photos that weren't yet uploaded. From earlier articles it sounded like the "voucher" is attached to the image upon upload, so it stands to reason that if you cancel an upload, results don't get uploaded for photos that didn't get uploaded. Who knows, though...
Why do people expect anything different? Every corporate promise is subject to change. When you hand your belongings to someone else, those things are liable to be tampered with.
That is the problem. CSAM is just smoke and mirrors paired with appeal to emotions to win approval more easily.
I don't want anyone, neither Apple, nor Microsoft, Google and others to sneak into my files. Did anyone realize that in the 21st century our cellphone is essentially our wallet?
Microsoft and Google already scan all files uploaded to them regardless of your preferences.
No they don't, they scan them on the servers. If you have a source that says otherwise I'd love to see. I'm 99.99% sure you won't find any.
If they are "uploaded to them", they are on their servers. They just do it there instead of before they are uploaded.
Exactly, and 100% Google and Microsoft have on device scanning as well. We are saying "Apple are bad for admitting it" what about the others that are not?
You have 0 proof of that other than conjecture. They are not scanning stuff and uploading it to government organizations to come and arrest you.
The point is that until devices and all their software/firmware become fully auditable, there's no way to be 100% safe, and we must resort to trust.
That wouldn't be a problem in an ideal world, but the one in which we live is far from even resembling one. Mining data is already a huge business, and governments everywhere would love tools to use to get advantage over people they don't like. There's huge motivation and demand for those tools at all levels, and at least governments have the resources to buy them and the power to force whoever implements them to stay silent. I'm not implying that spyware tools exist in any phone, PC, smart TV, car, etc. because we can't prove they don't; that's the argument used for UFOs, witches and unicorns, no thanks, but we better think like they do because technology, resources and demand for their adoption are real, and the rest is probability.
That wouldn't be a problem in an ideal world, but the one in which we live is far from even resembling one. Mining data is already a huge business, and governments everywhere would love tools to use to get advantage over people they don't like. There's huge motivation and demand for those tools at all levels, and at least governments have the resources to buy them and the power to force whoever implements them to stay silent. I'm not implying that spyware tools exist in any phone, PC, smart TV, car, etc. because we can't prove they don't; that's the argument used for UFOs, witches and unicorns, no thanks, but we better think like they do because technology, resources and demand for their adoption are real, and the rest is probability.
A literal feature of Google Photos is to find photos on your device.
Apple are not uploading anything to government organisations, in fact they are uploading less than google, by their own(and googles admission).
You have 0 proof that they are uploading to gov orgs… right?
Apple are not uploading anything to government organisations, in fact they are uploading less than google, by their own(and googles admission).
You have 0 proof that they are uploading to gov orgs… right?
[deleted]
But who was complaining about google and microsoft doing the cloud scanning?
I don’t mind my one drive being scanned for “bad stuff”, I very much mind my personally owned data stores being scanned, with no opt out.
I don’t mind my one drive being scanned for “bad stuff”, I very much mind my personally owned data stores being scanned, with no opt out.
The only thing Apple scans are files you upload to iCloud Photos.
If you turn off iCloud Photos, nothing is scanned.
Microsoft scans everything.
>The system that scans cloud drives for illegal images was created by Microsoft and Dartmouth College and donated to NCMEC. The organization creates signatures of the worst known images of child pornography, approximately 16,000 files at present. These file signatures are given to service providers who then try to match them to user files in order to prevent further distribution of the images themselves, a Microsoft spokesperson told NBC News. (Microsoft implemented image-matching technology in its own services, such as Bing and SkyDrive.)
https://www.nbcnews.com/technolog/your-cloud-drive-really-pr...
If you turn off iCloud Photos, nothing is scanned.
Microsoft scans everything.
>The system that scans cloud drives for illegal images was created by Microsoft and Dartmouth College and donated to NCMEC. The organization creates signatures of the worst known images of child pornography, approximately 16,000 files at present. These file signatures are given to service providers who then try to match them to user files in order to prevent further distribution of the images themselves, a Microsoft spokesperson told NBC News. (Microsoft implemented image-matching technology in its own services, such as Bing and SkyDrive.)
https://www.nbcnews.com/technolog/your-cloud-drive-really-pr...
> If you turn off iCloud Photos, nothing is scanned.
According to Apple. and For Now. Patriot Act was only for terrorists. Apple makes concessions for China. Creating this technology, makes it very easy for China to go, "Look at all photos, always". If they only want to scan stuff on iCloud Photos, no worries, just implement on it on their end. This tech does not need to exist in that case.
> Microsoft scans everything.
Everything uploaded to the Cloud. The Cloud bit is fairly important here. That's me, willingly, putting information on their property. Perfectly acceptable and fine for them to ensure that it's nothing unethical.
However, someone else snooping through your drawers looking for something to pin on you is not private, nor the same as checking their own property.
According to Apple. and For Now. Patriot Act was only for terrorists. Apple makes concessions for China. Creating this technology, makes it very easy for China to go, "Look at all photos, always". If they only want to scan stuff on iCloud Photos, no worries, just implement on it on their end. This tech does not need to exist in that case.
> Microsoft scans everything.
Everything uploaded to the Cloud. The Cloud bit is fairly important here. That's me, willingly, putting information on their property. Perfectly acceptable and fine for them to ensure that it's nothing unethical.
However, someone else snooping through your drawers looking for something to pin on you is not private, nor the same as checking their own property.
“And for now” is not a particularly strong argument because it applies to literally anything that software could ever conceivably do on an iPhone, because Apple has the ability to release any software updates they want.
If Microsoft's scan is a server-side scan, how do they scan "everything" if "everything" includes things that you do not upload?
By your definition of "everything", both Microsoft and Apple scan "everything". (Or Apple will, after this new system is rolled out.)
By your definition of "everything", both Microsoft and Apple scan "everything". (Or Apple will, after this new system is rolled out.)
I have no idea what I expected but 16,000 photos feels… disgustingly high and ridiculously low at the same time.
It's well known that there are also videos. So these criminals won't be caught watching them.
In my view, Apple is not going to take the risk of changing their mind and not implementing this new scanning feature in order to test the hypothesis that you can satisfy both the people who want to stop CSAM and the people who don't want their files scanned. And also, Apple or any other company announcing they will not scan or give up any user data means that criminals, foolish or not, will take notice and start uploading CSAM to their servers. If they get away with it, they've found their new home. That's why regulation against this kind of data exists. Possessing CSAM is still a felony.
Nobody is arguing about the legality of CSAM itself. It's an issue that is not popular to discuss, and of course being on the wrong side of it results in near-universal, justifiable derision. Stopping its spread is absolutely the right thing to do.
So at the point that companies will always be held liable for storing it, they will have to put up countermeasures of some kind or find themselves sued out of existence. Server-side scanning is one method, and Apple's on-device scanning is another.
There are certainly ways that Apple can go too far with whatever it happens to come up with as its solution to stopping CSAM, but there is still seems to have been a line behind which nobody particularly cares how the detection is implemented and life continues as usual. If Apple had chosen not to cross that line, maybe many of the arguments being made here would never have been brought up at all.
Nobody is arguing about the legality of CSAM itself. It's an issue that is not popular to discuss, and of course being on the wrong side of it results in near-universal, justifiable derision. Stopping its spread is absolutely the right thing to do.
So at the point that companies will always be held liable for storing it, they will have to put up countermeasures of some kind or find themselves sued out of existence. Server-side scanning is one method, and Apple's on-device scanning is another.
There are certainly ways that Apple can go too far with whatever it happens to come up with as its solution to stopping CSAM, but there is still seems to have been a line behind which nobody particularly cares how the detection is implemented and life continues as usual. If Apple had chosen not to cross that line, maybe many of the arguments being made here would never have been brought up at all.
That's the point: catching the bad guy foolish enough to keep known CSAM images on their phone, while not technically invading the privacy of any good guys. Anyway, if apple wanted to covertly invade their users' privacy, they'd have no technical problems to do so.
What it takes to accept is the "nothing to hide" mentality: your files are safe to scan (locally) because they can't be known CSAM files. You have to trust the scanner. You allow the scanner to touch your sensitive files because you're not the bad guy, and you want the bad guy be caught (or at least forced off the platform).
And this is, to my mind, the part Apple wasn't very successful at communicating. The whole thing should have started with an educational campaign well ahead of time. The privacy advantage should have been explained again and again: "unlike every other vendor, we won't siphon your files unecrypted for checking; we do everything locally and are unable to compromise your sensitive bits". Getting one's files scanned should have become a badge of honor among the users.
But, for some reason, they tried to do it in a low-key and somehow hasty manner.
What it takes to accept is the "nothing to hide" mentality: your files are safe to scan (locally) because they can't be known CSAM files. You have to trust the scanner. You allow the scanner to touch your sensitive files because you're not the bad guy, and you want the bad guy be caught (or at least forced off the platform).
And this is, to my mind, the part Apple wasn't very successful at communicating. The whole thing should have started with an educational campaign well ahead of time. The privacy advantage should have been explained again and again: "unlike every other vendor, we won't siphon your files unecrypted for checking; we do everything locally and are unable to compromise your sensitive bits". Getting one's files scanned should have become a badge of honor among the users.
But, for some reason, they tried to do it in a low-key and somehow hasty manner.
The "nothing to hide" mentality is exactly what's wrong about all this.
Yes. You don't leave the toilet doors open, not because you don't "have nothing to hide", but because privacy is a right.
Everybody knows what goes on behind that door. And yet, everyone would much prefer to close it. Apple's method is equivalent to the toilet door being removed, so that you cannot do anything nefarious behind that door.
Your phone also (on average) have nothing to hide, but that's also why you need privacy on your devices.
Everybody knows what goes on behind that door. And yet, everyone would much prefer to close it. Apple's method is equivalent to the toilet door being removed, so that you cannot do anything nefarious behind that door.
Your phone also (on average) have nothing to hide, but that's also why you need privacy on your devices.
I mostly agree, though I will argue for the other side; child predators don't exactly have a track record for intelligence. Pedophiles still to this day get caught trading CP on Facebook of all places. I think engaging in that kind of activity requires a certain number of brain cells misfiring. Watch some old episodes of To Catch A Predator. Most of those guys were borderline retarded or had obvious personality disorders.
This is not true and a harmful generalization. There are dumb people of every type. There are probably smart people you respect that engage in CSAM and are minor-attracted. This kind of privacy invasion will only net a tiny section of that population, as most know that lack of caution is life or death.
> This is not true and a harmful generalization. There are dumb people of every type.
That's purely specious, and you say it as if I ever asserted there aren't "dumb people of every type.", which I didn't say.
Yes, there are dumb people of every "type". Are there as many intelligent people as those of low intelligence who struggle to think abstractly, struggle to read and write, have poor motor control, rely heavily on dogma for a moral compass, or have trouble with impulse control?
If you think there are as many people with a high IQ as a low IQ with those traits, I don't think you're being honest.
Here's some data for you:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4478390/
https://pubmed.ncbi.nlm.nih.gov/14744183/
https://pubmed.ncbi.nlm.nih.gov/961457/
https://www.dw.com/en/scientists-find-brain-differences-in-p...
https://sci-hubtw.hkvisa.net/10.1023/a:1018754004962
https://www.livescience.com/4671-study-pedophiles-tend-short...
https://pubmed.ncbi.nlm.nih.gov/29625377/
Are most pedophiles of low intelligence overall? It's doubtful, but there's room for more study.
Are the pedophiles that are offenders of low intelligence? Significantly more offending pedophiles are of low intelligence than the general population. Yes, the factors at play may also apply to other forms of crime. But when you factor out non-offending pedophiles, those we aren't as concerned with in regards to CSAM, the ones that are caught aren't exactly the cream of the crop.
Furthermore, I've met a handful of pedophiles, both convicted and not. Every one of them either was clearly mentally retarded or bordering on retardedness, or they had a severe personality disorder. Yes, it's a small sample size, but the ones that were caught did so doing the dumbest of shit (putting aside the horrific nature of child exploitation in the first place). This even includes a female pedophile, who was both stupid and psychopathic, and I find it funny that one of those studies questions the very existence of female pedophiles.
There are many pedophiles that are of above-average intelligence. Most may not even be considered "dumb". But to imply that there's no difference in distribution of, frankly, dumb people to smart people between different behaviors is absurd in that it ignores any sort of relation implication that intelligence has on said behaviors.
As a group, the data does not suggest that pedophiles are exceptionally intelligent, but rather the opposite.
So yes, while I believe there is a privacy concern (I said earlier I would not want such software or hardware running on my own compuer), I stand by my conclusion that CSAM detection would catch a significant number of pedophiles and probably continue to do so for some time. The point of such systems isn't to be a superweapon to catch every pedophile. The ones that it might catch happen to be the ones who are most likely to offend. If you play the stupid game of uploading CSAM to Apple iCloud, you win the stupid prize of getting convicted, and it just so happens that a ton of pedophiles are likely to do something that boneheaded.
That's purely specious, and you say it as if I ever asserted there aren't "dumb people of every type.", which I didn't say.
Yes, there are dumb people of every "type". Are there as many intelligent people as those of low intelligence who struggle to think abstractly, struggle to read and write, have poor motor control, rely heavily on dogma for a moral compass, or have trouble with impulse control?
If you think there are as many people with a high IQ as a low IQ with those traits, I don't think you're being honest.
Here's some data for you:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4478390/
https://pubmed.ncbi.nlm.nih.gov/14744183/
https://pubmed.ncbi.nlm.nih.gov/961457/
https://www.dw.com/en/scientists-find-brain-differences-in-p...
https://sci-hubtw.hkvisa.net/10.1023/a:1018754004962
https://www.livescience.com/4671-study-pedophiles-tend-short...
https://pubmed.ncbi.nlm.nih.gov/29625377/
Are most pedophiles of low intelligence overall? It's doubtful, but there's room for more study.
Are the pedophiles that are offenders of low intelligence? Significantly more offending pedophiles are of low intelligence than the general population. Yes, the factors at play may also apply to other forms of crime. But when you factor out non-offending pedophiles, those we aren't as concerned with in regards to CSAM, the ones that are caught aren't exactly the cream of the crop.
Furthermore, I've met a handful of pedophiles, both convicted and not. Every one of them either was clearly mentally retarded or bordering on retardedness, or they had a severe personality disorder. Yes, it's a small sample size, but the ones that were caught did so doing the dumbest of shit (putting aside the horrific nature of child exploitation in the first place). This even includes a female pedophile, who was both stupid and psychopathic, and I find it funny that one of those studies questions the very existence of female pedophiles.
There are many pedophiles that are of above-average intelligence. Most may not even be considered "dumb". But to imply that there's no difference in distribution of, frankly, dumb people to smart people between different behaviors is absurd in that it ignores any sort of relation implication that intelligence has on said behaviors.
As a group, the data does not suggest that pedophiles are exceptionally intelligent, but rather the opposite.
So yes, while I believe there is a privacy concern (I said earlier I would not want such software or hardware running on my own compuer), I stand by my conclusion that CSAM detection would catch a significant number of pedophiles and probably continue to do so for some time. The point of such systems isn't to be a superweapon to catch every pedophile. The ones that it might catch happen to be the ones who are most likely to offend. If you play the stupid game of uploading CSAM to Apple iCloud, you win the stupid prize of getting convicted, and it just so happens that a ton of pedophiles are likely to do something that boneheaded.
I think your response is emotionally-charged and mixes feelings with data and anecdotes, which is unhelpful in a complex discussion of a sensitive topic, so I hesitate to engage further, but…
I’ve reviewed much of the existing data on minor-attracted persons (which is scant). The problem is that the samples are heavily biased towards including individuals who are caught or self-admit for care, which is not a very good representative sample. As a few of the studies and articles you posted mention, there was little study done until very recently. You also make the leap without supporting data that this system would catch those most likely to offend. Generally I would say the data is insufficient to conclude in any direction other than dumber people get caught doing crimes at a higher rate than smarter people.
Lastly, I have some serious doubts about your expertise in mental health and the study thereof from your use of “mentally retarded” and “retardedness”. You should know for future discussions that the clinical terminology has shifted to “intellectually disability”.
I’ve reviewed much of the existing data on minor-attracted persons (which is scant). The problem is that the samples are heavily biased towards including individuals who are caught or self-admit for care, which is not a very good representative sample. As a few of the studies and articles you posted mention, there was little study done until very recently. You also make the leap without supporting data that this system would catch those most likely to offend. Generally I would say the data is insufficient to conclude in any direction other than dumber people get caught doing crimes at a higher rate than smarter people.
Lastly, I have some serious doubts about your expertise in mental health and the study thereof from your use of “mentally retarded” and “retardedness”. You should know for future discussions that the clinical terminology has shifted to “intellectually disability”.
That sounds like survivorship bias. Of course the people who get caught for doing things the stupid way are going to be branded as stupid. How about the people who don't get caught, or the people who get caught despite doing a decent job of covering their tracks? What evidence do you have that most pedophiles are of the stupid sort?
> child predators don't exactly have a track record for intelligence
Maybe that's true of the ones that get caught, but if so there's a good chance that at least part of that is inverse survivorship bias. (Similar to “Why are most of the CIA covert missions you hear about failures?”)
Maybe that's true of the ones that get caught, but if so there's a good chance that at least part of that is inverse survivorship bias. (Similar to “Why are most of the CIA covert missions you hear about failures?”)
You don't have to be intelligent to know that CSAM is super illegal and you probably don't want them to be co-mingled with pictures of your mum and last night's dinner.
Except intelligence is clustered with other pathologies such as poor impulse control. Knowledge of the law is not the end-all-be-all of human behavior. Both the intelligent and unintelligent are capable of poor impulse control, but the unintelligent are statistically much more likely to exhibit a lack of impulse control, among other things such as difficulty in long-term reasoning and connecting one's actions with adverse effects.
I've never lacked the impulse control to import regular porn into my photo library. And regular porn only comes with the risk of embarrassment, not prison.
Where's the line though? Would you be upset that the Dropbox client scanned things before uploadng it? What about the Google Drive client JS?
> catch the bad people doing very bad things to children
You may catch a few perverts looking at the stuff, but I'm not convinced this will lead to catching the producers. How would that happen?
You may catch a few perverts looking at the stuff, but I'm not convinced this will lead to catching the producers. How would that happen?
The theory is that if you make it too risky to possess CSAM, demand for it will drop, leading to less exploitation. I don't know if I buy that theory, but... there it is.
Compare this to the war on drugs. There is huge demand for drugs. Many drugs are highly illegal to possess, and a lot of people get jailed for possessing them. And yet most of us consider the war on drugs to be an abject failure that hasn't done much more than create inequity.
Why should something like CSAM possession be any different? I wish it was different, and these sorts of tactics would reduce this awful problem, but I'm just not convinced it does.
Compare this to the war on drugs. There is huge demand for drugs. Many drugs are highly illegal to possess, and a lot of people get jailed for possessing them. And yet most of us consider the war on drugs to be an abject failure that hasn't done much more than create inequity.
Why should something like CSAM possession be any different? I wish it was different, and these sorts of tactics would reduce this awful problem, but I'm just not convinced it does.
Known CSAM detection was one of three features they announced under the banner of child safety. Certainly they understand it is by itself an incomplete intervention for a much larger problem.
> Apple genuinely believe this is a novel and unique method to catch CSAM without invading people's privacy
What’s novel about this? The technique and issues with it are fairly obvious to anyone with experience in computer vision. It seems not too different from research from 1993 https://proceedings.neurips.cc/paper/1993/file/288cc0ff02287...
The issues are also well known and encompass the whole subfield of adversarial examples.
What’s novel about this? The technique and issues with it are fairly obvious to anyone with experience in computer vision. It seems not too different from research from 1993 https://proceedings.neurips.cc/paper/1993/file/288cc0ff02287...
The issues are also well known and encompass the whole subfield of adversarial examples.
Presumably the system is novel enough that a very similar proposal which lacked one important insight (using perceptual hashes to turn image similarity problems into set intersection problems) was accepted to USENIX this year: https://www.usenix.org/conference/usenixsecurity21/presentat...
The privacy aspect does seem novel, and that paper evaluates the various aspects of privacy and the quality of perceptual hashes. However, anything concerning privacy (of client or server data) is orthogonal to the vision methods underpinning the system, which are well understood and are the primary cause of the post. For that matter, I don’t see how the paper’s proofs of privacy and/or formalization of the problem help with hash collisions. And that particular paper doesn’t use deep learning methods which are probably easier to attack with off-the-shelf methods.
Maybe Apple was focused on keeping CSAM off a cloud resource that really is Apple's in both a practical and technical sense. Merely scanning after upload maybe doesn't accomplish that because the material already has arrived in an Apple resource and someone might define that as a failure. Viewed from a perspective of iCloud compliance it sorta makes sense. No one ought to feel a lot of (legal) security deploying a system that could be used to host CSAM, from the Blockchain to whatever, it sounds like a don't-wizz-on-the-electric-fence kind of risk.
Saddened by the privacy-adverse functionality on handsets but Apple seems to be hitting the nail on the head that poor communication principally is driving outrage.
Saddened by the privacy-adverse functionality on handsets but Apple seems to be hitting the nail on the head that poor communication principally is driving outrage.
With the new system, it still arrives on the server, and they can't even know about it until some threshold has been passed - and then the servers still have to scan it.
So it's not even accomplishing "keeping it off the servers."
So it's not even accomplishing "keeping it off the servers."
Actually it'll do an excellent job of keeping it off their servers.
Apple's actions will be effective at making sure every CSAM aficionado will not trust their device. Or, if they're tech savvy, they will at least know not to co-mingle their deepest darkest secrets alongside photos of their mum and last night's dinner.
If you think Apple's very big, very public blow-up is making waves in the hacker/security/libertarian crowd, just imagine how big it's blowing up in the CSAM community right now. I dare say it's probably all they've been talking about for the past two weeks. Unlike places like Hacker News where there's plenty of people desperate to drag Apple through the coals, the CSAM community will be highly motivated to have a precise understanding of what Apple is doing. I dare say that they'll be extremely well informed about exactly what Apple's plans entail and how to work around them.
Apple's actions will be effective at making sure every CSAM aficionado will not trust their device. Or, if they're tech savvy, they will at least know not to co-mingle their deepest darkest secrets alongside photos of their mum and last night's dinner.
If you think Apple's very big, very public blow-up is making waves in the hacker/security/libertarian crowd, just imagine how big it's blowing up in the CSAM community right now. I dare say it's probably all they've been talking about for the past two weeks. Unlike places like Hacker News where there's plenty of people desperate to drag Apple through the coals, the CSAM community will be highly motivated to have a precise understanding of what Apple is doing. I dare say that they'll be extremely well informed about exactly what Apple's plans entail and how to work around them.
> Apple found a way to preserve that privacy
This is Orwellian doublespeak.
You don't preserver someone's privacy by snooping on their devices.
This is Orwellian doublespeak.
You don't preserver someone's privacy by snooping on their devices.
> other major cloud providers catch CSAM content on their platform by inspecting every file uploaded
That is very unlikely. Most likely they compare some hash against a database - just like apple.
That is very unlikely. Most likely they compare some hash against a database - just like apple.
That's what they're saying, the key difference is Apples happens on your device, not theirs.
That's not really better or different. In any meaningful way.
It is indeed meaningfully different: your sensitive data never leaves your device in order to be scanned. There can't be a crack in Apple servers that would expose files of millions of users uploaded for scanning.
Eventually it could lead to Apple platform not having any CSAM content, or any known legally objectionable content, because any sane perpetrator would migrate to other platforms, and less sane, caught.
This, of course, takes the user base to overwhelmingly agree that keeping legally objectionable content is a bad thing and should be exposed to police, and to trust the whatever body which defines the hashes of objectionable content. I'm afraid this order is not as tall as we could imagine.
Eventually it could lead to Apple platform not having any CSAM content, or any known legally objectionable content, because any sane perpetrator would migrate to other platforms, and less sane, caught.
This, of course, takes the user base to overwhelmingly agree that keeping legally objectionable content is a bad thing and should be exposed to police, and to trust the whatever body which defines the hashes of objectionable content. I'm afraid this order is not as tall as we could imagine.
> your sensitive data never leaves your device in order to be scanned. There can't be a crack in Apple servers that would expose files of millions of users uploaded for scanning.
And yet photos that get scanned are still uploaded to iCloud Photos, so they do end up on Apple's servers.
And yet photos that get scanned are still uploaded to iCloud Photos, so they do end up on Apple's servers.
You can disable iCloud backup.
Doing so right while activating a new iDevice is the way to prevent its private keys from ending up in iCloud, and so preventing Apple, or law enforcement, or some malicious hackers from cracking into your device.
Doing so right while activating a new iDevice is the way to prevent its private keys from ending up in iCloud, and so preventing Apple, or law enforcement, or some malicious hackers from cracking into your device.
Photos that users choose to upload to iCloud Photos do indeed get uploaded to iCloud Photos?
I'm failing to see the issue.
I'm failing to see the issue.
I think you might be tilting at windmills here. The issue is that the post I'm replying to claims that data never ends up on Apple's servers even though it does. Thanks for confirming that it does, though.
Am I supposed to be shocked that photos the user uploads to iCloud are on iCloud?
The results of the scan looking for kiddie porn cannot be read by Apple until the device finds 30 examples of photos that match known kiddie porn, whereupon Apple gets the decryption key so they can see which images on their server need review, and a human review is triggered to make sure there haven't been 30 false positives.
The results of the scan looking for kiddie porn cannot be read by Apple until the device finds 30 examples of photos that match known kiddie porn, whereupon Apple gets the decryption key so they can see which images on their server need review, and a human review is triggered to make sure there haven't been 30 false positives.
> Am I supposed to be shocked that photos the user uploads to iCloud are on iCloud?
I implore you to read the comment I originally replied in order to understand the context of my replies. Personally, I don't care what you're shocked about or not, as my OP wasn't directed at you at all.
I implore you to read the comment I originally replied in order to understand the context of my replies. Personally, I don't care what you're shocked about or not, as my OP wasn't directed at you at all.
Then why the outrage if they're the same?
Because this is Apple and people are very passionate about both hating and liking Apple.
Apple keeps the scan results encrypted with a key they don't have until the device informs them that the threshold of 30 images that match known kiddie porn has been reached.
After that, they get the decryption key and trigger a human review to make sure there haven't been 30 false positives at once.
That is better, and more private, in a very meaningful way.
False positive scan data sitting on the server is open to malicious misuse by anyone who can get a warrant.
>Innocent man, 23, sues Arizona police for $1.5million after being arrested for murder and jailed for six days when Google's GPS tracker wrongly placed him at the scene of the 2018 crime
https://www.dailymail.co.uk/news/article-7897319/Police-arre...
Also, I sincerely doubt that Google has ended it's practice of refusing to hire a human being to check for false positives when a poorly performing algorithm is cheaper.
After that, they get the decryption key and trigger a human review to make sure there haven't been 30 false positives at once.
That is better, and more private, in a very meaningful way.
False positive scan data sitting on the server is open to malicious misuse by anyone who can get a warrant.
>Innocent man, 23, sues Arizona police for $1.5million after being arrested for murder and jailed for six days when Google's GPS tracker wrongly placed him at the scene of the 2018 crime
https://www.dailymail.co.uk/news/article-7897319/Police-arre...
Also, I sincerely doubt that Google has ended it's practice of refusing to hire a human being to check for false positives when a poorly performing algorithm is cheaper.
I do agree that it's more private, but I'm not sure it's better.
I'm fine with the idea that if I upload stuff to someone else's server, they may take a look at it and maybe even punish me for what I've uploaded. Certainly if I encrypt the data before I upload it, they can't do that. But if I don't, then it's fine with me if they do.
But my device should not be snitching on me. Yes, this device-side scanning is supposedly gated on enabling uploads to iCloud, but that doesn't really make for much of a distinction to me. And since Apple certainly has the capability, they are likely one secret court order away from being required to scan even photos that aren't being uploaded, at least on some targeted subset of devices.
I'm fine with the idea that if I upload stuff to someone else's server, they may take a look at it and maybe even punish me for what I've uploaded. Certainly if I encrypt the data before I upload it, they can't do that. But if I don't, then it's fine with me if they do.
But my device should not be snitching on me. Yes, this device-side scanning is supposedly gated on enabling uploads to iCloud, but that doesn't really make for much of a distinction to me. And since Apple certainly has the capability, they are likely one secret court order away from being required to scan even photos that aren't being uploaded, at least on some targeted subset of devices.
> And since Apple certainly has the capability, they are likely one secret court order away from being required to scan even photos that aren't being uploaded, at least on some targeted subset of devices.
Apple has the ability to upload literally any software to iPhones, so this argument applies equally to literally any conceivable bad thing that software could do on iPhones.
Apple has the ability to upload literally any software to iPhones, so this argument applies equally to literally any conceivable bad thing that software could do on iPhones.
As well as any conceivable bad thing that Google could be ordered to add to Android.
Or Google with Chrome (way more reach than Android), or Microsoft with Windows, etc. This is why I reject slippery slope arguments for software updates to auto-updating platforms. They already have the ability to ship literally any conceivable software to millions of people, so the "slippery slope" already exists trivially for the evilest software you can imagine.
Making it obviously and unquestionably more invasive.
I just don't get this. Say you're given two options when going through the TSA.
1. The TSA agent opens your luggage and searches everything for banned items.
2. The TSA agent hands you a scanner for you to wave over your luggage in private, it prints out a receipt of banned items it saw, and you present that receipt to the agent.
Which one is more invasive?
1. The TSA agent opens your luggage and searches everything for banned items.
2. The TSA agent hands you a scanner for you to wave over your luggage in private, it prints out a receipt of banned items it saw, and you present that receipt to the agent.
Which one is more invasive?
The scenario is far more like this
3. The home builder installs a TSA scanner in all newly built homes. The scanners will scan all items as they are put away into drawers and cupboards, attempting to detect the presence of banned items.
Then the TSA Agents say they won't report you until at least 30 banned items are detected, even though you haven't flown in 10 years and don't have banned materials. As the TSA Agents walk back to their car, you overhear words like warrant, trouble, and tap amidst their chuckles. What could go wrong with that?
3. The home builder installs a TSA scanner in all newly built homes. The scanners will scan all items as they are put away into drawers and cupboards, attempting to detect the presence of banned items.
Then the TSA Agents say they won't report you until at least 30 banned items are detected, even though you haven't flown in 10 years and don't have banned materials. As the TSA Agents walk back to their car, you overhear words like warrant, trouble, and tap amidst their chuckles. What could go wrong with that?
Apple’s solution more like the TSA agent shows up at your house and inspects your luggage before you even leave for the airport and he pinky promises not to report on anything else he might see while in your home.
This version is incorrect because Apple’s solution by design does not reveal the non-suspect contents of your luggage.
1. Cops on the street
2. Cops in your house
Which one is more invasive?
I don't think there is much to not understand. Apple is proposing putting a cop in your house. On device scanning is vastly vastly different than server side scanning. One is over apples property, the other is over YOUR property. Its a big deal, and a huge change in policy. And as myself and others have indicated from years of observation, it will grow and grow...
Which one is more invasive?
I don't think there is much to not understand. Apple is proposing putting a cop in your house. On device scanning is vastly vastly different than server side scanning. One is over apples property, the other is over YOUR property. Its a big deal, and a huge change in policy. And as myself and others have indicated from years of observation, it will grow and grow...
3. TSA installs scanners in all homes, premises to only use them to scan your luggages as you head to the airport and nothing else.
The TSA installs this scanner in your house so it can detect before you even get to the airport. They give a pinky promise they will only run it when you book your ticket and intend on going to the airport where you would be expected to be scanned anyway.
The scanner technology can also be used for detecting dissident journalism, but they say it won't be, it is just for preventing terrorism, but still they say, they really need to install it in your house even though it is only for scanning when you intend on going to the airport.
The scanner technology can also be used for detecting dissident journalism, but they say it won't be, it is just for preventing terrorism, but still they say, they really need to install it in your house even though it is only for scanning when you intend on going to the airport.
> The scanner technology can also be used for detecting dissident journalism, but they say it won't be, it is just for preventing terrorism
This has always been possible with server side scanning. Yeah we have to trust Apple to not enable it for users who don’t use iCloud photos but we’ve always had to trust Apple given their closed source system.
This has always been possible with server side scanning. Yeah we have to trust Apple to not enable it for users who don’t use iCloud photos but we’ve always had to trust Apple given their closed source system.
[deleted]
> other major cloud providers catch CSAM content on their platform by inspecting every file uploaded
So does Apple.
EDIT: some people don’t like that answer, but “inspecting” in this context clearly means “digitally inspecting” (Google does not physically look at every file) and Apple does this with files that are uploaded. They do it in device, but it’s still inspected. That’s the whole point of this controversy, that there’s not much difference to people WHERE apple inspects and on device is actually arguably worse. Your sentence does not in any way distinguish what Apple does from what others do.
So does Apple.
EDIT: some people don’t like that answer, but “inspecting” in this context clearly means “digitally inspecting” (Google does not physically look at every file) and Apple does this with files that are uploaded. They do it in device, but it’s still inspected. That’s the whole point of this controversy, that there’s not much difference to people WHERE apple inspects and on device is actually arguably worse. Your sentence does not in any way distinguish what Apple does from what others do.
> Apple found a way to preserve that privacy but still catch the bad people doing very bad things to children.
They're not going to find predators, all they are going to find is people incompetent enough to have years old (otherwise how would it end up at NCMEC?) CSAM on their iPhones and dumb enough to have iCloud backup turned on. To catch actual predators they would need to run AI scanning on phones for all photos and risk a lot of false positives by parents taking photos of their children on a beach.
Plus a load of people who will inadvertently have stuff that "looks" like CSAM on their devices because some 4chan trolls will inevitably create colliding material and spread it in ads etc. so it ends up downloaded in browser caches.
All of this "let's use technology to counter pedos" is utter, utter crap that is not going to save one single child and will only serve to tear down our freedoms, one law at a time. Want to fight against pedos? Make photos of your airbnb, hotel and motel rooms to help identify abuse recording locations and timeframes, and teach children from an early age about their bodies, sexuality and consent so that they actually have the words to tell you that they are being molested.
They're not going to find predators, all they are going to find is people incompetent enough to have years old (otherwise how would it end up at NCMEC?) CSAM on their iPhones and dumb enough to have iCloud backup turned on. To catch actual predators they would need to run AI scanning on phones for all photos and risk a lot of false positives by parents taking photos of their children on a beach.
Plus a load of people who will inadvertently have stuff that "looks" like CSAM on their devices because some 4chan trolls will inevitably create colliding material and spread it in ads etc. so it ends up downloaded in browser caches.
All of this "let's use technology to counter pedos" is utter, utter crap that is not going to save one single child and will only serve to tear down our freedoms, one law at a time. Want to fight against pedos? Make photos of your airbnb, hotel and motel rooms to help identify abuse recording locations and timeframes, and teach children from an early age about their bodies, sexuality and consent so that they actually have the words to tell you that they are being molested.
> people incompetent enough to have years old (otherwise how would it end up at NCMEC?) CSAM on their iPhones
Who do you think has 30 or more CSAM images on their phone?
Who do you think has 30 or more CSAM images on their phone?
Let's be real, of course many of those people will be child abusers.
But let's also be real about something else. Think of the Venn diagram of child abusers and people who share CSAM online. Those circles are not identical. Not everybody with CSAM is necessarily a child abuser. Worse, how many child abusers don't share CSAM online? Those can't ever be found by Apple-style invasions of privacy, so one has to wonder if we're being asked to give up significant privacy for a crime fighting strategy that may not even be all that effective.
The cynic in me is also wondering what fraction of people working at places like NCMEC are pedophiles. It'd be a rather convenient job for them. And after all, there's a long history of ostensibly child-serving institutions harboring the worst kind of offenders.
But let's also be real about something else. Think of the Venn diagram of child abusers and people who share CSAM online. Those circles are not identical. Not everybody with CSAM is necessarily a child abuser. Worse, how many child abusers don't share CSAM online? Those can't ever be found by Apple-style invasions of privacy, so one has to wonder if we're being asked to give up significant privacy for a crime fighting strategy that may not even be all that effective.
The cynic in me is also wondering what fraction of people working at places like NCMEC are pedophiles. It'd be a rather convenient job for them. And after all, there's a long history of ostensibly child-serving institutions harboring the worst kind of offenders.
All CSAM is produced by child abuse. Everyone who shares 30 images online is a participant in child abuse, even if only by creating a demand for images.
> a crime fighting strategy
Is it a crime fighting strategy? I thought it was just about not making their devices and services into aids for these crimes.
> The cynic in me is also wondering what fraction of people working at places like NCMEC are pedophiles.
Good question. What has that to do with this?
> a crime fighting strategy
Is it a crime fighting strategy? I thought it was just about not making their devices and services into aids for these crimes.
> The cynic in me is also wondering what fraction of people working at places like NCMEC are pedophiles.
Good question. What has that to do with this?
That's a tedious debate that has been rehashed to death. Finding somebody who has CSAM without being a child abuser doesn't save even a single child. Finding everybody who has CSAM without being a child abuser would likely save some children (drying out commercial operations, perhaps?), but is also basically impossible.
And how many children can really be helped in this way? Surely the majority of child abuse happens independently of the online sharing of CSAM.
And is that really worth the suspension of important civil rights? That is really what this discussion is about. Nobody is arguing against fighting child abuse, the question is about how, and about what means can be justified.
And how many children can really be helped in this way? Surely the majority of child abuse happens independently of the online sharing of CSAM.
And is that really worth the suspension of important civil rights? That is really what this discussion is about. Nobody is arguing against fighting child abuse, the question is about how, and about what means can be justified.
> Surely the majority of child abuse happens independently of the online sharing of CSAM.
Does it? I see no reason to assume that.
> And is that really worth the suspension of important civil rights?
No civil rights are being suspended. It’s not about that at all.
Does it? I see no reason to assume that.
> And is that really worth the suspension of important civil rights?
No civil rights are being suspended. It’s not about that at all.
Pedophiles. But these people are consumers of extremely old content, which means the CSAM can won't even make a tiny dent against active, current abusers!
CSAM scanning is sold as beneficial, while in reality it won't do shit while it opens dangerous precedence backdoors!
CSAM scanning is sold as beneficial, while in reality it won't do shit while it opens dangerous precedence backdoors!
> extremely old content
Why do you assume it’s extremely old?
Why do you assume it’s extremely old?
The thing that's shocking to me is that Google, Microsoft and all the big names in tech have been scanning everything in your account (email, cloud drive, photos, etc) for the past decade, without any noticeable uproar.
Apple announces that it is going to start scanning iCloud Photos only, and that their system is set to ignore anything below a threshold of ~30 positives before triggering a human review, and people lose their minds.
Apple announces that it is going to start scanning iCloud Photos only, and that their system is set to ignore anything below a threshold of ~30 positives before triggering a human review, and people lose their minds.
I think the difference here is that it's ON YOUR DEVICE. I think there's a pretty clear understanding that if you upload stuff to a cloud provider they can do whatever they want with it. This is different. This is reaching into what has up until now mostly been considered a private place. Law enforcement often has to get warrants to search this kind of thing.
This is the difference between putting CSAM on a sign in your front yard (maybe not quite front yard but I can't come up with quite the same physical equivalent to a cloud provider) and keeping it in a password protected vault in your basement. One of those things is protected in the U.S. by laws against unlawful search and seizure. Cloud and on your device are two very different things and consumers are right to be alarmed.
I'll say it again, if you are concerned with this privacy violation, sell your Apple stock and categorically refuse to purchase Apple devices. Also go to https://www.nospyphone.com/ and make your voice heard there.
This is the difference between putting CSAM on a sign in your front yard (maybe not quite front yard but I can't come up with quite the same physical equivalent to a cloud provider) and keeping it in a password protected vault in your basement. One of those things is protected in the U.S. by laws against unlawful search and seizure. Cloud and on your device are two very different things and consumers are right to be alarmed.
I'll say it again, if you are concerned with this privacy violation, sell your Apple stock and categorically refuse to purchase Apple devices. Also go to https://www.nospyphone.com/ and make your voice heard there.
Only the safety tokens are generated on your device, the action triggering scan happens in the cloud (just like all the others) and then it goes to human review, so if it's a hash collision it'd be caught there when they review the images, and they can only review the images that matched CSAM.
I honestly can't find the uproar here. Google devices can face match photos offline... so they are applying a neural net (scanning) ON THE DEVICE! How is that not worse than what apple do?
I honestly can't find the uproar here. Google devices can face match photos offline... so they are applying a neural net (scanning) ON THE DEVICE! How is that not worse than what apple do?
One is matching a face--yes, concerning, and I don't like it but IMO Google has had a much worse privacy reputation for quite some time--and the other is reporting private data to law enforcement and potentially abusive parents. It's quite a bit different.
The difference can also be seen from a customer service perspective. One is a feature that lets you sort according to which friends you were with. The other is a feature that puts you in jail. No thanks. Not gonna pay money for that.
The difference can also be seen from a customer service perspective. One is a feature that lets you sort according to which friends you were with. The other is a feature that puts you in jail. No thanks. Not gonna pay money for that.
No it reports you to Apple, Apple report you to police. Exactly like google, they will also report you to the police, just they search your library unencrypted.
Literally no difference.
If you have illegal stuff only on your phone neither google or Apple will be notified or notify anyone else.
Literally no difference.
If you have illegal stuff only on your phone neither google or Apple will be notified or notify anyone else.
ON YOUR DEVICE (where it's encrypted in a way that Apple can't read until the 30 image threshold is crossed) is more private than doing the same scan on server where a single false positive can be misused by anyone who can get a subpoena.
A poster upthread made an analogy that I really like. Sure, like all analogies, it's imperfect, but I think it strikes at why many people are uneasy about this.
Let's say the TSA were to install air-travel-contraband scanners in everyone's homes, but promise only to scan things that are being put into your luggage as you prepare to go to the airport. And let's say that this became a requirement if you want to board a plane.
That's what this feels like. I'm fine with Google scanning through everything in my GMail account, or everything I've uploaded to GDrive, or created in GDocs. That stuff is on their servers, unencrypted, and I explicitly put it there.
But I'm sure as hell not going to let Google install something on my laptop (or phone!) that lets them look at my stuff, even if they pinky-promise that they'll only scan stuff that I intend to upload.
Let's say the TSA were to install air-travel-contraband scanners in everyone's homes, but promise only to scan things that are being put into your luggage as you prepare to go to the airport. And let's say that this became a requirement if you want to board a plane.
That's what this feels like. I'm fine with Google scanning through everything in my GMail account, or everything I've uploaded to GDrive, or created in GDocs. That stuff is on their servers, unencrypted, and I explicitly put it there.
But I'm sure as hell not going to let Google install something on my laptop (or phone!) that lets them look at my stuff, even if they pinky-promise that they'll only scan stuff that I intend to upload.
> ON YOUR DEVICE (where it's encrypted in a way that Apple can't read until the 30 image threshold is crossed)
First of all, you have to be able to read it to do the comparison that can increment the counter to 30. So regardless of whether it is or is not encrypted there, they're accessing the unencrypted plaintext to calculate the hash.
And yes, on my device is definitively more private than on someone else's server--just like in my bedside drawer is more private than in an office I rent in a co-working space.
First of all, you have to be able to read it to do the comparison that can increment the counter to 30. So regardless of whether it is or is not encrypted there, they're accessing the unencrypted plaintext to calculate the hash.
And yes, on my device is definitively more private than on someone else's server--just like in my bedside drawer is more private than in an office I rent in a co-working space.
> where it's encrypted in a way that Apple can't read
This doesn't matter because Apple can read iCloud data, including iCloud Photos. They hold the encryption keys, and they hand over customers' data for about 150,000 users/accounts a year in response to requests from the government[1].
[1] https://www.apple.com/legal/transparency/us.html
This doesn't matter because Apple can read iCloud data, including iCloud Photos. They hold the encryption keys, and they hand over customers' data for about 150,000 users/accounts a year in response to requests from the government[1].
[1] https://www.apple.com/legal/transparency/us.html
Of course Apple can read iCloud data.
How do you think Google and Microsoft scan everything in your account? They all have the capability to read your cloud data.
What Apple cannot read are the results of your device scanning your iCloud Photos. Those results are encrypted and stay that way until your device finds 30 matches for known kiddie porn.
Once you pass the threshhold, Apple gets the decryption key and a human review is triggered to make sure there weren't just 30 false positives.
How do you think Google and Microsoft scan everything in your account? They all have the capability to read your cloud data.
What Apple cannot read are the results of your device scanning your iCloud Photos. Those results are encrypted and stay that way until your device finds 30 matches for known kiddie porn.
Once you pass the threshhold, Apple gets the decryption key and a human review is triggered to make sure there weren't just 30 false positives.
what kind of encryption can't be decrypted until some threshold has been crossed?
maybe you mean to say that apple says they won't read it until that threshold has been crossed.
maybe you mean to say that apple says they won't read it until that threshold has been crossed.
> what kind of encryption can't be decrypted until some threshold has been crossed?
The kind Apple has built. You should read the docs. This is literally how it works.
The kind Apple has built. You should read the docs. This is literally how it works.
[deleted]
This is the same company that saved the disk encryption password as the password hint. You really trust them to not screw this up when the stakes are that it could ruin your life and/or land you in jail? I'm simply not ok with that.
How exactly do you imagine a bug in this will land you in jail?
Ever heard of planted evidence?
You haven’t explained how evidence could be planted. That is what you are being asked.
No, I'm saying they designed the system so that they don't have the key to decypt the scan result "vouchers" until after the device tells them that the 30 image threshold is crossed.
>Apple is unable to process individual vouchers; instead, all the properties of our system mean that it’s only once an account has accumulated a collection of vouchers associated with illegal, known CSAM images that we are able to learn anything about the user’s account.
Now, why to do it is because, as you said, this is something that will provide that detection capability while preserving user privacy.
https://techcrunch.com/2021/08/10/interview-apples-head-of-p...
Meanwhile, a single false positive from an on server scan is open to malicious use by anyone who can get a subpeona.
>Apple is unable to process individual vouchers; instead, all the properties of our system mean that it’s only once an account has accumulated a collection of vouchers associated with illegal, known CSAM images that we are able to learn anything about the user’s account.
Now, why to do it is because, as you said, this is something that will provide that detection capability while preserving user privacy.
https://techcrunch.com/2021/08/10/interview-apples-head-of-p...
Meanwhile, a single false positive from an on server scan is open to malicious use by anyone who can get a subpeona.
The lady doth protest too much, methinks
Just going to respond to every post on here with these absurd points? K apple guy.
Just going to respond to every post on here with these absurd points? K apple guy.
Here's the reason for the sudden uproar: The scanning of things on third party servers was just barely tolerated by a lot of people, whether it's for advertising purposes or government intrusion. People accept it because it's considered reasonable for these third parties to scan data in exchange for providing a service for free (e.g. Google), or because they need to have some degree of accountability for what is on their servers.
When that scanning gets moved from the cloud to being on your device, a boundary is violated.
When that boundary is violated by a company who makes extreme privacy claims like saying that privacy is a "fundamental human right"[1], yes, people will "lose their minds" over it. This shouldn't be shocking at all.
[1] https://apple.com/privacy
When that scanning gets moved from the cloud to being on your device, a boundary is violated.
When that boundary is violated by a company who makes extreme privacy claims like saying that privacy is a "fundamental human right"[1], yes, people will "lose their minds" over it. This shouldn't be shocking at all.
[1] https://apple.com/privacy
When the scanning gets moved from the cloud to being on device, Apple itself cannot see the results of the scan until the risk that the result is only a false positive is greatly reduced.
You would have to have 30 false positives before Apple can see anything, which is unlikely, but the next step is still a human review, since it's not impossible.
You would have to have 30 false positives before Apple can see anything, which is unlikely, but the next step is still a human review, since it's not impossible.
I don't care. It's my device (at least that's how Apple used to advertise it) and I disagree with Apple's policy, full stop. I don't care about "think of the children" (especially since they will be scanning pictures of my own children), and furthermore I don't trust Apple not to change the policy in the future. They've created a backdoor and now the device is compromised and privacy is broken. If you want to trust Apple go ahead. They have eroded my trust and I doubt they could regain it within a decade.
>I don't care
If anything, you should be outraged that Google and Microsoft have been scanning much more of your data, and doing so in a much more intrusive way.
Apple only scans iCloud Photos and they do so in a way that they can't see the results until they can be reasonably sure it's not just a false positive.
If anything, you should be outraged that Google and Microsoft have been scanning much more of your data, and doing so in a much more intrusive way.
Apple only scans iCloud Photos and they do so in a way that they can't see the results until they can be reasonably sure it's not just a false positive.
You have clearly misunderstood my position as you have replied to 2 of my comments already. You seem to have an "Apple vs the world" mentality when my point is entirely about holding Apple accountable to their own marketing. You sound like a bizarre Apple apologist and I'm tired of trying to explain myself to you when all you want to do is explain to me why Apple is better than everybody when I do not care and also fundamentally disagree.
If you think Apple's approach is the best you're allowed to think that. I disagree.
If you think Apple's approach is the best you're allowed to think that. I disagree.
[deleted]
Whether or not Apple can see the results is completely irrelevant, to me at least. Automated surveillance is still surveillance.
The other companies didn't advertise and pride themselves on being privacy focused. Part of the appeal of Apple was that you could avoid that issue and they touted it regularly. Now they're telling their customers to go fuck themselves (so long as they're 18 or older).
Conducting the scan on the user's device instead of on the companies server is more private.
Apple can't decrypt the results of the scan until the ~30 image threshold is crossed and a human review is triggered.
Given Google's reluctance to hire humans when a poorly performing algorithm is cheaper, are they turning over every single false positive without a human review?
Apple can't decrypt the results of the scan until the ~30 image threshold is crossed and a human review is triggered.
Given Google's reluctance to hire humans when a poorly performing algorithm is cheaper, are they turning over every single false positive without a human review?
I guess people had a sense of ownership of these devices and they feel that it’s doing some they they don’t want. If ownership means control, maybe in the digital age full ownership/control is not really possible on a managed device like the iPhone. There are other example like the John Deer tractor issues.
I have a sense of ownership over my non-publicly-accessible data when it's backed up to a cloud drive.
Apple isn't scanning that.
Google and Microsoft are.
Apple isn't scanning that.
Google and Microsoft are.
Sure, you own the data, but you don’t have any control on what’s happening to it. Just like iPhone users, we own the physical device but we only control certain aspects of what it’s doing, and the boundaries are not set by us, but by Apple.
If I back up my files to Apple's cloud server, I don't have to worry about them scanning my private files at all.
They don't cross that line like Google and Microsoft do.
With Apple, nothing but files you upload to iCloud Photos get scanned.
They don't cross that line like Google and Microsoft do.
With Apple, nothing but files you upload to iCloud Photos get scanned.
[deleted]
Google is open about the fact they scan and track everything, they even make it a point of convenience "hey, we looked at your email and found a plane ticket, based on where you are, you should leave in 2 hours if you want to be at the airport on time".
Facebook, even more so, they are explicitly anti-privacy to the point of being insulting.
Microsoft will happily show you everything they may send when you install Windows, you can sometimes refuse, but not always. They are a bit less explicit than Google, but privacy is rarely on the menu.
As for Amazon, their cloud offers are mostly for businesses, different market, but still, for consumers, they don't really insist on privacy either.
So that if any of these company scan your pictures for child porn, it won't shock anyone, because we know it is what they do.
But Apple claims privacy as a core value, half of their ads are along the lines of "we are not like the others, we respect your privacy, everything on your device stays on your device, etc...", they announce every (often legitimate) privacy feature with great fanfare, etc... So much that people start to believe it. But with that, people realize that Apple is not so different from the others after all, and if they bought an overpriced device based on that promise, I understand why they are pissed off.
Facebook, even more so, they are explicitly anti-privacy to the point of being insulting.
Microsoft will happily show you everything they may send when you install Windows, you can sometimes refuse, but not always. They are a bit less explicit than Google, but privacy is rarely on the menu.
As for Amazon, their cloud offers are mostly for businesses, different market, but still, for consumers, they don't really insist on privacy either.
So that if any of these company scan your pictures for child porn, it won't shock anyone, because we know it is what they do.
But Apple claims privacy as a core value, half of their ads are along the lines of "we are not like the others, we respect your privacy, everything on your device stays on your device, etc...", they announce every (often legitimate) privacy feature with great fanfare, etc... So much that people start to believe it. But with that, people realize that Apple is not so different from the others after all, and if they bought an overpriced device based on that promise, I understand why they are pissed off.
This seems like a common deflection, but get back to me when either company puts programs in my pocket that scan my data for crimes and snitch on me to authorities.
>a man [was] arrested on child pornography charges, after Google tipped off authorities about illegal images found in the Houston suspect's Gmail account
https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-le...
You don't consider the contents of your email account or the files you mirror to a cloud drive to be your own private data?
https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-le...
You don't consider the contents of your email account or the files you mirror to a cloud drive to be your own private data?
No OP, but yes. I expect that my emails can be read and that my files on a cloud service can be accessed. And I don’t even use Gmail.
I expect that my ISP tracks and stores my DNS resolutions (if I use their DNS) and has a good understanding of the websites I visit.
I expect that an app that I grant access to my contacts uploads as much data as it can to their servers.
I expect WhatsApp and similar apps to collect and upload meta data of my entire photo library such as GPS info the second I give them access.
Hence, I don’t give access. And hence, it’s a problem if there is no opt-out of local file scanning in the future.
I expect that my ISP tracks and stores my DNS resolutions (if I use their DNS) and has a good understanding of the websites I visit.
I expect that an app that I grant access to my contacts uploads as much data as it can to their servers.
I expect WhatsApp and similar apps to collect and upload meta data of my entire photo library such as GPS info the second I give them access.
Hence, I don’t give access. And hence, it’s a problem if there is no opt-out of local file scanning in the future.
There are worlds between. One case is about pictures on your device. The other case is about pictures on Googles "devices" or network. You had to upload it to Google. EMail is also nothing like a letter anyway. It's a postcard. Everybody can read it.
No, one case is about photos you upload to iCloud Photos only, and the other case is about every single thing in your Google account.
>So if iCloud Photos is disabled, the system does not work, which is the public language in the FAQ. I just wanted to ask specifically, when you disable iCloud Photos, does this system continue to create hashes of your photos on device, or is it completely inactive at that point?
If users are not using iCloud Photos, NeuralHash will not run
https://techcrunch.com/2021/08/10/interview-apples-head-of-p...
>So if iCloud Photos is disabled, the system does not work, which is the public language in the FAQ. I just wanted to ask specifically, when you disable iCloud Photos, does this system continue to create hashes of your photos on device, or is it completely inactive at that point?
If users are not using iCloud Photos, NeuralHash will not run
https://techcrunch.com/2021/08/10/interview-apples-head-of-p...
No, that happens on Google's servers and is not an agent in my pocket that scans and invades my personal property for evidence of crimes and snitches on me to authorities.
At least Google does.
https://protectingchildren.google/intl/en/
> CSAI Match is our proprietary technology, developed by the YouTube team, for combating child sexual abuse imagery (CSAI) in video content online. It was the first technology to use hash-matching to identify known violative videos and allows us to identify this type of violative content amid a high volume of non-violative video content. When a match of violative content is found, it is then flagged to partners to responsibly report in accordance to local laws and regulations. Through YouTube, we make CSAI Match available for free to NGOs and industry partners like Adobe, Reddit, and Tumblr, who use it to counter the spread of online child exploitation videos on their platforms as well.
> We devote significant resources—technology, people, and time—to detecting, deterring, removing, and reporting child sexual exploitation content and behavior. Since 2008, we’ve used “hashing” technology, which creates a unique digital ID for each known child sexual abuse image, to identify copies of images on our services that may exist elsewhere.
https://protectingchildren.google/intl/en/
> CSAI Match is our proprietary technology, developed by the YouTube team, for combating child sexual abuse imagery (CSAI) in video content online. It was the first technology to use hash-matching to identify known violative videos and allows us to identify this type of violative content amid a high volume of non-violative video content. When a match of violative content is found, it is then flagged to partners to responsibly report in accordance to local laws and regulations. Through YouTube, we make CSAI Match available for free to NGOs and industry partners like Adobe, Reddit, and Tumblr, who use it to counter the spread of online child exploitation videos on their platforms as well.
> We devote significant resources—technology, people, and time—to detecting, deterring, removing, and reporting child sexual exploitation content and behavior. Since 2008, we’ve used “hashing” technology, which creates a unique digital ID for each known child sexual abuse image, to identify copies of images on our services that may exist elsewhere.
Note that they're not pushing it to devices that consumers have purchased, just running it on their cloud services. I think it's an important distinction here with what Apple is doing.
No, that happens on Google's servers and isn't an agent in my pocket that runs on and invades my personal property. It's also only for videos.
The images being scanned are the ones that go to iCloud.
The Google page has a section later down that also says they use hashing of images.
The Google page has a section later down that also says they use hashing of images.
When the scan is carried out on device, Apple can't read the results of the scan until the 30 image threshold is reached.
When Google scans on server, a single false positive result can be abused by anyone who can get a warrant.
>Innocent man, 23, sues Arizona police for $1.5million after being arrested for murder and jailed for six days when Google's GPS tracker wrongly placed him at the scene of the 2018 crime
https://www.dailymail.co.uk/news/article-7897319/Police-arre...
Apple's method is more private.
When Google scans on server, a single false positive result can be abused by anyone who can get a warrant.
>Innocent man, 23, sues Arizona police for $1.5million after being arrested for murder and jailed for six days when Google's GPS tracker wrongly placed him at the scene of the 2018 crime
https://www.dailymail.co.uk/news/article-7897319/Police-arre...
Apple's method is more private.
[deleted]
BigCos, take note: you’re better off doing nefarious shit without telling anyone. Because, if you come clean, you’ll only invite an endless parade of bloggers who will misconstrue your technology to make you look bad.
No misconstrual needed. This technology is genuinely bad. It scans images against an arbitrary government-owned black-box database. There’s no guarantee that it’s only CSAM.
I have a serious problem imagining that Apple will not be willing to redeploy this technology in a novel and nefarious means in exchange for continued market access when billions are on the line.
Mainland China will probably be the first chip to fall. Can't imagine the Ministry of State Security not actively licking their lips, waiting for this functionality to arrive.
Mainland China will probably be the first chip to fall. Can't imagine the Ministry of State Security not actively licking their lips, waiting for this functionality to arrive.
NECMEC isn’t owned by the government and the database of hashes is available from Apple.
> NECMEC isn’t owned by the government
Even though NCMEC describes itself as "private", it was established by and has been heavily funded by the U.S. government.
From an archive of NCMEC's own history page, cited on Wikipedia (https://web.archive.org/web/20121029010231/http://www.missin...):
> In 1984, the U.S. Congress passed the Missing Children’s Assistance Act which established a National Resource Center and Clearinghouse on Missing and Exploited Children. The National Center for Missing & Exploited Children was designated to fulfill this role.
> On June 13, 1984, the National Center for Missing & Exploited Children was opened by President Ronald Reagan in a White House Ceremony. The national 24-hour toll-free missing children’s hotline 1-800-THE-LOST opened as well.
$40 million/year of U.S. government funding from a 2013 bill (https://en.wikipedia.org/wiki/Missing_Children%27s_Assistanc...):
> The Missing Children's Assistance Reauthorization Act of 2013 (H.R. 3092) is a bill that was introduced into the United States House of Representatives during the 113th United States Congress. The Missing Children's Assistance Reauthorization Act of 2013 reauthorizes the Missing Children's Assistance Act and authorizes $40 million a year to fund the National Center for Missing and Exploited Children.
Even though NCMEC describes itself as "private", it was established by and has been heavily funded by the U.S. government.
From an archive of NCMEC's own history page, cited on Wikipedia (https://web.archive.org/web/20121029010231/http://www.missin...):
> In 1984, the U.S. Congress passed the Missing Children’s Assistance Act which established a National Resource Center and Clearinghouse on Missing and Exploited Children. The National Center for Missing & Exploited Children was designated to fulfill this role.
> On June 13, 1984, the National Center for Missing & Exploited Children was opened by President Ronald Reagan in a White House Ceremony. The national 24-hour toll-free missing children’s hotline 1-800-THE-LOST opened as well.
$40 million/year of U.S. government funding from a 2013 bill (https://en.wikipedia.org/wiki/Missing_Children%27s_Assistanc...):
> The Missing Children's Assistance Reauthorization Act of 2013 (H.R. 3092) is a bill that was introduced into the United States House of Representatives during the 113th United States Congress. The Missing Children's Assistance Reauthorization Act of 2013 reauthorizes the Missing Children's Assistance Act and authorizes $40 million a year to fund the National Center for Missing and Exploited Children.
They're not owned by the federal government, but they do get a lot of federal government money.
> The National Center for Missing & Exploited Children® was established in 1984 as a private, nonprofit 501(c)(3) organization. Today, NCMEC performs the following 15 specific programs of work, funded in part by federal grants (34 U.S.C. § 11293): Source: https://www.missingkids.org/footer/about
US DOJ OJJDP lists recent grants totaling $84,446,366 in FY19 and FY20. Source: https://ojjdp.ojp.gov/funding/awards/list?awardee=NATIONAL%2...
> The National Center for Missing & Exploited Children® was established in 1984 as a private, nonprofit 501(c)(3) organization. Today, NCMEC performs the following 15 specific programs of work, funded in part by federal grants (34 U.S.C. § 11293): Source: https://www.missingkids.org/footer/about
US DOJ OJJDP lists recent grants totaling $84,446,366 in FY19 and FY20. Source: https://ojjdp.ojp.gov/funding/awards/list?awardee=NATIONAL%2...
And don’t forget, it’s way more than just money:
https://www.law.cornell.edu/uscode/text/18/2258A
You must report to them and only them.
For the GP to claim they’re not government “owned” is a rhetorical trick at best and outright ignorant absurdity at worst.
https://www.law.cornell.edu/uscode/text/18/2258A
You must report to them and only them.
For the GP to claim they’re not government “owned” is a rhetorical trick at best and outright ignorant absurdity at worst.
That’s like saying the federal reserve is “private”. No, the NECMEC is not a private entity. Not only was it heavily funded/created by the gov, but more importantly it is granted special legal status. You and I can’t just spin up our own CSAM database. Nor do we have any laws that say that any companies aware of CSAM must send it to us and only us.
It's important to keep nefarious stuff on the server side because eventually someone will reverse engineer what's on the client side.
Imagine if Apple had done this on the client side without telling anyone, and later it was discovered. I think things would be a whole worse for Apple in that case.
Imagine if Apple had done this on the client side without telling anyone, and later it was discovered. I think things would be a whole worse for Apple in that case.
Devices with proprietary OSes spend more and more time phoning home, then exchanging data officially for "updates". They probably tell the truth, but should one of them decide to hide users data exfiltration or other monitoring practices behind those updates, it would be quite hard to catch them. In other words we have no way to tell that they're not already doing this.
That being the case, why do it client side at all, when presumably every claim they make is verifiable?
Most people don't pay attention to whether or not their stuff is being scanned. Those of us who do pay attention have known for a long time that it's being scanned. Especially by Google and Facebook. Basically anyone whose business model is based off of advertising. My default assumption is anything I upload is scanned.
Because it isn’t about the CSAM scanning. Apple keeps reframing the argument back to that. most people expect that google and Microsoft to scan files on their servers for CSAM, people object to Apple turning your phone against its owner, and once building out the capability to do this only policy prevents it from being abused.
> all the big names in tech have been scanning everything in your account [...] without any noticeable uproar
In part because people didn't know.
And if you were one of innocent people caught by them, you wouldn't want people to know.
In part because people didn't know.
And if you were one of innocent people caught by them, you wouldn't want people to know.
Repeating myself from another thread but… China.
Conjecture, admittedly:
1. Apple cannot lose the Chinese market. Huge and more important fastest growing geo for the company.
2. China is self deprecating elements of its own tech sector (aggressive crackdowns on both established companies like tencent and Alibaba as well as individual web sites). They are clearly cleaning house from a surveillance and control perspective. Apple is not immune, but it’s an American behemoth, so open door crackdowns are impossible.
I don’t think Apple’s CSAM push and China’s crackdown are purely coincidental.
Who can argue with stemming child abuse? It’s the type of hot button issue that affords broad acceptance for intrusive tech.
The leap from scanning for abuse to scanning for anti regime content is more like a tiny step.
It seems obvious from afar that the company adamant about refusal to unlock a potential terrorist’s iPhone on privacy principles (with the attendant marketing benefits) would so suddenly force push (and therefore ensure collection massive training data with or without opt-in for Chinas v2.0) such a boldly invasive feature addition.
Turns out vertical integration is both gift and curse (dependent on the whims of the integrator) for on-device privacy and autonomy.
Conjecture, admittedly:
1. Apple cannot lose the Chinese market. Huge and more important fastest growing geo for the company.
2. China is self deprecating elements of its own tech sector (aggressive crackdowns on both established companies like tencent and Alibaba as well as individual web sites). They are clearly cleaning house from a surveillance and control perspective. Apple is not immune, but it’s an American behemoth, so open door crackdowns are impossible.
I don’t think Apple’s CSAM push and China’s crackdown are purely coincidental.
Who can argue with stemming child abuse? It’s the type of hot button issue that affords broad acceptance for intrusive tech.
The leap from scanning for abuse to scanning for anti regime content is more like a tiny step.
It seems obvious from afar that the company adamant about refusal to unlock a potential terrorist’s iPhone on privacy principles (with the attendant marketing benefits) would so suddenly force push (and therefore ensure collection massive training data with or without opt-in for Chinas v2.0) such a boldly invasive feature addition.
Turns out vertical integration is both gift and curse (dependent on the whims of the integrator) for on-device privacy and autonomy.
If you use Apple devices, you are agreeing to their ToS which since 2019 give them the right to pre-screen your uploads for any potentially illegal (sic) content, not specifically CSAM imagery.
If Apple happens to use similar ToS in China (no idea if true), you bet CCP would be all over this clause.
A worst-case wild guess from the pessimist in me: they had to add this clause in 2019 to appease CCP, and then they got thinking if they could make use of it for the greater good (tm), too. Hopefully it’s very incorrect.
If Apple happens to use similar ToS in China (no idea if true), you bet CCP would be all over this clause.
A worst-case wild guess from the pessimist in me: they had to add this clause in 2019 to appease CCP, and then they got thinking if they could make use of it for the greater good (tm), too. Hopefully it’s very incorrect.
Of course the CCP loves stuff like this. The big thing is will apple acquiesce to keeping mum about scanning everyone's stuff at the directive of the government. Also will that just be a "china" build or will it be in the USA code base but "flipped off"
Right now, it seems like there are two specific groups of people that are upset with Apple: Freedom evangelists (e.g. EFF) and tech futurists (e.g. HN). They're saying, essentially:
"Apple does not have my permission to use my device to scan my iCloud uploads for CSAM"
and
"This is a slippery slope that could result in Apple enforcing thoughtcrimes"
Neither of these viewpoints are particularly agreeable to the general public in the US, as far as I can determine from my non-tech farming city. Once the fuss in tech dies down, I expect Apple will see a net increase in iCloud adoption — all the fuss we're generating is free advertising for their efforts to stop child porn, and the objections raised are too domain-specific to matter.
It's impossible to say for certain which of your outcomes will occur, but there's definitely two missing from your list. Corrected, it reads:
"Either there will be heads rolling at management, or Apple takes a permanent hit to consumer trust, or Apple sees no effect whatsoever on consumer trust, or Apple sees a permanent boost in consumer trust."
I expect it'll be "no effect", but if I had to pick a second guess, it would be "permanent boost", well offsetting any losses among the tech/free/lib crowd.
"Apple does not have my permission to use my device to scan my iCloud uploads for CSAM"
and
"This is a slippery slope that could result in Apple enforcing thoughtcrimes"
Neither of these viewpoints are particularly agreeable to the general public in the US, as far as I can determine from my non-tech farming city. Once the fuss in tech dies down, I expect Apple will see a net increase in iCloud adoption — all the fuss we're generating is free advertising for their efforts to stop child porn, and the objections raised are too domain-specific to matter.
It's impossible to say for certain which of your outcomes will occur, but there's definitely two missing from your list. Corrected, it reads:
"Either there will be heads rolling at management, or Apple takes a permanent hit to consumer trust, or Apple sees no effect whatsoever on consumer trust, or Apple sees a permanent boost in consumer trust."
I expect it'll be "no effect", but if I had to pick a second guess, it would be "permanent boost", well offsetting any losses among the tech/free/lib crowd.
You forgot the most important point of the last 24 hours:
"Apple has created a system for detecting CSAM on local devices which has already proven vulnerable to cheap perceptual hash collision attacks. It's now highly inconceivable Apple will be able to deploy this technology as-is without having their users exploited."
In other words it's not just about privacy or thoughtcrimes anymore but should be viewed as actually dangerous to use their devices. I feel a bit dramatic even typing that out but I.. think it's true?
"Apple has created a system for detecting CSAM on local devices which has already proven vulnerable to cheap perceptual hash collision attacks. It's now highly inconceivable Apple will be able to deploy this technology as-is without having their users exploited."
In other words it's not just about privacy or thoughtcrimes anymore but should be viewed as actually dangerous to use their devices. I feel a bit dramatic even typing that out but I.. think it's true?
I would also warn you against owning any device with a radio. Carriers control the radio towers and can be compelled by government agencies and selfish corporate interests to exploit remote execution vulnerabilities in radio chips in order to plant CSAM content onto devices.
How dramatic is too dramatic? When does something that hasn’t happened to you or anyone you know become a risk you’re willing to sacrifice personal convenience to mitigate? Will you be divesting yourself of all wireless radio hardware? If not, then why would you be worried about users being exploited through a more clumsy and less effective process such as CSAM signature hacking?
The piece of information you’re taking for granted, that few in free/tech/lib are confronting, is the assumption that this process can be exploited at scale to harm millions of people.
So far as I can tell, there will probably be zero or one false positive CSAM matches that pass the known algo, the unknown algo, the human blurred comparison, and the human unblurred comparison — all steps that must occur before law enforcement is invoked to collect digital evidence - in the first year.
How many false positives (to the nearest 10^X) do you think the system will generate in the first year that result in law enforcement actions? Your words suggest that everyone is vulnerable, and there are 10^9 users, so do you believe there will be 10^9 false positives in the first year? Do you think only a thousand people will be affected, so 10^3? How do you judge which is more likely correct?
It is unlikely that this system will generate 10^9 false positives, or else it never would have passed QA. I encourage you to consider how you would personally quantify this risk, and then also look up the quantified risks for killing someone while driving a car or getting struck by lightning while indoors. I don’t know what the actual reality will be, but I don’t think it's a very large X.
How dramatic is too dramatic? When does something that hasn’t happened to you or anyone you know become a risk you’re willing to sacrifice personal convenience to mitigate? Will you be divesting yourself of all wireless radio hardware? If not, then why would you be worried about users being exploited through a more clumsy and less effective process such as CSAM signature hacking?
The piece of information you’re taking for granted, that few in free/tech/lib are confronting, is the assumption that this process can be exploited at scale to harm millions of people.
So far as I can tell, there will probably be zero or one false positive CSAM matches that pass the known algo, the unknown algo, the human blurred comparison, and the human unblurred comparison — all steps that must occur before law enforcement is invoked to collect digital evidence - in the first year.
How many false positives (to the nearest 10^X) do you think the system will generate in the first year that result in law enforcement actions? Your words suggest that everyone is vulnerable, and there are 10^9 users, so do you believe there will be 10^9 false positives in the first year? Do you think only a thousand people will be affected, so 10^3? How do you judge which is more likely correct?
It is unlikely that this system will generate 10^9 false positives, or else it never would have passed QA. I encourage you to consider how you would personally quantify this risk, and then also look up the quantified risks for killing someone while driving a car or getting struck by lightning while indoors. I don’t know what the actual reality will be, but I don’t think it's a very large X.
Thank you for your comments. And I think you are probably correct about the true risk/X entailed here. As well as the risk of simply using anything with a radio - I can't think of a single device I own that hasn't had a radio attack of some sort published (including all Apple devices, of course).
If I'm honest what makes me feel bad about this is probably just that- a feeling based on what I consider to be an algorithm designed around the presumption of guilt. It's much the same way I feel about taking my shoes off in line at airport security. It's an act which I've largely come to ignore but which still produces that vague feeling of discomfort that somehow feels like the opposite of security.
That this is occurring on my Apple devices - my favorite devices - is also just depressing.
If I'm honest what makes me feel bad about this is probably just that- a feeling based on what I consider to be an algorithm designed around the presumption of guilt. It's much the same way I feel about taking my shoes off in line at airport security. It's an act which I've largely come to ignore but which still produces that vague feeling of discomfort that somehow feels like the opposite of security.
That this is occurring on my Apple devices - my favorite devices - is also just depressing.
I'm kinda amazed.
I mentioned I was thinking of moving from an Android phone to Apple soon, somewhat privacy related.
My friends lectured me on "they're scanning your photos" ... meanwhile they share their google photos albums with me and marvel about how easy they are to search ...
Maybe we (humans) only get outraged based on more specific narratives and not so much the general topics / issues?
I don't know but they didn't seem to notice the conflict.
I mentioned I was thinking of moving from an Android phone to Apple soon, somewhat privacy related.
My friends lectured me on "they're scanning your photos" ... meanwhile they share their google photos albums with me and marvel about how easy they are to search ...
Maybe we (humans) only get outraged based on more specific narratives and not so much the general topics / issues?
I don't know but they didn't seem to notice the conflict.
Isn't there a small difference between these? A) They scan everything I have released to google photos B) They scan everything that exists on my device
Psychologically, you'll feel a difference in what you accept between the two, I think
Psychologically, you'll feel a difference in what you accept between the two, I think
As has been repeated over and over, apple only scans photos that are part if icloud photos (ie, uploaded).
Don't want your photo's scanned, don't sync them to icloud. Seriously! Please include the actual system when discussing this system, not your bogeyman system.
"To help address this, new technology in iOS and iPadOS* will allow Apple to detect known CSAM images stored in iCloud Photos."
To increase privacy - they perform the scan on device prior to upload.
Don't want your photo's scanned, don't sync them to icloud. Seriously! Please include the actual system when discussing this system, not your bogeyman system.
"To help address this, new technology in iOS and iPadOS* will allow Apple to detect known CSAM images stored in iCloud Photos."
To increase privacy - they perform the scan on device prior to upload.
As has been repeated over and over, apple only scans photos that are part if icloud photos (ie, uploaded).
"for now"
Which is the part most people have a problem with -- they say that they are only scanning iCloud uploads now, but it's simple extension of the scanner to scan all files.
I don't care if Apple scans my iCloud uploads on iCloud servers, I don't want them scanning photos on my device.
"for now"
Which is the part most people have a problem with -- they say that they are only scanning iCloud uploads now, but it's simple extension of the scanner to scan all files.
I don't care if Apple scans my iCloud uploads on iCloud servers, I don't want them scanning photos on my device.
I will believe them if they put their money where their mouth is: as a clause to iOS user agreement saying that if they ever use they ever use this functionality for anything other than CSAM or on anything other than iCloud photos, ever person who was subjected to this scan will be paid 100 million dollars by Apple. I will believe them if they put this clause in, and I know when they have changed their plans when they remove the clause. No more pinky swears, let's add some stakes for breaking the promise.
Until one day a box pops up. It says "We've updated our terms. See this 10,000 line document here. Please accept to continue using your device." Then your clause is gone.
I'm fine with that. Apple is a big company and changing its TOS will be instantly reported on. It will act as a canary of sorts to know when they turn evil and we know to stop using their products.
Isn't the design of this system enough of a canary?
I'd be satisfied with a money back guarantee -- if they change the policy then I can return the phone for a full refund.
People are pretending it scans everything on your device. This conversation is already bad enough without adding additional confusion over what this does.
I don't have a problem if it scans everything, but its not. Let's stick to what is doing. Android could do this as well, so talking about what companies like google could do is not so interesting - they could do almost anything.
I don't have a problem if it scans everything, but its not. Let's stick to what is doing. Android could do this as well, so talking about what companies like google could do is not so interesting - they could do almost anything.
Google does everything they can to backup your photos ... and do it automatically.
I'm not sure there's a real difference unless you want to watch your settings all the time. In google land they tend to reset ... and really that happens a lot of places.
I think for most people if you use google, you're in their cloud.
I'm not sure there's a real difference unless you want to watch your settings all the time. In google land they tend to reset ... and really that happens a lot of places.
I think for most people if you use google, you're in their cloud.
Isn't the default on Android these days that all images get uploaded ("backed up") to Photos? And how many users are actually altering defaults?
> B) They scan everything that exists on my device
No ... They scan everything that I have released to apple photos that exists on my device.
Same scan - different place.
No ... They scan everything that I have released to apple photos that exists on my device.
Same scan - different place.
The issue is that as soon as you set that precedent, it’s only a matter of time before it extends beyond iCloud. That’s the problem with doing any device-level scanning. This is dystopian and scary. And yes I understand the technology in its current iteration. The current form has problems (weaponizing collisions etc) but the real issue comes with future developments.
> They scan everything that exists on my device
I get to select the issue and it was in response to the previous claim that 'they' are scanning everything that exists on the device right now.
A year ago this was a possible 'future development'. 10 years from now I could be living on Mars. This is all hypothetical.
I get to select the issue and it was in response to the previous claim that 'they' are scanning everything that exists on the device right now.
A year ago this was a possible 'future development'. 10 years from now I could be living on Mars. This is all hypothetical.
> This is all hypothetical.
Exactly. However, people don't seem to have an issue when hypothesizing that CSAM detection is good actually, because Apple might implement E2EE, despite no evidence of such intentions.
For some reason, though, people take issue when others hypothesize that CSAM detection is bad actually, because Apple might expand it further and violate users' privacy even more. And there's actually precedent for this hypothesis, given Apple's actions here and their own words[1]:
> This program is ambitious, and protecting children is an important responsibility. These efforts will evolve and expand over time.
[1] https://www.apple.com/child-safety/
Exactly. However, people don't seem to have an issue when hypothesizing that CSAM detection is good actually, because Apple might implement E2EE, despite no evidence of such intentions.
For some reason, though, people take issue when others hypothesize that CSAM detection is bad actually, because Apple might expand it further and violate users' privacy even more. And there's actually precedent for this hypothesis, given Apple's actions here and their own words[1]:
> This program is ambitious, and protecting children is an important responsibility. These efforts will evolve and expand over time.
[1] https://www.apple.com/child-safety/
Was scanning server side not a precedent?
Furthermore, was uploading (backing up) the entire contents of your phone to a server to be freely scaned and distributed (with warrant of course) server side not also a precedent?
Must have been absloutely nuts round here back in 2011 when they announced the particular slippery slope that is iCloud backup.
Must have been absloutely nuts round here back in 2011 when they announced the particular slippery slope that is iCloud backup.
It set the precedent for scanning server side. Which I don’t like but there’s really no way to avoid it anyway.
Now we have clientside scanning, and actually being led by the [note the scare quotes] “pro-privacy” two ton gorilla. It’s a whole different ballgame.
Now we have clientside scanning, and actually being led by the [note the scare quotes] “pro-privacy” two ton gorilla. It’s a whole different ballgame.
It was but it was one we were not in a position to resist at the time. Surrendered to almighty convenience.
But here appears to be beyond the pale such that people are saying "this far, no farther!"
But here appears to be beyond the pale such that people are saying "this far, no farther!"
They are very likely aware of backlash but since this is a easily defendable hill with a very slippery slope down the way, it is of their interest to push for it IMHO.
Apple has declared in interviews that the slope shall not be slipped, but you're indicating that they chose this specifically because they can slip that slope.
How did you determine that their intentions contradict their words? Please share the framework for your belief, so that we're able to understand how you arrived at that belief and to evaluate your evidence with an open mind.
(Or, if your claim is unsupported conjecture, please don't misrepresent your opinions and beliefs as facts here at HN.)
How did you determine that their intentions contradict their words? Please share the framework for your belief, so that we're able to understand how you arrived at that belief and to evaluate your evidence with an open mind.
(Or, if your claim is unsupported conjecture, please don't misrepresent your opinions and beliefs as facts here at HN.)
Words in interviews mean little to me. If it's impossible to hold a cooperation accountable when the verbal promise is broken, that promise, not word-by-word but in its spirit, is void from the beginning.
This is just my biased view but there _could_ be countless ways to slip the slope without violating the current framing. For example:
Q: What happens when other governments ask Apple to use this for other purposes.
A: We will inform them that we did not build the thing they’re thinking of.[1]
Note that the question could be interpreted as "use this exact implementation" (rather than use the algorithm in general), and the answer does not rule out the possibility of "but we can build the thing for them then" (rather than "we don't have it and we will never have it"). The reader is free to interprete the conversation as they see it.
[1] https://daringfireball.net/2021/08/apple_child_safety_initia... citing [2]
[2] https://www.nytimes.com/2021/08/05/technology/apple-iphones-...
This is just my biased view but there _could_ be countless ways to slip the slope without violating the current framing. For example:
Q: What happens when other governments ask Apple to use this for other purposes.
A: We will inform them that we did not build the thing they’re thinking of.[1]
Note that the question could be interpreted as "use this exact implementation" (rather than use the algorithm in general), and the answer does not rule out the possibility of "but we can build the thing for them then" (rather than "we don't have it and we will never have it"). The reader is free to interprete the conversation as they see it.
[1] https://daringfireball.net/2021/08/apple_child_safety_initia... citing [2]
[2] https://www.nytimes.com/2021/08/05/technology/apple-iphones-...
Apple's management can also be prone to hubris. This is also very much a case where the engineers were left unbridled without any proper check from marketing and comms, I suspect because of the extreme complexity of the problem and the sheer impossibility of putting it into layman terms effectively.
Was this a pure engineering driven exercise? I hadn't heard that before and it doesn't match up with what I've heard about Apple's internal culture. The level of defensive pushback really makes me feel that they are very bought into the system.
Definitely would appreciate a link to anything substantial indicating that this was a bunch of eng in over their heads.
Definitely would appreciate a link to anything substantial indicating that this was a bunch of eng in over their heads.
Also have to agree: I don't see how this could originate from engineers. Every engineer I've spoken with at the company I work for has been mortified by this insanity.
It’s really unpopular inside the fruit company. I’ve not spoken with a single engineer in Cupertino who thinks this is a good path. There has been radio silence from management on how to address this. It almost feels like they want this to fail. In the past management has given us resources and talking points we can use with our friends and family. Not this time.
> In the past management has given us resources and talking points we can use with our friends and family.
Wow. That feels like an overreach. 'We don't just buy your labor, but also your private opinion and tell you how to talk to your family'.
Wow. That feels like an overreach. 'We don't just buy your labor, but also your private opinion and tell you how to talk to your family'.
Certainly people tow the official line, but it does help us to articulate the position of the company so that we aren’t so misinformed that we spread FUD.
This doesn't hold much water. As a counter example, we have all of the engineers that designed Facebook, Google, etc. We have people working in ad tech. We have people working in lots of places that you and the people that you talk with directly would be moritified, but there are countless others that need a paycheck and do whatever pointy haired bosses tell them to do.
Maybe the implementation is difficult to put into layman’s terms, but the high level goals - scanning the devices for child porn - most definitely is not.
I’m not buying the “engineers were left unbridled” argument, I think there just have been a level of obliviousness in much wider a part of the organization for something like this to happen.
I’m not buying the “engineers were left unbridled” argument, I think there just have been a level of obliviousness in much wider a part of the organization for something like this to happen.
> What's shocking to me is how little Apple management understood of what their actions looked like. Really stunning.
Maybe because they underestimated people's ignorance.
I think they saw (and still do see) it as a better, more privacy preserving technique than what everyone else is doing.
Maybe because they underestimated people's ignorance.
I think they saw (and still do see) it as a better, more privacy preserving technique than what everyone else is doing.
The thing is that this is a better and more privacy preserving technique.
Their hubris is in not seeing or thinking that they will be able to stand up to all kinds of abuses of this system that its mere existence will invite; their hubris is also in thinking that they will be able to perfectly and without mistakes manage and overview a system making accusations so heinous that even the mere act of accusing destroy people's lives and livelihoods.
Their hubris is in not seeing or thinking that they will be able to stand up to all kinds of abuses of this system that its mere existence will invite; their hubris is also in thinking that they will be able to perfectly and without mistakes manage and overview a system making accusations so heinous that even the mere act of accusing destroy people's lives and livelihoods.
What abuses apply to CSAM scanning in this hybrid pipeline which don’t apply to iCloud Backup? If they scanned on the server, why couldn’t governments “slippery slope” abuse the system by requiring that all files on iOS end up in iCloud Backup, where they can be scanned by the system?
Since the announcement, I can think of a dozen ways Apple could be easily forced into scanning all the contents of your device by assembling features they’ve already shipped. Yet they haven’t. At some point, people need to produce evidence that Apple cannot hold the line they’ve said they will.
Since the announcement, I can think of a dozen ways Apple could be easily forced into scanning all the contents of your device by assembling features they’ve already shipped. Yet they haven’t. At some point, people need to produce evidence that Apple cannot hold the line they’ve said they will.
The governments always could do that, but then they'd be the target of the pushback, and there's already significant mistrust of governments wrt surveillance because of how it can be abused.
What we have now is Apple, with its "strong privacy" record, normalizing this. If it succeeds, it would be that much easier for the governments to tackle other stuff onto it. Or, say, lower the threshold needed to submit images for review. I can easily picture some senator ranting about how unacceptable it is that somebody with only 20 CSAM photos won't be flagged, and won't somebody please think of the children?
And yes, if it comes to that, Apple definitely cannot hold the line. After all, they already didn't hold it on encrypted cloud storage - and that wasn't even legally forced on them, merely "not recommended".
What we have now is Apple, with its "strong privacy" record, normalizing this. If it succeeds, it would be that much easier for the governments to tackle other stuff onto it. Or, say, lower the threshold needed to submit images for review. I can easily picture some senator ranting about how unacceptable it is that somebody with only 20 CSAM photos won't be flagged, and won't somebody please think of the children?
And yes, if it comes to that, Apple definitely cannot hold the line. After all, they already didn't hold it on encrypted cloud storage - and that wasn't even legally forced on them, merely "not recommended".
Exactly. Also people are strongly objecting that their own device is being used to report them to law enforcement. Surely someone at Apple noticed this beforehand ...
> For a company that marketed itself as one of the few digital service providers that consumers could trust, I just don't understand how they acted this way at all.
Because privacy stance is mostly PR to differentiate from Google. And while there're invalid reasons to get users data, there're also valid ones (at least from legal requirement point of view - let's not get into weeds about personal freedom here and if the laws and its implementations need to be changed).
Their PR was just writing the checks they cannot cash without going on a war with governments.
Because privacy stance is mostly PR to differentiate from Google. And while there're invalid reasons to get users data, there're also valid ones (at least from legal requirement point of view - let's not get into weeds about personal freedom here and if the laws and its implementations need to be changed).
Their PR was just writing the checks they cannot cash without going on a war with governments.
It was obvious from the beginning that the privacy nonsense was a convenient excuse to cover for their poor (especially at the time) cloud offerings compared to competitors like Google.
I assumed they pivoted to focus on privacy, but clearly it was just a marketing department innovation rather than a core value (as their marketing department claimed).
I assumed they pivoted to focus on privacy, but clearly it was just a marketing department innovation rather than a core value (as their marketing department claimed).
In retrospect this makes sense to me. I don't like it, but I get it now. When Apple said "privacy" what they meant was "we don't like tracking cookies or hackers but everything else is fine".
Privacy means “you pay for the device so we don’t sell your attention to advertisers.” That’s it. There’s no protection against state level actors (Pegasus, CSAM scanning, etc.).
If your threat model includes being the target of someone who will plant child pornography on your phone, you are already fucked. And no, Apple isn’t suddenly going to scan Chinese iPhones for Winnie the Pooh memes. They don’t have to. China already has the 50 cent party to do that for them, on WeChat.
Basically everything everyone seems to think is just around the corner has already been possible for years.
If your threat model includes being the target of someone who will plant child pornography on your phone, you are already fucked. And no, Apple isn’t suddenly going to scan Chinese iPhones for Winnie the Pooh memes. They don’t have to. China already has the 50 cent party to do that for them, on WeChat.
Basically everything everyone seems to think is just around the corner has already been possible for years.
Is there a consumer computing device that’s more secure than iPhone against state actors? A Chromebook, maybe?
There is no consumer grade device that is secure against state actors.
HN's Point of View doesn't represent the PoV of the 1 billion active iPhone owners.
Most people won't have any idea about the meaning of "hashes" and "databases". Not everyone is trying to actively fight the system and shit on everything, most people just want to live happily with their friends and family, they won't care that Apple scans their devices.
> "Either there will be heads rolling at management, or Apple takes a permanent hit to consumer trust."
Oh god ! How wasn't all of this obvious to the top Apple management, but so obvious to epistasis! Damn, thanks man for correcting and leading Apple to the right track !
Most people won't have any idea about the meaning of "hashes" and "databases". Not everyone is trying to actively fight the system and shit on everything, most people just want to live happily with their friends and family, they won't care that Apple scans their devices.
> "Either there will be heads rolling at management, or Apple takes a permanent hit to consumer trust."
Oh god ! How wasn't all of this obvious to the top Apple management, but so obvious to epistasis! Damn, thanks man for correcting and leading Apple to the right track !
We will see. I've heard lots of these predictions and I don't buy them AT ALL.
What I have seen is a selling point for apple products.
I'd encourage folks to get out of the HN bubble on this - talk to a wife, a family especially those with kids.
What I have seen is a selling point for apple products.
I'd encourage folks to get out of the HN bubble on this - talk to a wife, a family especially those with kids.
Yeah, talk to people that know nothing on the issue besides that they want to “protect the kids”.
Why stop there? Get out of the HN bubble on the patriot act, instead ask your neighbor’s wife her thoughts on it. Get out of the HN bubble on immigration, go ask a stereotypical boomer conservative about it.
I think my sarcasm already made it overtly obvious but, this is horrible advice you are giving and the fact that you don’t seem to be aware that pedophilia and terrorism are the two most classic “this gives us an excuse to exert totalitarian control” topics betrays your own ignorance (or, worse, you are aware and just don’t care).
Why stop there? Get out of the HN bubble on the patriot act, instead ask your neighbor’s wife her thoughts on it. Get out of the HN bubble on immigration, go ask a stereotypical boomer conservative about it.
I think my sarcasm already made it overtly obvious but, this is horrible advice you are giving and the fact that you don’t seem to be aware that pedophilia and terrorism are the two most classic “this gives us an excuse to exert totalitarian control” topics betrays your own ignorance (or, worse, you are aware and just don’t care).
Nah, their stock is up which means they'll continue on this trajectory.
Consumers are probably in favor this, or don’t care.
The only people who are bothered are people claiming this is going to be misused by authoritarian governments.
The only people who are bothered are people claiming this is going to be misused by authoritarian governments.
Why are hash collisions relevent?
There are atleast 2-3 further checks to account for this.
There are atleast 2-3 further checks to account for this.
Because it's not a cryptographic hash where a one bit difference results in a completely different hash. It's a perceptual hash that operates on a smaller bitmap derived from the image so it's plausible that some innocuous images might result in similar derivations; and there might be intentionally crafted innocently-looking images that result in an offensive derivative.
Salvador Dali could do something similar by hand in 1973 in Gala Contemplating the Mediterranean Sea [1]
[1] https://en.wikipedia.org/wiki/Lincoln_in_Dalivision
Salvador Dali could do something similar by hand in 1973 in Gala Contemplating the Mediterranean Sea [1]
[1] https://en.wikipedia.org/wiki/Lincoln_in_Dalivision
This is a great answer but that’s not actually the GP’s contention. Their argument is essentially “so what if there’s a collision, the human review will catch it”. And to that I’d say that the same is supposed to occur for the no-fly list and we all know how that works in practice.
The mere accusal itself of possessing CSAM can be life ruining if it gets to that stage. More importantly, a collision will effectively allow warrantless searches, at least of the collided images.
The mere accusal itself of possessing CSAM can be life ruining if it gets to that stage. More importantly, a collision will effectively allow warrantless searches, at least of the collided images.
Indeed, I touched on that in another comment: https://news.ycombinator.com/item?id=28227141
That’s one check. There are other system checks to make the client side hash collision meaningless.
Do you understand that anyone can take absolutely legal porn and make it match CSAM hash? And no one except NCMEC can know the difference because they all only compare hashes and not actual images.
And whoever going to check images for Apple will see that yeah, there is porn on picture. Flag it. Then you'll have unlimited amount of time to explain to FBI why some porn on your device match CSAM hash.
And whoever going to check images for Apple will see that yeah, there is porn on picture. Flag it. Then you'll have unlimited amount of time to explain to FBI why some porn on your device match CSAM hash.
> The system relies on a database of hashes—cryptographic representations of images—of known CSAM photos provided by National Center for Missing & Exploited Children (NCMEC) and other child protection organizations.
“Cryptographic representations of images”. That’s not the case though right? These are “neuralhashes” afaik which are nowhere close to cryptographic hashes but rather locality sensitive hashes which is a fancy speak for “the closer two images look like, the more similar the hash”.
Vice and others keeps calling the cryptographic. Am I missing something here?
“Cryptographic representations of images”. That’s not the case though right? These are “neuralhashes” afaik which are nowhere close to cryptographic hashes but rather locality sensitive hashes which is a fancy speak for “the closer two images look like, the more similar the hash”.
Vice and others keeps calling the cryptographic. Am I missing something here?
Something to note here is that in the hash collision that was discovered, the two images look nothing alike. One is a picture of a dog, the other is blobby grey static.
Take a look at this collision https://twitter.com/SarahJamieLewis/status/14280837442802565...
Those are two almost identical images based off each other.
I know that that "think of the children" is a meme, but I think this illustrates the point for apple. If you have a croped or modified image of CSA the system will identify it. As long as your image is different enough from CSA, you are safe.
The point here is that Apple is specifically looking for matches against known CSA material.
If someone can demonstrate that a legal NSFW image (eg. regular old-fashioned pornography), can be collided with a legal, completely 100% SFW image then I'll be concerned.
But until then, this looks like a reasonable and supportable way for finding CSAM in real time.
I know that that "think of the children" is a meme, but I think this illustrates the point for apple. If you have a croped or modified image of CSA the system will identify it. As long as your image is different enough from CSA, you are safe.
The point here is that Apple is specifically looking for matches against known CSA material.
If someone can demonstrate that a legal NSFW image (eg. regular old-fashioned pornography), can be collided with a legal, completely 100% SFW image then I'll be concerned.
But until then, this looks like a reasonable and supportable way for finding CSAM in real time.
> If someone can demonstrate that a legal NSFW image (eg. regular old-fashioned pornography), can be collided with a legal, completely 100% SFW image then I'll be concerned.
Look at this other collision: https://twitter.com/SarahJamieLewis/status/14282060881181491... An attacker can send you an innocent looking picture that embbed some CSA material and you get swatted the next day.
Look at this other collision: https://twitter.com/SarahJamieLewis/status/14282060881181491... An attacker can send you an innocent looking picture that embbed some CSA material and you get swatted the next day.
I think you are getting downvoted because thats the same image I replied to.
Looking at those two images its plain to see why they are identical.
And if someone is sending you CSA material, isn't that the point of this process. Apple identifies it as CSA, can give you a warning its sensitive, and identify they authorities that people are sending CSA.
Again - this seems like a win. If Apple can automatically identify CSA material, en-masse thats good.
edit: It looks like they are different URLs, but Twitter only allows people to see replies if they are logged in, so I can't see that example.
edit 2: On further thought, if someone can use CSA material and produce an innocuous image with a similar hash, if they send that image to you, its still proof that the sender had CSA material. Again, its still good.
Looking at those two images its plain to see why they are identical.
And if someone is sending you CSA material, isn't that the point of this process. Apple identifies it as CSA, can give you a warning its sensitive, and identify they authorities that people are sending CSA.
Again - this seems like a win. If Apple can automatically identify CSA material, en-masse thats good.
edit: It looks like they are different URLs, but Twitter only allows people to see replies if they are logged in, so I can't see that example.
edit 2: On further thought, if someone can use CSA material and produce an innocuous image with a similar hash, if they send that image to you, its still proof that the sender had CSA material. Again, its still good.
They actually do look alike to my eye, but in a “the way the algorithm sees it” kind of way. I can see the obvious similarity. But to your point it’s not like it’s two very slightly different photos of dogs.
You must be a champion CAPTCHA solver.
Furthermore, it may well be possible to combine that blobby grey static with another image, manipulating it's visual hash to create a "sleeper" positive. If this was possible in a week, then it's going to be very interesting to watch the technology change/evolve over the next few years.
Doing so would create a positive that still doesn't pass Apple's human visual check against the (blurred) CSAM content associated with that checksum, and if it somehow did, it would still then also have to occur at qty.30 or more, and they'd have to pass a human visual check against the (unblurred) CSAM content by one of the agencies in possession of it. It's not possible to spoof that final test unless you possess real CSAM content, at which point you don't need to spoof that final test and you'll end up arrested for possession.
There are concerns without the images needing to make it all the way through their CSAM process. Apple is a US company with obligations to report certain crimes they become aware of, and NCMEC is a US government-tied organisation that dismissed privacy concerns as "the screeching voices of the minority".
Consider if the honeypot images (manipulated to match CSAM hashes) are terrorist recruitment material for example.
Consider if the honeypot images (manipulated to match CSAM hashes) are terrorist recruitment material for example.
And if you make the second image be actual, 18+ porn? Will Apple be able to tell the difference between that and CSAM after the blur is applied?
Bonus points if you match poses, coloration, background, etc.
Bonus points if you match poses, coloration, background, etc.
The only way to match poses, coloration, background, etc is to possess illegal CSAM content. If you do so successfully, you will result in your crafted image passing the blur check and reaching the agency that possesses the original image for final verification, where it will immediately fail because it is obviously a replica. You will then trigger that agency leading law enforcement to find the creator of the image, so that they can arrest whoever possesses CSAM content in order to model replicas after it.
I think that's a perfectly acceptable outcome, since anyone with the hubris to both possess CSAM content and create replicas of it especially deserves to be arrested. Do you see a more problematic outcome here?
I think that's a perfectly acceptable outcome, since anyone with the hubris to both possess CSAM content and create replicas of it especially deserves to be arrested. Do you see a more problematic outcome here?
Cops no-knock raid the wrong houses all the time; what makes you think that the procedure you outlined above will be followed without error? Especially against suspected child abusers, from an anti-child-abuse agency?
What if the visual check gets accidentally signed off on, or even gets fraudulently marked as positive by a burned out/lazy/competent-but-distracted employee? There are just so many failure modes for this dragnet that all end with innocent people suffering a very difficult process, even if they don't eventually land in prison. I don't think it's unreasonable to not want to be volunteered to participate.
What if the visual check gets accidentally signed off on, or even gets fraudulently marked as positive by a burned out/lazy/competent-but-distracted employee? There are just so many failure modes for this dragnet that all end with innocent people suffering a very difficult process, even if they don't eventually land in prison. I don't think it's unreasonable to not want to be volunteered to participate.
This assumes that it's impossible to reverse the perceptual hashes used here in such a way that you could determine poses and coloration, for one.
And in retrospect, you don't need to match that - you just need it to appear obviously pornographic after the blur is applied in order to get past Apple's reviewers. After that, the lucky individual's life is in the hands of the police/prosecutors. (I have to imagine that both real and faked cases will look pretty much like "Your honor/members of the jury, this person's device contained numerous photos matching known CSAM. No, we won't be showing you the pictures. No, the defence can't see them either." Can you imagine a "tough on crime" prosecutor taking the faked case to trial too? Would the police and prosecutors even know it was faked?)
And in retrospect, you don't need to match that - you just need it to appear obviously pornographic after the blur is applied in order to get past Apple's reviewers. After that, the lucky individual's life is in the hands of the police/prosecutors. (I have to imagine that both real and faked cases will look pretty much like "Your honor/members of the jury, this person's device contained numerous photos matching known CSAM. No, we won't be showing you the pictures. No, the defence can't see them either." Can you imagine a "tough on crime" prosecutor taking the faked case to trial too? Would the police and prosecutors even know it was faked?)
Apple's reviewers are comparing blurred originals to blurred matches. It needs to look like the blurred original associated with the checksum that matched. It is irrelevant whether the blurred match looks pornographic or not.
This has already been demonstrated.
https://twitter.com/SarahJamieLewis/status/14280837442802565...
https://twitter.com/SarahJamieLewis/status/14280837442802565...
At which point the image will be handed to the relevant CSAM group unblurred, who will do a visual comparison and find out immediately that it’s not a match, and then reject it without invoking law enforcement.
Why would attacker need to create anything from scratch? If you want to build a dataset of images with people with specific skin and hair color, body types, etc in very specific poses it's can be quite hard and expensive to do so. Because even photo stocks have limited number of such photos. Unless...
Unless you're looking to build a porn dataset and you're don't care about copyright. Porn is industry where exabytes of material are produced and published on internet almost every week.
Who will agency come to? To some OnlyFans creators?
Unless you're looking to build a porn dataset and you're don't care about copyright. Porn is industry where exabytes of material are produced and published on internet almost every week.
Who will agency come to? To some OnlyFans creators?
[deleted]
At the point where it is submitted for a human visual check, your privacy has already been violated.
My privacy is violated every time I leave my home. Anyone can take a photo of me and look up the FBI Most Wanted and see if I’m there. If they think someone is me and they’re wrong, they can still summon law enforcement, and I’ll still be mistreated for their poor judgement.
Is this just as unacceptable as the CSAM scanning? Should all public photography be banned, in order to reduce the risk of false positive identifications of innocent people as criminals to zero? Or is that an acceptable degree of privacy impingement for the good of society?
Is Apple’s implementation an acceptable trade of impingement and risk, for good for society? We do live in a society, and so zero impingement upon privacies is never going to be acceptable (sorry, free/tech/libs). But instead of discussing whether Apple’s approach violates privacy minimally or not in order to get the job done, these discussions here just keep circling the drain of “putting my privacy at risk by any degree is never acceptable”, when that drain is cemented shut by the existence of society and will never lead to a valid outcome.
Is this just as unacceptable as the CSAM scanning? Should all public photography be banned, in order to reduce the risk of false positive identifications of innocent people as criminals to zero? Or is that an acceptable degree of privacy impingement for the good of society?
Is Apple’s implementation an acceptable trade of impingement and risk, for good for society? We do live in a society, and so zero impingement upon privacies is never going to be acceptable (sorry, free/tech/libs). But instead of discussing whether Apple’s approach violates privacy minimally or not in order to get the job done, these discussions here just keep circling the drain of “putting my privacy at risk by any degree is never acceptable”, when that drain is cemented shut by the existence of society and will never lead to a valid outcome.
By leaving your home and going out in public, you are implicitly consenting to being observed. So, no, that's not a violation of your privacy.
Apple's implementation is more like a stranger sneaking into your house if they suspect you're up to something bad. That would be equally unacceptable - we have warrants for a reason.
Apple's implementation is more like a stranger sneaking into your house if they suspect you're up to something bad. That would be equally unacceptable - we have warrants for a reason.
I dont really think it'd be a valid second pre-image for this type of hash if they did look similar.
I'd be more interested in whether you could take an image of, say, an underage dog, and tweak a vaguely similar image of a grown dog so that it was a match. That's what will get innocent people thrown in jail.
No - the reporting is absolutely terrible here.
1) These are more share similar visual features than crypto hashes.
2) HN posters have been claiming that apple reviewing flagged photos is a felony -> because HN commentators are claiming flagged photos are somehow "known" CASM - this is also likely totally false. The images may not be CASM and the idea that a moderation queue results in felony charges is near ridiculous.
3) This illustrates why apple's approach here (manual review after 30 images or so flagged) is not unreasonable. The push to say that this review is unnecessary is totally misguided.
4) They use words like "hash collision" for something that is not a hash. In fact, different devices will calculate DIFFERENT hashes for the SAME image at times.
One request I have - before folks cite legal opinions - those opinions should have the name of a lawyer on them. Not this "I talked to a lawyer" because we have no idea if you described things accurately.
1) These are more share similar visual features than crypto hashes.
2) HN posters have been claiming that apple reviewing flagged photos is a felony -> because HN commentators are claiming flagged photos are somehow "known" CASM - this is also likely totally false. The images may not be CASM and the idea that a moderation queue results in felony charges is near ridiculous.
3) This illustrates why apple's approach here (manual review after 30 images or so flagged) is not unreasonable. The push to say that this review is unnecessary is totally misguided.
4) They use words like "hash collision" for something that is not a hash. In fact, different devices will calculate DIFFERENT hashes for the SAME image at times.
One request I have - before folks cite legal opinions - those opinions should have the name of a lawyer on them. Not this "I talked to a lawyer" because we have no idea if you described things accurately.
>> those opinions should have the name of a lawyer on them.
Not going to happen. Lawyers in the US have issues with offering unsolicited advice, and other problems with issuing advice into states where they are not admitted. So likely none of the US lawyers (and the great many more law students) here will ever put their real name to a comment.
Not going to happen. Lawyers in the US have issues with offering unsolicited advice, and other problems with issuing advice into states where they are not admitted. So likely none of the US lawyers (and the great many more law students) here will ever put their real name to a comment.
This. Also try contacting a lawyer who knows this area and asking to pay for a legal opinion brief so that you can post it online to be debated by legions of software developers.
Lawyers I know would politely decline that.
Lawyers I know would politely decline that.
No - this is actually done supposedly as part of biz dev.
So your own firm may cover some costs if you have something to say. If you found someone to pay for you to do an analysis or offer your thoughts - you'd be in heaven!
So your own firm may cover some costs if you have something to say. If you found someone to pay for you to do an analysis or offer your thoughts - you'd be in heaven!
This is incorrectly applied. Offering a legal opinion is fundamentally different from offering legal advice. We publish legal opinions in academic and professional publications all the time. That doesn't mean we have an attorney-client relationship with anyone who reads those opinions, or that we would advise that someone act in accordance with such an opinion, particularly if no court has adopted our position yet.
but there are legal opinions that law firms and various orgs(political or not) that wite on occasion from actual lawyers sharing a legal opinion, publicly(or not).
But it's the law, it's fuzzy at best, much like your HR department. It's only after a court decision has been reached on your particular issue that it's anywhere near "settled" case law, and even that's up for possible change tomorrow.
But it's the law, it's fuzzy at best, much like your HR department. It's only after a court decision has been reached on your particular issue that it's anywhere near "settled" case law, and even that's up for possible change tomorrow.
Right. That's why there's never been an amicus brief or a friends of the court type of document drafted and signed by lawyers.
Those are addressed to a court or government agency in a specific place. The advice offered isn't for a bunch of rando people on the internet across all number of jurisdictions. And it is only general advice about an area of law, normally at the appellate level, not specific fact-dependent advice to a real flesh-and-blood person. Amicus is also normally an opinion to sway the court on broad policy, not a determination of facts in a specific case.
Sure - but if you are going to write blog posts / articles - and all you can say is based on the lawyers I talked to apple is committing child porn felonies, that is just unacceptable.
At least HN should flag these and get these taken down. Over and over the legal analysis is either trash or it's clear the article author didn't understand something (so how can lawyer give good advice?).
These conversations become so uninteresting when people take these extreme type positions. Apple's brand is destroyed - apple is committing child porn felonies.
I would have rather just had a link to the apple technical paper and a discussion personally vs the over the top random article feed with all sorts of misunderstandings.
And in contract law there are LOTS of legal articles online - with folks name on them! They are useful! I read them and enjoy them. Can we ask for that here where it matters maybe more?
At least HN should flag these and get these taken down. Over and over the legal analysis is either trash or it's clear the article author didn't understand something (so how can lawyer give good advice?).
These conversations become so uninteresting when people take these extreme type positions. Apple's brand is destroyed - apple is committing child porn felonies.
I would have rather just had a link to the apple technical paper and a discussion personally vs the over the top random article feed with all sorts of misunderstandings.
And in contract law there are LOTS of legal articles online - with folks name on them! They are useful! I read them and enjoy them. Can we ask for that here where it matters maybe more?
>> there are LOTS of legal articles online - with folks name on them
Articles are not legal advice. They are opinions on the law applicable generally, rather than fact-based advice to specific clients. Saying whether apple is doing something illegal or not in this case, with a lawyer's name stamped on that opinion, is very different.
Articles are not legal advice. They are opinions on the law applicable generally, rather than fact-based advice to specific clients. Saying whether apple is doing something illegal or not in this case, with a lawyer's name stamped on that opinion, is very different.
> The images may not be CASM and the idea that a moderation queue results in felony charges is near ridiculous.
Agree, and I think this is backed up by real world experience. Has Facebook or anyone working on their behalf ever been charged for possession of CSAM? I guarantee they've seen some. Probably a lot, in fact. That's why we have recurring discussions about the workers and the compensation they get (or not) for the really horrid work they are tasked with.
Agree, and I think this is backed up by real world experience. Has Facebook or anyone working on their behalf ever been charged for possession of CSAM? I guarantee they've seen some. Probably a lot, in fact. That's why we have recurring discussions about the workers and the compensation they get (or not) for the really horrid work they are tasked with.
From what I understand it's even more cut and dry than that. If you submit a report to the NCMEC you are generally required to (securely) keep a copy of the image(s) being reported and any related info for a period of time. That's part of the rules. You are also only compelled to report things that you know are bad, so verification before reporting makes sense. The idea that they would be charged with a crime for doing what is essentially required by law is flat out wrong. Unless they are storing the material insecurely once confirmed bad or otherwise mishandling the process, they seem to be following the law as I understand it.
IANAL but I have an account with the NCMEC for a service I run, so I have looked through their documentation and relevant laws to try to understand the requirements placed on me as a service provider.
Thank you for a first hand report. All that makes sense - you clearly can't and shouldn't "pass around images" even in the office - so fair that it is super strict once something is confirmed as X.
More of an anecdote, but our software allows users to upload files, documents, images and such. This in turn means, technically speaking, our platform could be used to distribute malware or other illegal content. I've asked: Well what happens if that occurs and we as a company becomes aware?
Overall, we're advised to take about these steps there. First off, report it. Second, remove all access for the customer, terminate the accounts, lock them out asap. Third, prevent access to the content without touching it. For example, if it sits on a file system and a web server could serve it, blacklist URLs on a loadbalancer. Fourth, if necessary, begin archiving and securing evidence. But if possible in any way, disable content deletion mechanisms and wait for legal advice, or the law enforcement to tell you how to gather the data.
But overall, you're not immediately guilty for someone abusing your service, and no one is instantly guilty for detecting someone is abusing your service.
Overall, we're advised to take about these steps there. First off, report it. Second, remove all access for the customer, terminate the accounts, lock them out asap. Third, prevent access to the content without touching it. For example, if it sits on a file system and a web server could serve it, blacklist URLs on a loadbalancer. Fourth, if necessary, begin archiving and securing evidence. But if possible in any way, disable content deletion mechanisms and wait for legal advice, or the law enforcement to tell you how to gather the data.
But overall, you're not immediately guilty for someone abusing your service, and no one is instantly guilty for detecting someone is abusing your service.
> Probably a lot, in fact.
> In 2020, FotoForensics received 931,466 pictures and submitted 523 reports to NCMEC; that's 0.056%. During the same year, Facebook submitted 20,307,216 reports to NCMEC
https://www.hackerfactor.com/blog/index.php?/archives/929-On....
> In 2020, FotoForensics received 931,466 pictures and submitted 523 reports to NCMEC; that's 0.056%. During the same year, Facebook submitted 20,307,216 reports to NCMEC
https://www.hackerfactor.com/blog/index.php?/archives/929-On....
The problem isn't the idea of CASM, it's that what happens when they start scanning for "misinformation", or whatever else they want to come up with at that point.
Then why not wait until that actual issue comes up. The look is so bad / awkward with this big protest over something that many people are not going to find at all objectionable.
Do you really think Apple's brand has been "destroyed" over this?
Do you really think Apple's brand has been "destroyed" over this?
IMO the risk is that the reaction will be viewed amongst the general population as an overreaction, leading to it being ignored on the next go around. Many people are going to listen to what Apple says, look at their piles of technical documentation, see the word CSAM, and conclude that the opposing side is crazy.
The neural hash itself isn’t cryptographic, but there’s cryptography involved in the process.
They use “private set intersection” (https://en.wikipedia.org/wiki/Private_set_intersection) to compute a value that itself doesn’t say whether an image is in the forbidden list, yet when combined with sufficiently many other such values can be used to do that.
They also encrypt the “NeuralHash and a visual derivative” on iCloud in such a way that Apple can only decrypt that if they got sufficiently many matching images (using https://en.wikipedia.org/wiki/Secret_sharing)
(For details and, possibly, corrections on my interpretation, see Apple’s technical summary at https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni... and https://www.apple.com/child-safety/pdf/Apple_PSI_System_Secu...)
They use “private set intersection” (https://en.wikipedia.org/wiki/Private_set_intersection) to compute a value that itself doesn’t say whether an image is in the forbidden list, yet when combined with sufficiently many other such values can be used to do that.
They also encrypt the “NeuralHash and a visual derivative” on iCloud in such a way that Apple can only decrypt that if they got sufficiently many matching images (using https://en.wikipedia.org/wiki/Secret_sharing)
(For details and, possibly, corrections on my interpretation, see Apple’s technical summary at https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni... and https://www.apple.com/child-safety/pdf/Apple_PSI_System_Secu...)
This is one of the most common misunderstandings, both when people are arguing for or against.
People who understand tech well enough to recognize hashes like MD5 and SHA don't dive deep enough to understand that this is something completely different.
I even suspect this is deliberate from Apple's side when announcing and talking about these changes - making people wrongly believe that only exact matches will trigger, except possibly in extremely rare cases and under concious attacks.
They could have called it "fingerprint" or something but deliberately went with a technical term that even confuses technical people who know well enough what a hash usually means.
Vice is falling victim to this misunderstanding stemming from the conflation of "hash".
People who understand tech well enough to recognize hashes like MD5 and SHA don't dive deep enough to understand that this is something completely different.
I even suspect this is deliberate from Apple's side when announcing and talking about these changes - making people wrongly believe that only exact matches will trigger, except possibly in extremely rare cases and under concious attacks.
They could have called it "fingerprint" or something but deliberately went with a technical term that even confuses technical people who know well enough what a hash usually means.
Vice is falling victim to this misunderstanding stemming from the conflation of "hash".
Yea by reading about this whole scandal I learned about perceptual hashes, because the whole time I kept thinking "can't they just alter one pixel and entirely change the hash" because I was only familiar with cryptographic hashes.
It's a huge, huge, huge distinction.
It's a huge, huge, huge distinction.
The CSAM hashes are encrypted and blinded, possible Vice and others conflated and/or combined steps.
In a certain sense it is cryptographic because the original CSAM images remains secret. While the hash is not useful for maintaining integrity it still provides confidentiality.
Edit: this is apparently not true as demonstrated by researchers.
Edit: this is apparently not true as demonstrated by researchers.
This has technically been proven false:
> Microsoft says that the "PhotoDNA hash is not reversible". That's not true. PhotoDNA hashes can be projected into a 26x26 grayscale image that is only a little blurry. 26x26 is larger than most desktop icons; it's enough detail to recognize people and objects. Reversing a PhotoDNA hash is no more complicated than solving a 26x26 Sudoku puzzle; a task well-suited for computers.
https://www.hackerfactor.com/blog/index.php?/archives/929-On...
> Microsoft says that the "PhotoDNA hash is not reversible". That's not true. PhotoDNA hashes can be projected into a 26x26 grayscale image that is only a little blurry. 26x26 is larger than most desktop icons; it's enough detail to recognize people and objects. Reversing a PhotoDNA hash is no more complicated than solving a 26x26 Sudoku puzzle; a task well-suited for computers.
https://www.hackerfactor.com/blog/index.php?/archives/929-On...
PhotoDNA and NeuralHash are not the same thing.
Apple is using a private perceptual hash on the backend, likely to further filter out bad/spam tickets.
https://twitter.com/fayfiftynine/status/1427899951120490497?...
Given neuralhash is a hash of a hash, I imagine they’re running photodna and not some custom solution which would require Apple ingesting and hasing all of the images themselves using another custom perceptual hash system.
> . Instead of scanning images in the cloud, the system performs on-device matching using a database of known CSAM image hashes provided by NCMEC and other child-safety organizations. Apple further transforms this database into an unreadable set of hashes, which is securely stored on users’ devices.
https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
https://twitter.com/fayfiftynine/status/1427899951120490497?...
Given neuralhash is a hash of a hash, I imagine they’re running photodna and not some custom solution which would require Apple ingesting and hasing all of the images themselves using another custom perceptual hash system.
> . Instead of scanning images in the cloud, the system performs on-device matching using a database of known CSAM image hashes provided by NCMEC and other child-safety organizations. Apple further transforms this database into an unreadable set of hashes, which is securely stored on users’ devices.
https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
My reading of that is NCMEC provides NeuralHash hashes of their CSAM library to Apple, and Apple encrypts and blinds that database before storing it on devices.
Edit: "The main purpose of the hash is to ensure that identical and visually similar images result in the same hash, and images that are different from one another result in different hashes."[1]
Apple isn't using a "similar image, similar hash" system. They're using a "similar image, same hash" system.
[1]: https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
Apple isn't using a "similar image, similar hash" system. They're using a "similar image, same hash" system.
[1]: https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
> There really is no sound concept of "the more similar the hash."
Perceptual hashes are not cryptographic hashes. Perceptual hashing systems do compare hashes using a distance metric like the Hamming distance.
If two images have similar hashes, then they look kind of similar to one another. That's the point of perceptual hashing.
Perceptual hashes are not cryptographic hashes. Perceptual hashing systems do compare hashes using a distance metric like the Hamming distance.
If two images have similar hashes, then they look kind of similar to one another. That's the point of perceptual hashing.
This isn’t what Apple is doing, regardless of the fact they are using the same terminology. See my edit above.
[deleted]
Except when you're not measuring the similarity you think you are, because NeuralNets are f&#@ing weird.
You know, let me put it this way. Yiu know that one really weird family member I'm pretty sure everyone either has or is?
Guess what? They're a neural net too.
This is what Apple is asking you to trust.
You know, let me put it this way. Yiu know that one really weird family member I'm pretty sure everyone either has or is?
Guess what? They're a neural net too.
This is what Apple is asking you to trust.
I agree, I'm just talking about the difference between cryptographic hashes and other types of hashes.
EDIT: Parent edited his comment to clarify. I understand the point now. I'm wrong about similar images needing to have "similar" hashes. Those hashes either need to match exactly, or else not be considered at all.
IGNORE THIS: I think that's the parent comment's point. These are definitely not cryptographic hashes, since they—by design and necessity—need to mirror hash similarity to the perceptual similarity of the input images.
IGNORE THIS: I think that's the parent comment's point. These are definitely not cryptographic hashes, since they—by design and necessity—need to mirror hash similarity to the perceptual similarity of the input images.
They are almost the opposite. The programs create different hashes (though very similar) depending on platform you run on even with SAME input. No crypto hash works even close to this.
[deleted]
[deleted]
I created a proof of concept showing how OpenAI's CLIP model can function as a "sanity check" similar to how Apple says their server-side model works.
In order for a collision to get through to the human checkers, the same image would have to fool both networks independently:
https://blog.roboflow.com/apples-csam-neuralhash-collision/
In order for a collision to get through to the human checkers, the same image would have to fool both networks independently:
https://blog.roboflow.com/apples-csam-neuralhash-collision/
Cool project. Wouldn't padding all generated images with a few items that match CLIP get around this though?
Doing that would mean the NeuralHash would change though. And you'd have to not only get CLIP to identify CSAM in the generated image but also negate the parts of the generated image that are causing CLIP to label it as "generated" (while still colliding with the target NeuralHash).
Unclear how hard this would actually be in practice (if I were going to attempt it, the first thing I'd try is to evolve a colliding image with something like CLIP+VQGAN) but certainly harder than finding a collision alone.
Unclear how hard this would actually be in practice (if I were going to attempt it, the first thing I'd try is to evolve a colliding image with something like CLIP+VQGAN) but certainly harder than finding a collision alone.
Take 2 CSAM pictures, known. Combine into one image.
Swing and a miss. Not in the CSAM dataset. Take two images. Encode alternating pixels. Decode to get the original image back. Convert to different encodings print to PDF or Postscript. Encode as base64 representations of the image file...
Who are we trying to fool here? This is kiddie stuff.
This is more about trying to implant scanning capabilities on client devices.
Swing and a miss. Not in the CSAM dataset. Take two images. Encode alternating pixels. Decode to get the original image back. Convert to different encodings print to PDF or Postscript. Encode as base64 representations of the image file...
Who are we trying to fool here? This is kiddie stuff.
This is more about trying to implant scanning capabilities on client devices.
Those are examples of triggering false negatives (CSAM that gets missed by the system), this collision attack is about triggering false positives (non-CSAM that triggers human intervention).
> This is more about trying to implant scanning capabilities on client devices.
Did we think they didn't already have that ability?
Did we think they didn't already have that ability?
No. The brazen part of this is Apple trying to sell the public on this being okay, effective to the point of mitigating the abuse potential. False negative AND false positive have to counterweight the existence of the vector for abuse. That's what you're measuring against. You can't even guarantee that. This isn't even close to a safe or effective move. The risk management on this is hosed.
it's kind of silly how this is treated like some deeply technical issue. Apple's 'one of a trillion' claim is either true and the software is useless, because I'm pretty sure pedophiles can figure out what a watermark or a gaussian blur in paint net is, or it's imprecise and actually detects things which is the very thing that makes it dangerous to the public.
It's a direct trade-off and the error tolerance of any such filter is the only thing that makes it useful so we can basically stop arguing about the depths of implementation details or how high the collision rate of the hashing algorithm is etc. If this thing is supposed to catch anyone it needs to be magnitudes more lenient than any of those minor faults.
It's a direct trade-off and the error tolerance of any such filter is the only thing that makes it useful so we can basically stop arguing about the depths of implementation details or how high the collision rate of the hashing algorithm is etc. If this thing is supposed to catch anyone it needs to be magnitudes more lenient than any of those minor faults.
> I'm pretty sure pedophiles can figure out what a watermark or a gaussian blur in paint net is, or it's imprecise and actually detects things which is the very thing that makes it dangerous to the public
Or better yet, they can just not store their stuff on an iPhone. While meanwhile, millions of innocent people are have their photos scanned and risking being reported as a pedophile.
Or better yet, they can just not store their stuff on an iPhone. While meanwhile, millions of innocent people are have their photos scanned and risking being reported as a pedophile.
Exactly right. The tech Apple uses can be one of two things:
1. It requires a perfect 1:1 match (their documentation says this is not the case); 2. Or it has some freedom in detecting a match, probably including a match with a certain percentage.
If it's the former, it's completely useless. A watermark or a randomly chosen pixel with a slightly different hue and the hash would be completely different.
So, it's not #1. It's going to be #2. And that's where it becomes dangerous. The government of the USA is going to look for child predators. The government of Saudi Arabia is going to track down known memes shared by atheists, and they will be put to death; heresy is a capital offence over there. And China will probably do their best to track down Uyghurs so they can make the process of elimination even easier.
It's not like Apple hasn't given in to dictatorships in the past. This tech is absolutely going to kill people.
1. It requires a perfect 1:1 match (their documentation says this is not the case); 2. Or it has some freedom in detecting a match, probably including a match with a certain percentage.
If it's the former, it's completely useless. A watermark or a randomly chosen pixel with a slightly different hue and the hash would be completely different.
So, it's not #1. It's going to be #2. And that's where it becomes dangerous. The government of the USA is going to look for child predators. The government of Saudi Arabia is going to track down known memes shared by atheists, and they will be put to death; heresy is a capital offence over there. And China will probably do their best to track down Uyghurs so they can make the process of elimination even easier.
It's not like Apple hasn't given in to dictatorships in the past. This tech is absolutely going to kill people.
What? Why would memes be on the CSAM list?
They got to scan for whatever the local law requires them to scan.
How long until thishashcollisionisnotporn.com is a thing?
I can confirm it was a fun weekend project and
https://thishashcollisionisnotporn.com/ has now become reality!
Sounds like a fun weekend project.
sounds like a future NFT market
Sounds like hipster T-shirts with hash collision images.
I can confirm it was a fun weekend project and
https://thishashcollisionisnotporn.com/ has now become reality! I included a button to buy a hipster T-shirt ;)
>… Apple also said that after a user passes the 30 match threshold, a second non-public algorithm that runs on Apple's servers will check the results.
>"This independent hash is chosen to reject the unlikely possibility that the match threshold was exceeded due to non-CSAM images that were adversarially perturbed to cause false NeuralHash matches against the on-device encrypted CSAM database," …
>"This independent hash is chosen to reject the unlikely possibility that the match threshold was exceeded due to non-CSAM images that were adversarially perturbed to cause false NeuralHash matches against the on-device encrypted CSAM database," …
Apple wants to enforce one of their corporate policies at my expense: battery, CPU, and unknown risks if/when they're wrong. What could possibly go wrong?
I have to say that I’m just sad about this. I was really convinced that Apple really believed in privacy and they will never do this (having a framework which can scan your device content). But maybe they already had it and this was like: “there is this delta and Afghanistan let’s make announcement now”
We are just one stupid terrorist attack from full surveillance of everybody.
We are just one stupid terrorist attack from full surveillance of everybody.
They are slowly turning what is a philosophical argument into a technical question. Next step is for them to conjure up a technical answer before claiming victory. Most people are already bored by all the technical talk.
Neural hash this: Its about Trust. Its about Privacy. Its about Boundaries between me and corporations/governments/etc.
Neural hash this: Its about Trust. Its about Privacy. Its about Boundaries between me and corporations/governments/etc.
For everyone upset about Apple's CSAM scanning, I think we all forgot about the EARN IT Act. It was nearly passed last year but Congress was finished before it could be voted on. It had Bipartisan support and would've virtually banned E2E of any kind. And it would have required scanning everywhere according to the recommendations of a 19-member board of NGOs and unelected experts. The reason for this mandatory backdoor was child safety... even though AG Barr admitted that reducing the use of encryption had useful side effects.
If you are Apple, even though EARN IT failed... you know where Washington's heart lies. Is CSAM scanning a "better alternative", a concession, an appeasement, a lesser evil, in the hope this prevents EARN IT from coming back?
Also, many people forgot about the Lawful Access to Encrypted Data Act of 2020, or LAED, which would unilaterally banned E2E encryption in entirety and required that all devices featuring encryption must be unlockable by the manufacturer. That also was on the table.
If you are Apple, even though EARN IT failed... you know where Washington's heart lies. Is CSAM scanning a "better alternative", a concession, an appeasement, a lesser evil, in the hope this prevents EARN IT from coming back?
Also, many people forgot about the Lawful Access to Encrypted Data Act of 2020, or LAED, which would unilaterally banned E2E encryption in entirety and required that all devices featuring encryption must be unlockable by the manufacturer. That also was on the table.
Why can’t we have the alternative we already have, where we don’t have our phones scans and E2EE exists without compromise? Why do we have to accept it any other way?
If you’re trying to frame this as “we need to prevent Congress from ever passing something like the EARN IT act”, I agree. Apple and other tech companies already lobby Congress. Why aren’t they lobbying for encryption?
If you’re trying to frame this as “we need to prevent Congress from ever passing something like the EARN IT act”, I agree. Apple and other tech companies already lobby Congress. Why aren’t they lobbying for encryption?
They did, they lobbied heavily against EARN IT, and so did groups like the ACLU and EFF. However, it didn't stop Congress members from moving it along through the process - they were only stopped because Congress just ran out of time. It was clear that Congress was not interested in listening at all to the lobbyists on the issue.
It's clear that EARN IT could literally be revived any day if Apple didn't do something to say "we don't need it because we've already satisfied your requirements."
It's clear that EARN IT could literally be revived any day if Apple didn't do something to say "we don't need it because we've already satisfied your requirements."
It's clear that EARN IT could literally be revived any day if Apple didn't do something to say "we don't need it because we've already satisfied your requirements."
Alternatively, "Apple has shown that it's possible to do without undue hardship, so we should make everyone else do it too".
Alternatively, "Apple has shown that it's possible to do without undue hardship, so we should make everyone else do it too".
Congress doesn't give a rip about "undue hardship." Otherwise why would we pay income tax?
They were going to legally mandate that everything be scanned through methods less private than the ones Apple has developed here, through EARN IT and potentially LAED (which would have banned E2E in all circumstances and any device that could not be unlocked by the manufacturer). While that crisis was temporarily averted, the risk of it coming back was and is very real.
Apple decided to get ahead of it with a better solution, even though that solution is still bad. It's a lesser evil to prevent the return of something worse.
They were going to legally mandate that everything be scanned through methods less private than the ones Apple has developed here, through EARN IT and potentially LAED (which would have banned E2E in all circumstances and any device that could not be unlocked by the manufacturer). While that crisis was temporarily averted, the risk of it coming back was and is very real.
Apple decided to get ahead of it with a better solution, even though that solution is still bad. It's a lesser evil to prevent the return of something worse.
While I broadly agree that this (and other attempts in Australia and rumblings about it in UK) may have been the impetus for developing the technology, I'm not sure it explains why they released it on Photos, where Apple holds the keys, and not very far since you can look at your entire photo roll on icloud.com with just your username and password.
That tech is not being deployed on iMessage which is the only e2ee(ish) service from Apple (with Keychain) and is what those legislative attempts are usually targeting. One could argue it would have made sense (technically) there though, sure.
Was it a reason to release it preventively, on something unrelated, to be in the good graces of legislators ? I'm not sure it's a good calculation, and it doesn't cover other platforms like Signal and Telegram that would still be seen as a problem by those legislators and require them to legislate anyway.
That tech is not being deployed on iMessage which is the only e2ee(ish) service from Apple (with Keychain) and is what those legislative attempts are usually targeting. One could argue it would have made sense (technically) there though, sure.
Was it a reason to release it preventively, on something unrelated, to be in the good graces of legislators ? I'm not sure it's a good calculation, and it doesn't cover other platforms like Signal and Telegram that would still be seen as a problem by those legislators and require them to legislate anyway.
This seems like speculation with no evidence. The government cares about more than just CSAM, they care about terrorism, human and drug trafficking, organized crime, gangs, fraud, drug manufacturing etc.
This would only make sense if Apple intends to expand their CSAM detection and reporting system to detect and report those other things, as well.
This would only make sense if Apple intends to expand their CSAM detection and reporting system to detect and report those other things, as well.
The EARN IT Act would have basically legally mandated a backdoor in all services, with shifting recommendations, in the name of preventing the spread of CSAM. Sound familiar?
Also, there is another reason why there is the CSAM Detecting and Reporting system. With Apple CSAM Scan, that big "excuse" Congress was planning to use through EARN IT to ban E2E is diffused, meaning now Apple has the potential to add E2E to their iCloud service before Congress can figure out a different excuse.
Also, there is another reason why there is the CSAM Detecting and Reporting system. With Apple CSAM Scan, that big "excuse" Congress was planning to use through EARN IT to ban E2E is diffused, meaning now Apple has the potential to add E2E to their iCloud service before Congress can figure out a different excuse.
CSAM scanning is only one type of scanning which is performed at the behest of governments and corporations around the world. This alone will not allow for end-to-end encryption of items in the cloud.
There would need to be end-device scanning for arbitrary objects, including full text search for strings including 'Taiwan', 'Tiananmen Square', '09 F9', and so forth to even begin looking at e2e encryption of your items in the cloud.
At which point… what's the point?
There would need to be end-device scanning for arbitrary objects, including full text search for strings including 'Taiwan', 'Tiananmen Square', '09 F9', and so forth to even begin looking at e2e encryption of your items in the cloud.
At which point… what's the point?
Not really. E2E iMessage is still available in China if you disable the iCloud Backup. It's also the only messenger in China with an E2E option.
>which would unilaterally banned E2E encryption in entirety
It did not do this. The bill was essentially asking for search warrants to become a part of the protocol. If you're only solution to allowing for search warrants to work is to stop encrypting data I feel you are intentionally ignoring other options to make this seem worse than it is.
It did not do this. The bill was essentially asking for search warrants to become a part of the protocol. If you're only solution to allowing for search warrants to work is to stop encrypting data I feel you are intentionally ignoring other options to make this seem worse than it is.
I am very intrigued about what you think the other options are that preserve E2E encryption. How do you make search warrants part of the process without some kind of escrow or third party involvement, at which point it is no longer "end to end"?
You can't ban encryption, it's practically impossible, it's like banning math.
You can ban it's use, you can't ban its existence. You can't mandate math work differently, you just mandate people not use math.
Its amazing since this would have decimated the American Tech sector in many unknown ways.
Its amazing since this would have decimated the American Tech sector in many unknown ways.
All they have to do is force Apple and Google to ban it and any apps that use it from their app store and they've effectively banned encryption for like 99% of people.
The ban won't make encryption disappear. You can still have it. But if you're found using it, it's a felony.
Well, we are already banning numbers (see: DeCSS), what’s the big difference?
In some countries even discussing the application of certain numbers is unlawful.
In some countries even discussing the application of certain numbers is unlawful.
If you can get jailed for speech, you can get jailed for encryption.
Technically in the real world you can get jailed for anything as long as corruption exists and humans remain imperfect.
But you shouldn’t get jailed for protected speech and you shouldn’t get jailed for preserving your privacy (via encryption or otherwise.) As cynical as people may get, this is one thing that we have to agree on if we want to live in a free society.
And above all, most certainly, we shouldn’t allow being jailed over encryption to become codified as law, and if it does, we certainly must fight it and not become complacent.
Apathy over politics, especially these days, is understandable with the flood of terrible news and highly divisive topics, but we shouldn’t let the fight for privacy become a victim to apathy. (And yes, I realize big tech surveillance creep is a fear, but IMO we’re starting to get into more direct worst cases now.)
But you shouldn’t get jailed for protected speech and you shouldn’t get jailed for preserving your privacy (via encryption or otherwise.) As cynical as people may get, this is one thing that we have to agree on if we want to live in a free society.
And above all, most certainly, we shouldn’t allow being jailed over encryption to become codified as law, and if it does, we certainly must fight it and not become complacent.
Apathy over politics, especially these days, is understandable with the flood of terrible news and highly divisive topics, but we shouldn’t let the fight for privacy become a victim to apathy. (And yes, I realize big tech surveillance creep is a fear, but IMO we’re starting to get into more direct worst cases now.)
> If you can get jailed for speech, you can get jailed for encryption.
Not in any democratic country.
How can you prove that I'm using encryption? How you even define what encryption is to make it possible to have a law that bans it? If me and you we put down a code to communicate, is that encryption and thus banned?
Also you wouldn't need to ban encryption completely. There are cases where encryption is required by law (e.g. digital signature on documents) and others where the encryption is required for things to work (e.g. credit cards, mobile phones, etc).
You have to put down a list of cases where you can use encryption and others where you can't. That is nearly impossible.
Not in any democratic country.
How can you prove that I'm using encryption? How you even define what encryption is to make it possible to have a law that bans it? If me and you we put down a code to communicate, is that encryption and thus banned?
Also you wouldn't need to ban encryption completely. There are cases where encryption is required by law (e.g. digital signature on documents) and others where the encryption is required for things to work (e.g. credit cards, mobile phones, etc).
You have to put down a list of cases where you can use encryption and others where you can't. That is nearly impossible.
in a loose sense, mathematics, and its associated notations are a form of encryption
> If you are Apple, even though EARN IT failed... you know where Washington's heart lies.
If you are IG Farben, you know where Berlin's heart lies...
If you are IG Farben, you know where Berlin's heart lies...
Perhaps I have I missed this in all the discussions of Apple's latest move but has anyone considered the following questions.
Does Apple's solution only stop people from uploading illegal files to Apple's servers or does it stop them from uploading the files to any server.
If Apple intends to control the operation of a computer purchased from Apple after the owner begins using it, does Apple have a duty to report illegal files found on that computer and stop them from being shared (anywhere, not just through Apple's datacenters).
To me, this is why there is a serious distinction between a company detecting and policing what files are stored on their computers (i.e., how other companies approach this problem) and a company detecting and policing what files someone else's computer is storing and can transfer over the internet (in this case, unless I am mistaken, only to Apple's computers).
Mind you, I am not familiar with the details of exactly how Apple's solution works nor the applicable criminal laws so these questions might be irrelevant. However I was thinking that if Apple really wanted to prevent the trafficking of ostensibly illegal files then wouldn't Apple seek to prevent their transfer not only to Apple's computers but to any computer (and also report them to the proper authorities). What duty does Apple have if they can "see into the owner's computer" and they detect illegal activity. If Apple is in remote control of the computer, e.g., they can detect the presence/absence of files remotely and allow or disallow full user control through the OS, then does Apple have a duty to take action.
Does Apple's solution only stop people from uploading illegal files to Apple's servers or does it stop them from uploading the files to any server.
If Apple intends to control the operation of a computer purchased from Apple after the owner begins using it, does Apple have a duty to report illegal files found on that computer and stop them from being shared (anywhere, not just through Apple's datacenters).
To me, this is why there is a serious distinction between a company detecting and policing what files are stored on their computers (i.e., how other companies approach this problem) and a company detecting and policing what files someone else's computer is storing and can transfer over the internet (in this case, unless I am mistaken, only to Apple's computers).
Mind you, I am not familiar with the details of exactly how Apple's solution works nor the applicable criminal laws so these questions might be irrelevant. However I was thinking that if Apple really wanted to prevent the trafficking of ostensibly illegal files then wouldn't Apple seek to prevent their transfer not only to Apple's computers but to any computer (and also report them to the proper authorities). What duty does Apple have if they can "see into the owner's computer" and they detect illegal activity. If Apple is in remote control of the computer, e.g., they can detect the presence/absence of files remotely and allow or disallow full user control through the OS, then does Apple have a duty to take action.
> Does Apple's solution only stop people from uploading illegal images to Apple's servers or does it stop them from uploading the images to any server.
Only applies to iCloud Photos uploads, but the photos are still uploaded: when there's a match, the photo and a 'ticket' are uploaded and Apple's servers (after the servers themselves verify the match[0]) send the image to human reviewers to verify the CSAM before submitting it to police as evidence.
0: https://twitter.com/fayfiftynine/status/1427899951120490497 and https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
Only applies to iCloud Photos uploads, but the photos are still uploaded: when there's a match, the photo and a 'ticket' are uploaded and Apple's servers (after the servers themselves verify the match[0]) send the image to human reviewers to verify the CSAM before submitting it to police as evidence.
0: https://twitter.com/fayfiftynine/status/1427899951120490497 and https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
> Does Apple's solution only stop people from uploading illegal files to Apple's servers or does it stop them from uploading the files to any server.
Perhaps they're trying to prevent being in "posession" of illegal images by having them on their own servers, rather than preventing copying to arbitrary destinations.
Perhaps they're trying to prevent being in "posession" of illegal images by having them on their own servers, rather than preventing copying to arbitrary destinations.
> does Apple have a duty to report illegal images found on that computer and stop them from being shared
Duty? No, that's the secondary question. The primary question is whether they have the right.
Duty? No, that's the secondary question. The primary question is whether they have the right.
What has been stated is that this all _only_ happens as photos are about to be uploaded to iCloud photos (not to be confused with iCloud backup, a totally separate thing). This is currently done server-side on iCloud photoes. This appears to move it client side.
Am I the only one who finds no issue with this? Personally I’d prefer this than no encryption and scanning in the cloud, which is already possible.
All of the slippery slope arguments have already been possible for nearly a decade now with the cloud.
Can someone illustrate something wrong with this that’s not already possible today.
Fundamentally unless you audited the client and the server yourself either (client or backend) scanning is possible, and therefore this “problem”.
Where’s the issue?
All of the slippery slope arguments have already been possible for nearly a decade now with the cloud.
Can someone illustrate something wrong with this that’s not already possible today.
Fundamentally unless you audited the client and the server yourself either (client or backend) scanning is possible, and therefore this “problem”.
Where’s the issue?
> All of the slippery slope arguments have already been possible for nearly a decade now with the cloud.
That doesn't make it ok. The fact that you and my mom have normalized bad behavior doesn't make it any less offensive to my civil liberties.
> Am I the only one who finds no issue with this?
If you only think about the first order effects. This is great, we catch a bunch of kid sex pedos. The second order effects are much more dire. "slippery slope" sure... but false arrests, lives ruined based on mere investigations revealing nothing, expanding the role of government peering in to our personal lives, resulting suicides, corporations launching these programs - then defending them - then expanding them due to government pressure is *guaranteed*, additional PR and propaganda from corporations and government for further invasion in our personal lives due to the marketed/claimed success of these programs. The FBI has been putting massive pressure on Apple for years, due to the dominance and security measures of iOS.
Say what you want about the death penalty, but many many innocent people have dead in a truly horrific way, with some actually being tortured while being executed. That is a perfect example of second order effects on something most of us without any further information (ending murderous villains is a good thing) would agree on. So many Death Row inmates have been exonerated and vindicated.
edit: https://en.wikipedia.org/wiki/List_of_exonerated_death_row_i...
That doesn't make it ok. The fact that you and my mom have normalized bad behavior doesn't make it any less offensive to my civil liberties.
> Am I the only one who finds no issue with this?
If you only think about the first order effects. This is great, we catch a bunch of kid sex pedos. The second order effects are much more dire. "slippery slope" sure... but false arrests, lives ruined based on mere investigations revealing nothing, expanding the role of government peering in to our personal lives, resulting suicides, corporations launching these programs - then defending them - then expanding them due to government pressure is *guaranteed*, additional PR and propaganda from corporations and government for further invasion in our personal lives due to the marketed/claimed success of these programs. The FBI has been putting massive pressure on Apple for years, due to the dominance and security measures of iOS.
Say what you want about the death penalty, but many many innocent people have dead in a truly horrific way, with some actually being tortured while being executed. That is a perfect example of second order effects on something most of us without any further information (ending murderous villains is a good thing) would agree on. So many Death Row inmates have been exonerated and vindicated.
edit: https://en.wikipedia.org/wiki/List_of_exonerated_death_row_i...
> All of the slippery slope arguments have already been possible for nearly a decade now with the cloud.
Not only has it been possible for a decade, it’s been happening for a decade. Every major social media company already scans for and reports child pornography to the feds. Facebook submits millions of reports per year.
Not only has it been possible for a decade, it’s been happening for a decade. Every major social media company already scans for and reports child pornography to the feds. Facebook submits millions of reports per year.
I really don’t understand how you can, with sound mind, compare a public facing ‘social media’ with my very personal and very private phone photos. These are two completely different things.
They are not scanning photos stored on your phone, their are scanning photos stored on iCloud
My iCloud is not accessible to all my friends, family, work associates, and the public. Facebook/Twitter/insta is. (Depending on your privacy settings)
Scanning publicly accessible material makes a lot of sense and is within the boundaries of section 230, tech companies are responsible for policing removing illegal or threatening material. For private or encrypted information, section 230 does not apply.
The purposes these services were designed for matters. iCloud is meant to be private, facebook would prefer everything you post is public as it’s built in to benefit their business model. Facebook should scan, iCloud shouldn’t. You are purposely obfuscating that detail.
Scanning publicly accessible material makes a lot of sense and is within the boundaries of section 230, tech companies are responsible for policing removing illegal or threatening material. For private or encrypted information, section 230 does not apply.
The purposes these services were designed for matters. iCloud is meant to be private, facebook would prefer everything you post is public as it’s built in to benefit their business model. Facebook should scan, iCloud shouldn’t. You are purposely obfuscating that detail.
Yes, exactly. This is honestly nothing new.
This is new for Apple's customers. And its new in that the device you bought and paid for is nosing through your files.
Apple is introducing a reverse 'Little Snitch' where instead of the app warning you what apps are doing on the network, the OS is scanning your photos. Introducing a 5th columnist into a device that you've bought and paid for is a huge philosophical jump from Apple's previous stances on privacy, where they'd gone as far as fighting the Feds about trying to break into terrorist's iPhones.
Apple is introducing a reverse 'Little Snitch' where instead of the app warning you what apps are doing on the network, the OS is scanning your photos. Introducing a 5th columnist into a device that you've bought and paid for is a huge philosophical jump from Apple's previous stances on privacy, where they'd gone as far as fighting the Feds about trying to break into terrorist's iPhones.
Luckily for their customers it can be turned off. So, what’s the issue?
A huge part of Apple’s value proposition is iCloud. If I have to turn that off to keep the spy out of my OS, it’s value to me is dramatically diminished.
Since you presumably don’t trust Apple to scan your photos, it sounds like Apple might not be for you, then. Who will you move to?
Motte and Bailey much?
"Well you can turn it off" to "Well, but then you just don't trust Apple".
Clearly these are users who did trust Apple. Apple betrayed their trust. Given that Apple bulk handed over iCloud data to China, I don't really believe their pinky promise that they are, by policy only, going to resist government use of this tech. They can cave to government cases _and_ the government can certainly force them to.
"Well you can turn it off" to "Well, but then you just don't trust Apple".
Clearly these are users who did trust Apple. Apple betrayed their trust. Given that Apple bulk handed over iCloud data to China, I don't really believe their pinky promise that they are, by policy only, going to resist government use of this tech. They can cave to government cases _and_ the government can certainly force them to.
The point is that turning it off resolves the issue, but if someone refuses to turn it off because they want to use iCloud there’s clearly a contradiction.
Therefore you must trust Apple. So if you still have issues then you don’t trust Apple.
Therefore you must trust Apple. So if you still have issues then you don’t trust Apple.
In the past I trusted Apple to resist government efforts to spy on me to the best the law allowed. Now they are innovating ways to help a government spy literally live in my pocket.
Trust is fragile and Apple has taken what in the past I believed it understood to be a strategic advantage and stomped it into little pieces.
Trust is fragile and Apple has taken what in the past I believed it understood to be a strategic advantage and stomped it into little pieces.
Why prefer your owned device tattling on you to Apple looking at data you give them on their servers?
The reason people don't like this, as opposed to, for example, Dropbox scanning your synced files on their servers, is that a compute tool you ostensibly own is now turned completely against you. Today, that is for CSAM, tomorrow, what else?
The reason people don't like this, as opposed to, for example, Dropbox scanning your synced files on their servers, is that a compute tool you ostensibly own is now turned completely against you. Today, that is for CSAM, tomorrow, what else?
There’s no difference - all cloud services that aren’t encrypting your data is subject to the same thing. Dropbox could do the same thing tomorrow. If we’re talking about hypotheticals any vendor that handles your unencrypted files can do this now.
You’re not understanding. Dropbox already does scan your files and I’m comparing Apples actions explicitly to that fact. I’m pointing out that people don’t care about that because you’re literally, voluntarily, giving Dropbox your files. Here, Apple is controlling your phone to tattle on you at an OS level (yes, I know, Apple says you need to turn on iCloud Photos for this to run but the precedent is the problem).
> Dropbox already does scan your files
> people don’t care about that because you’re literally, voluntarily, giving Dropbox your files.
Correct and I agree. I don’t upload my most personal photos to Dropbox for this very specific reason. In fact I stopped using Dropbox when Condi Rice joined the board because she lacks good sense and doesn’t respect civil rights. See ‘The Patriot Act’ and ‘the Invasion of Afghanistan’. It was easy to stop using Dropbox because the alternatives were vast. Apple has me very purposely locked in to this scheme to where the alternatives are a huge transition and compromise on privacy no matter where I turn.
> people don’t care about that because you’re literally, voluntarily, giving Dropbox your files.
Correct and I agree. I don’t upload my most personal photos to Dropbox for this very specific reason. In fact I stopped using Dropbox when Condi Rice joined the board because she lacks good sense and doesn’t respect civil rights. See ‘The Patriot Act’ and ‘the Invasion of Afghanistan’. It was easy to stop using Dropbox because the alternatives were vast. Apple has me very purposely locked in to this scheme to where the alternatives are a huge transition and compromise on privacy no matter where I turn.
big difference: you're now paying for the compute load, instead of the cloud providers, for something that offers no direct benefit to you and will most likely betray you in unforeseen ways in the future (especially in less 'free' countries)
Why did you think you owned a closed source device?
Open Source fanatics are the worst. Even if all the software on your phone was Open Source, you wouldn’t have the mental bandwidth to ever verify you trust it. You’re just farming out your trust to a different set of people and hoping they’re more trustworthy than those with closed source. At best this MAY be reasonable because of incentive alignment but that’s super weak.
Not to mention the meaning of ownership is completely unrelated to ability to view the designs of a given object. I think I own the fan currently blowing air at me without ever having seen a schematic for its controls circuitry just fine and everyone for all of history has pretty much felt the same.
Not to mention the meaning of ownership is completely unrelated to ability to view the designs of a given object. I think I own the fan currently blowing air at me without ever having seen a schematic for its controls circuitry just fine and everyone for all of history has pretty much felt the same.
That's because your fan is hardware and your phone is software. The hardware in your phone is only 1% of what your phone is.
The point is it's hard to hide your evil plans in daylight. Either you trust Apple or your don't. Same with Microsoft's telemetry, they wrote the whole OS, if they were evil they have 10000 easier ways to do it.
All Apple has to do is lie, you'll never know.
The point is it's hard to hide your evil plans in daylight. Either you trust Apple or your don't. Same with Microsoft's telemetry, they wrote the whole OS, if they were evil they have 10000 easier ways to do it.
All Apple has to do is lie, you'll never know.
Personally, I'd prefer no privacy intrusions at all.
The issue is that Apple previously was not intruding into their user's privacy (at least publicly), but now they are.
It sounds like your argument is that Apple could have been doing this all along and just not telling us. I find that unlikely mainly because they've marketed themselves as a privacy-focused company up until now.
The issue is that Apple previously was not intruding into their user's privacy (at least publicly), but now they are.
It sounds like your argument is that Apple could have been doing this all along and just not telling us. I find that unlikely mainly because they've marketed themselves as a privacy-focused company up until now.
> The issue is that Apple previously was not intruding into their user's privacy
Apple reserves the right at all times to determine whether Content is appropriate and in compliance with this Agreement, and may screen, move, refuse, modify and/or remove Content at any time, without prior notice and in its sole discretion, if such Content is found to be in violation of this Agreement or is otherwise objectionable.
You must not have been paying attention for the last 20 years.
Apple reserves the right at all times to determine whether Content is appropriate and in compliance with this Agreement, and may screen, move, refuse, modify and/or remove Content at any time, without prior notice and in its sole discretion, if such Content is found to be in violation of this Agreement or is otherwise objectionable.
You must not have been paying attention for the last 20 years.
I’m your situation can’t you just turn off the scanning? What’s the issue?
There are two different "features" being implemented - Messages scanning for minors on a family plan (which can be turned off) and iCloud Photo scanning (which can't be turned off, as far as I know). So no, you can't just turn it off.
Yes, it can be turned off. https://www.macrumors.com/2021/08/05/apple-csam-detection-di...
Only if you disable iCloud photos completely. So no, you cannot turn off the feature unless you stop using iCloud photos.
Correct. I'm not sure how that's a distinction from "you can't turn it off" though. I believe they've been doing this scan server-side for quite some time already.
Here's the distinction: I used to be able to use iCloud photos without having my photos scanned, and now I can't. So I have to make a choice of either dropping iCloud photos completely or submit to having all of my photos scanned.
I don't think they have been doing server-side scanning until now, hence the publicity. Do you have any evidence that shows they've been doing this before?
I don't think they have been doing server-side scanning until now, hence the publicity. Do you have any evidence that shows they've been doing this before?
https://www.dailymail.co.uk/sciencetech/article-7865979/Appl...
I wasn’t able to find the whole video though.
No time to watch, but. https://www.ces.tech/Videos/2020/Chief-Privacy-Officer-Round...
I wasn’t able to find the whole video though.
No time to watch, but. https://www.ces.tech/Videos/2020/Chief-Privacy-Officer-Round...
The article you linked makes a surprising claim, and it contradicts what the EFF said recently about this here - https://www.eff.org/deeplinks/2021/08/apples-plan-think-diff...
> Currently, although Apple holds the keys to view Photos stored in iCloud Photos, it does not scan these images
It also seems weird that the EFF wouldn't have complained about this before if Apple was known to be doing server-side scanning for some time now.
I also am not going to watch through an hour video, but I scanned the transcript and I didn't see anything that said that Apple currently (at that time) scanned content.
> Currently, although Apple holds the keys to view Photos stored in iCloud Photos, it does not scan these images
It also seems weird that the EFF wouldn't have complained about this before if Apple was known to be doing server-side scanning for some time now.
I also am not going to watch through an hour video, but I scanned the transcript and I didn't see anything that said that Apple currently (at that time) scanned content.
I can’t find any articles about eff complaining about gmail, onedrive, or discord doing similar scanning either. All of those services (and more) do similar scans.
But you can tho…
Please show evidence where the feature can be turned off (without having to completely disable iCloud photos).
You turn it off by turning off iCloud photos, I never claimed otherwise.
If you don’t trust Apple why would you use iCloud anyway? Makes no sense.
If you don’t trust Apple why would you use iCloud anyway? Makes no sense.
> You turn it off by turning off iCloud photos, I never claimed otherwise.
I just have to point out the ridiculousness of this statement. With your logic any feature in any product can be "turned off" by not using the entire product at all. For example, the radio in my car sounds like shit, I guess I should just stop driving completely to avoid having to hear it.
In reality, this new "feature" will be a requirement of using iCloud Photos. The feature itself cannot be turned off. If your answer is to stop using iCloud Photos, that is no help for the millions of people who currently use iCloud Photos.
> If you don’t trust Apple why would you use iCloud anyway? Makes no sense.
I've trusted Apple for a long time because I felt like they were one of the only companies that cared about consumer privacy. After these actions I am less convinced of that. I'm not sure why that stance is so surprising.
I just have to point out the ridiculousness of this statement. With your logic any feature in any product can be "turned off" by not using the entire product at all. For example, the radio in my car sounds like shit, I guess I should just stop driving completely to avoid having to hear it.
In reality, this new "feature" will be a requirement of using iCloud Photos. The feature itself cannot be turned off. If your answer is to stop using iCloud Photos, that is no help for the millions of people who currently use iCloud Photos.
> If you don’t trust Apple why would you use iCloud anyway? Makes no sense.
I've trusted Apple for a long time because I felt like they were one of the only companies that cared about consumer privacy. After these actions I am less convinced of that. I'm not sure why that stance is so surprising.
> In reality, this new "feature" will be a requirement of using iCloud Photos. The feature itself cannot be turned off. If your answer is to stop using iCloud Photos, that is no help for the millions of people who currently use iCloud Photos.
iCloud Photos can be used on Windows, for example. This scanning only applies to iOS devices. You could use iCloud Photos and not be subject to the scanning.
> I've trusted Apple for a long time because I felt like they were one of the only companies that cared about consumer privacy. After these actions I am less convinced of that. I'm not sure why that stance is so surprising.
Nothing about what they're doing is contradictory with privacy beyond what they're already doing. The only reason they're even implementing it this way is because they do care about privacy. They could just not encrypt and scan on the server like Google, Microsoft, Dropbox, Box.com and more.
iCloud Photos can be used on Windows, for example. This scanning only applies to iOS devices. You could use iCloud Photos and not be subject to the scanning.
> I've trusted Apple for a long time because I felt like they were one of the only companies that cared about consumer privacy. After these actions I am less convinced of that. I'm not sure why that stance is so surprising.
Nothing about what they're doing is contradictory with privacy beyond what they're already doing. The only reason they're even implementing it this way is because they do care about privacy. They could just not encrypt and scan on the server like Google, Microsoft, Dropbox, Box.com and more.
> iCloud Photos can be used on Windows, for example. This scanning only applies to iOS devices. You could use iCloud Photos and not be subject to the scanning.
That's great for the <0.001% of people who use iCloud Photos without an iPhone. Everyone else is SOL.
>Nothing about what they're doing is contradictory with privacy beyond what they're already doing. The only reason they're even implementing it this way is because they do care about privacy. They could just not encrypt and scan on the server like Google, Microsoft, Dropbox, Box.com and more.
False dichotomy. Apple doesn't have to do either of these things.
That's great for the <0.001% of people who use iCloud Photos without an iPhone. Everyone else is SOL.
>Nothing about what they're doing is contradictory with privacy beyond what they're already doing. The only reason they're even implementing it this way is because they do care about privacy. They could just not encrypt and scan on the server like Google, Microsoft, Dropbox, Box.com and more.
False dichotomy. Apple doesn't have to do either of these things.
NCMEC is an arm of the government. Either they share voluntarily or they get subpoenaed endlessly. Why do you think all of these companies even do this? lol
in any case I've given you the solution on how to use iCloud and not be scanned. Take your photos, sync your iDevice to your computer and manually upload to iCloud photos. there you go.
in any case I've given you the solution on how to use iCloud and not be scanned. Take your photos, sync your iDevice to your computer and manually upload to iCloud photos. there you go.
> NCMEC is an arm of the government. Either they share voluntarily or they get subpoenaed endlessly.
A subpoena is a lot different than scanning every single photo on every user's device automatically.
> Why do you think all of these companies even do this? lol
Because most companies don't give a shit about privacy? And they will cave at even the slightest government pressure to do so. Honestly it's probably easier for them that way (but worse for the consumer).
> in any case I've given you the solution on how to use iCloud and not be scanned. Take your photos, sync your iDevice to your computer and manually upload to iCloud photos. there you go.
Gee thanks, your "solution" is 1000x harder to use than just using iCloud Photos on your iPhone. One of the biggest selling points of it now is for convenience, and this is pretty obvious so I'm starting to think you're just trolling at this point.
A subpoena is a lot different than scanning every single photo on every user's device automatically.
> Why do you think all of these companies even do this? lol
Because most companies don't give a shit about privacy? And they will cave at even the slightest government pressure to do so. Honestly it's probably easier for them that way (but worse for the consumer).
> in any case I've given you the solution on how to use iCloud and not be scanned. Take your photos, sync your iDevice to your computer and manually upload to iCloud photos. there you go.
Gee thanks, your "solution" is 1000x harder to use than just using iCloud Photos on your iPhone. One of the biggest selling points of it now is for convenience, and this is pretty obvious so I'm starting to think you're just trolling at this point.
> Personally I’d prefer this than no encryption and scanning in the cloud, which is already possible.
Why is that the alternative? How about everything is encrypted and nothing is scanned.
Why is that the alternative? How about everything is encrypted and nothing is scanned.
This is already possible if you self host your stuff. I’m talking about iCloud specifically here - backups are not encrypted so it’s either scan in the backend or scan on the client.
If you don’t want to be scanned you can turn it off. I honestly don’t see the issue. It seems the only thing people can say are hypothetical situations here.
If you don’t want to be scanned you can turn it off. I honestly don’t see the issue. It seems the only thing people can say are hypothetical situations here.
Realistically, because if they don't have something to scan for it, countries / EU are going to require them to provide a backdoor to scan it. There are already proposals in the EU to that affect.
>Personally I’d prefer this than no encryption and scanning in the cloud
How about neither? Just let people have their privacy. Some will misuse it. Thats life.
How about neither? Just let people have their privacy. Some will misuse it. Thats life.
> Personally I’d prefer this than no encryption and scanning in the cloud, which is already possible.
What is also possible: No scanning on your device and encrypted cloud storage. E.g. borg + rsync.net, mega, proton drive.
What is also possible: No scanning on your device and encrypted cloud storage. E.g. borg + rsync.net, mega, proton drive.
If it's in the cloud you can choose not to use the cloud service. If it's on device - a device you'e spent over $1000 on and already own - you don't have a choice (unless you want to forgo all updates, including security patches).
Never let a good Apple controversy go to waste. Clickbait blogs are thrilled to have something like this.
Honest question - as far as I could find out the files will be scanned on device but before upload to iCloud. Before they were scanned in the cloud after upload. That's the change as far as I understand it. But if it's scanned before upload then what is the difference? A few seconds? It would be scanned either way, before or after upload. Is that it?
What I'm basically getting at is: are the files scanned after the user has expressed the intention of uploading them? That's what I understood. Am I wrong? Are the files scanned the moment they appear on your device, regardless of you iCloud status (even if you have disabled iCloud somehow)?
Edit: typo
What I'm basically getting at is: are the files scanned after the user has expressed the intention of uploading them? That's what I understood. Am I wrong? Are the files scanned the moment they appear on your device, regardless of you iCloud status (even if you have disabled iCloud somehow)?
Edit: typo
With the rise of end to end encrypted data and messaging, such as signal and WhatsApp, it makes sense for governments to shift the search to user’s devices and operating systems rather than cloud, before stuff gets encrypted.
Would this work as an attack?
1. Get a pornographic picture involving young though legal actors and actresses.
2. Encode a nonce into the image. Hash it checking for CSAM collisions. If you've found a collision go on to the next step, if not update the nonce and try again.
3. You now have an image that, to visual inspection will appear plausibly like CSAM, and to automated detection will appear like CSAM. Though, presumably, it is not illegal for you to have this image as it is, in fact, legal pornography. You can now text this to anyone with an iPhone who will be referred by Apple to law enforcement.
1. Get a pornographic picture involving young though legal actors and actresses.
2. Encode a nonce into the image. Hash it checking for CSAM collisions. If you've found a collision go on to the next step, if not update the nonce and try again.
3. You now have an image that, to visual inspection will appear plausibly like CSAM, and to automated detection will appear like CSAM. Though, presumably, it is not illegal for you to have this image as it is, in fact, legal pornography. You can now text this to anyone with an iPhone who will be referred by Apple to law enforcement.
Your hash will match to say image 105 of the dataset. Upon visual inspection, your 'legal porn' is going to have to at least look passably like image 105 of the dataset to get anywhere.
So at this point we have an image that computers think is CSAM and people think is CSAM, and when held up next to the original verified horrific image everyone agrees is the same image. At this point, someone is going to ask, rightly so, where that came from.
In order to generate this attack, you have had to go out and procure, deliberately, known CSAM. Ignoring that it would be easier just to send that to the target, rather than hiring talent to recreate the pose of a specific piece of child porn (or 30 pieces to trigger the reporting levels), the most likely person by orders of magnitude to be prosecuted in this scenario is the attacker.
So at this point we have an image that computers think is CSAM and people think is CSAM, and when held up next to the original verified horrific image everyone agrees is the same image. At this point, someone is going to ask, rightly so, where that came from.
In order to generate this attack, you have had to go out and procure, deliberately, known CSAM. Ignoring that it would be easier just to send that to the target, rather than hiring talent to recreate the pose of a specific piece of child porn (or 30 pieces to trigger the reporting levels), the most likely person by orders of magnitude to be prosecuted in this scenario is the attacker.
> Upon visual inspection, your 'legal porn' is going to have to at least look passably like image 105 of the dataset to get anywhere.
Define "get anywhere". Why won't you get raided by the police and have all your devices seized first?
Define "get anywhere". Why won't you get raided by the police and have all your devices seized first?
No, probably wouldnt even get to the desk of the police. End of the road would be NCMEC, to whom apple would refer hash matching images that pass human verification that are close enough to porn.
If your 30 or so hash matching images matched their corresponding known CSAM then that goes on to the police and then they knock on your door.
If your 30 or so hash matching images matched their corresponding known CSAM then that goes on to the police and then they knock on your door.
> to whom apple would refer hash matching images that pass human verification
I am under the impression that Apple's scheme allows them only to verify the output of the matching algorithm (the "safety vouchers"), and not the image content itself. So in the hypothetical situation described in the thread, it won't be possible for Apple to detect the false positive, and they could pass on the report to NCMEC.
I fear that this will lead to a law enforcement raid without any actual human verification of the offending image itself.
If I'm wrong, I welcome citations which demonstrate the opposite.
I am under the impression that Apple's scheme allows them only to verify the output of the matching algorithm (the "safety vouchers"), and not the image content itself. So in the hypothetical situation described in the thread, it won't be possible for Apple to detect the false positive, and they could pass on the report to NCMEC.
I fear that this will lead to a law enforcement raid without any actual human verification of the offending image itself.
If I'm wrong, I welcome citations which demonstrate the opposite.
Happy to provide citation of how the apple scheme works [1].
You should also point out that the NCMEC themselves are not law enforcment.
[1] https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
You should also point out that the NCMEC themselves are not law enforcment.
[1] https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
Right, and what in there suggests to you that Apple can view an image after a sufficient number of images match? I see the opposite: "Apple can’t access metadata or visual derivatives for matched CSAM images until a
threshold of matches is exceeded for an iCloud Photos account."
That suggests that Apple can't access the actual matched CSAM images at all.
> You should also point out that the NCMEC themselves are not law enforcment.
I don't think the distinction is relevant. The point is that they will get passed on for enforcement purposes, and at that point, innocent parties will find themselves raided and their devices seized without any human actually manually verifying that it is CSAM first.
That suggests that Apple can't access the actual matched CSAM images at all.
> You should also point out that the NCMEC themselves are not law enforcment.
I don't think the distinction is relevant. The point is that they will get passed on for enforcement purposes, and at that point, innocent parties will find themselves raided and their devices seized without any human actually manually verifying that it is CSAM first.
If you're gonna go through that much effort, you might as well just text someone an actually illegal image from something other than an iPhone
You're already attempting to frame someone for a crime, might as well commit another crime while you're at it
You're already attempting to frame someone for a crime, might as well commit another crime while you're at it
How would you know if it's a CSAM collision? I don't believe that database is publicly available anywhere, for obvious reasons.
Good point. I can offer three possibilities to how you might know. First, a data leak of the material. Second, law enforcement presumably gives hashes of known CSAM to service operators so they can detect and report it. You could pose as or be the operator of such a service and get the hashes that way. Third, if you were a government operator you may have access to the hashes that way. (Although I guess corrupt government agents would have other easier ways of getting to you)
Yes. That, plus it's unclear whether you can create a collision with a nonce. It's a perceptual hash, not a cryptographic one. Lastly, to compute that collision might be so expensive that you could instead compute small SHA-256 hashes, ie mine BTC, and use the money to obtain your sinister goals via other means.
Apple's client clearly has some revealing information on that...
Why would you save to your personal photo library some porn that a creepy rando sent to you in a text?
It's clear now that Apple is never going to back down. It's bittersweet, I'm sad to see them go but also excited for what I think will be a FOSS renaissance.
> Anti-Child Abuse Imagery Tech
Room for improvement in the headline.
Room for improvement in the headline.
Something just occurred to me. If this goes forward, and then Microsoft and/or Google copy it, then we might see lawmakers expecting it. If that becomes the case, then it'd be yet another barrier to entry (in addition to a cop on every phone.)
Isn't that where we already with things like Article 17 of the EU's Copyright Directive?
Isn't that where we already with things like Article 17 of the EU's Copyright Directive?
This is already the world we live in, with vendors like Thorn building SaaS solutions for people who accept UGC but don’t have the clout to integrate with NCMEC and Microsoft PhotoDNA directly.
> Apple however told Motherboard in an email that that version analyzed by users on GitHub is a generic version, and not the one final version that will be used for iCloud Photos CSAM detection. Apple said that it also made the algorithm public.
When did they make NeuralHash public?
When did they make NeuralHash public?
Hashes can always collide, but how likely is it for neuralhash?
Oh man. They really want to die on this hill don't they.
Apple does check it just before you upload it to icloud. How much money do they save it's is checked clientside? How much would it cost for Apple to do it on there own servers, like everybody else does it?
Iirc the rationale for doing it clientside isn't saving money, but maintaining encryption. If it's checked client side, Apple should never get unencrypted uploads unless they're suspected to be CSAM
Yes thats the official version. I still wonder what would be the cost difference. In this scale it is not just for free.
”the analyzed code is not the final implementation that will be used with the CSAM system itself and is instead a generic version”
So they are already running a generic version of this system since iOS 14.3?
So they are already running a generic version of this system since iOS 14.3?
Erosion of privacy and freedom always starts with "but think of the children"
> [..] the overall system is designed to account for this to happen in general, and that the analyzed code is not the final implementation that will be used with the CSAM system itself and is instead a generic version.
I think the claim here is that you won't have access to the source images, and therefore generating collisions will be more difficult. But, if you do have access to the source images, this has been shown to be trivial. This of course doesn't stop nations states generating images that cause hash collisions, in fact they would be incentivized to do so.
I would also add that Apple are behind the curve, attempts to crack the hashing algorithm more efficiently are still ongoing: https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue...
> [..] not the final implementation [..]
Why on earth would you invite people to come and test your algorithm and then say "sure, you broke it, but it's not the real one". This kind of defeats the point and seems like some bait and switch bullshit. I suspect this is some retroactive cope from management realising they can't deploy this version and whatever they do deploy needs to be heavily modified.
> If Apple finds they are CSAM, it will report the user to law enforcement.
One statistic I want to know is: How many people already trigger this report function in the wild? Surely currently is the largest number of positives they will ever have - if it turns out to be 0% - Apple should just scrap it.
> Apple also said that after a user passes the 30 match threshold, a second non-public algorithm that runs on Apple's servers will check the results.
So to avoid reporting, simply block Apple servers? Also, security by obscurity is not security - the algorithm supposedly being private just means that its not properly tested and Apple is not held to account.
> "Apple actually designed this system so the hash function doesn't need to remain secret, as the only thing you can do with 'non-CSAM that hashes as CSAM' is annoy Apple's response team with some garbage images until they implement a filter to eliminate those garbage false positives in their analysis pipeline," Nicholas Weaver, senior researcher at the International Computer Science Institute at UC Berkeley, told Motherboard in an online chat.
No. A report could be considered 'reasonable doubt' for law enforcement to do a full search. Imagine trying to explain to a judge why your iPhone shouldn't be searched because of a false-positive CSAM hash collision because of a malicious website you visited or a text message you received.
I think the claim here is that you won't have access to the source images, and therefore generating collisions will be more difficult. But, if you do have access to the source images, this has been shown to be trivial. This of course doesn't stop nations states generating images that cause hash collisions, in fact they would be incentivized to do so.
I would also add that Apple are behind the curve, attempts to crack the hashing algorithm more efficiently are still ongoing: https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue...
> [..] not the final implementation [..]
Why on earth would you invite people to come and test your algorithm and then say "sure, you broke it, but it's not the real one". This kind of defeats the point and seems like some bait and switch bullshit. I suspect this is some retroactive cope from management realising they can't deploy this version and whatever they do deploy needs to be heavily modified.
> If Apple finds they are CSAM, it will report the user to law enforcement.
One statistic I want to know is: How many people already trigger this report function in the wild? Surely currently is the largest number of positives they will ever have - if it turns out to be 0% - Apple should just scrap it.
> Apple also said that after a user passes the 30 match threshold, a second non-public algorithm that runs on Apple's servers will check the results.
So to avoid reporting, simply block Apple servers? Also, security by obscurity is not security - the algorithm supposedly being private just means that its not properly tested and Apple is not held to account.
> "Apple actually designed this system so the hash function doesn't need to remain secret, as the only thing you can do with 'non-CSAM that hashes as CSAM' is annoy Apple's response team with some garbage images until they implement a filter to eliminate those garbage false positives in their analysis pipeline," Nicholas Weaver, senior researcher at the International Computer Science Institute at UC Berkeley, told Motherboard in an online chat.
No. A report could be considered 'reasonable doubt' for law enforcement to do a full search. Imagine trying to explain to a judge why your iPhone shouldn't be searched because of a false-positive CSAM hash collision because of a malicious website you visited or a text message you received.
They are down to hash collision arguments against this now?
Sounds pretty desperate.
As if any normal user is going to upload a photo to iCloud that is a collision.
The fact that such images are possible to generate means nothing by itself.
Also, they would need to accidentally have 30 of them.
Also, a human would have to not be able to tell the difference.
Sounds pretty desperate.
As if any normal user is going to upload a photo to iCloud that is a collision.
The fact that such images are possible to generate means nothing by itself.
Also, they would need to accidentally have 30 of them.
Also, a human would have to not be able to tell the difference.
Is no one going to talk about how Apple is implementing this?
If Apple is training a neural network to detect this kind of imagery, I would imagine there to be thousands, if not millions of child pornography images on Apple's servers that are being used by their own engineers to train this system
If Apple is training a neural network to detect this kind of imagery, I would imagine there to be thousands, if not millions of child pornography images on Apple's servers that are being used by their own engineers to train this system
It's not a neural network. It's not AI. It's not trying to interpret image content. It's trying to identify known specific images, but is using a fuzzy fingerprinting match.
> If Apple is training a neural network to detect this kind of imagery
NCMEC generate the hashes using their CSAM corpus.
NCMEC generate the hashes using their CSAM corpus.
I think we're up for a new Apple CEO in a few days.
The turmoil of Apple abruptly booting the CEO who's overseen the most obscenely prosperous stretch in its history would cost the company vastly more than the current controversy about CSAM.
I'm not saying this as a fan of either Cook or their anti-CSAM measures; I'm neither, and if anyone is ever wrongfully arrested because Apple's system made a mistake, Cook may well wind up in disgrace depending on how much blame he can/can't shuffle off to subordinates. I don't think we're there yet, though.
I'm not saying this as a fan of either Cook or their anti-CSAM measures; I'm neither, and if anyone is ever wrongfully arrested because Apple's system made a mistake, Cook may well wind up in disgrace depending on how much blame he can/can't shuffle off to subordinates. I don't think we're there yet, though.
He's not Ballmer but Ballmer also presided over record profits even though MS barely did anything remotely interesting under his watch.
Just saying that money hides problems.
Just saying that money hides problems.
Hash collision in Apple NeuralHash model - https://news.ycombinator.com/item?id=28219068 - Aug 2021 (542 comments)
Convert Apple NeuralHash model for CSAM Detection to ONNX - https://news.ycombinator.com/item?id=28218391 - Aug 2021 (155 comments)