Unmasking the Privacy Risks of Apple Intelligence(lumia.security)
lumia.security
Unmasking the Privacy Risks of Apple Intelligence
https://www.lumia.security/blog/applestorm
24 comments
They also end it with trying to sell their service around AI which further devalues it, and even trying to give it a name like "AppleStorm".
I think some of the points are valid, but I think the over emphasis on Siri vs Private Cloud is massively overblown. That to me is just the nature of a transition like this and eventually more if Siri will likely fall under "Apple Intelligence" since it makes sense that they would have a single platform on the backend.
Then there is this header:
> "End-to-End Encryption? I’m Not Sure"
Well.. it is still end to end encrypted. Nothing about using Siri to dictate it changes that since you know... your on one of the ends. It is like saying that me taking a screenshot of the conversation somehow broke E2E.
That isn't to say that the concern here is not valid, but there are so many examples of things being twisted and manipulated to get you to use their product that I have a hard time really understanding what is an issue and what isn't.
Like ok you made an app using SiriKit using Apple's recommended settings (which may be recommended for a reason). But do you have the ability to have them not go to apple's servers if you configure it a certain way... it seems the author just ended with "Well it happens when I made this app" and never looked further.
I think some of the points are valid, but I think the over emphasis on Siri vs Private Cloud is massively overblown. That to me is just the nature of a transition like this and eventually more if Siri will likely fall under "Apple Intelligence" since it makes sense that they would have a single platform on the backend.
Then there is this header:
> "End-to-End Encryption? I’m Not Sure"
Well.. it is still end to end encrypted. Nothing about using Siri to dictate it changes that since you know... your on one of the ends. It is like saying that me taking a screenshot of the conversation somehow broke E2E.
That isn't to say that the concern here is not valid, but there are so many examples of things being twisted and manipulated to get you to use their product that I have a hard time really understanding what is an issue and what isn't.
Like ok you made an app using SiriKit using Apple's recommended settings (which may be recommended for a reason). But do you have the ability to have them not go to apple's servers if you configure it a certain way... it seems the author just ended with "Well it happens when I made this app" and never looked further.
They are arguing in bad faith. They clearly know how to disable the relevant subset of these features. They don't do this upfront because they would have nothing to write about otherwise.
As a user, you can configure these settings in the UI. You can use the defaults command. They can be configured using a configuration profile/MDM. You could block the domains based on their associated feature, which are publicly documented by Apple. [1]
It's like complaining about Windows telemetry without bothering to configure the registry (or even open the settings menu).
[1] https://support.apple.com/en-us/101555
As a user, you can configure these settings in the UI. You can use the defaults command. They can be configured using a configuration profile/MDM. You could block the domains based on their associated feature, which are publicly documented by Apple. [1]
It's like complaining about Windows telemetry without bothering to configure the registry (or even open the settings menu).
[1] https://support.apple.com/en-us/101555
Smartphone OS manufactures like Apple and Google do not allow strong secure features to black domain or IP addresses. There are attempts at cheep hacks to use VN or accessibility work a rounds but they can be overwritten by the OS and they prevent use a firewall and VPN at the same time.
I have used encrypted DNS profiles on iOS to block them at the resolver level. However, the correct thing to do is to disable the feature in a configuration profile. You can also block them on macOS using Little Snitch or similar.
No, you sometimes can't use two apps on iOS that attempt to configure DNS and a "VPN" for local filtering purposes at the same time (the latter is often a glorified hosts list).
You absolutely can use encrypted DNS and/or a VPN (or Private Relay). None of these have bearing on using an application firewall or pf on macOS.
No, you sometimes can't use two apps on iOS that attempt to configure DNS and a "VPN" for local filtering purposes at the same time (the latter is often a glorified hosts list).
You absolutely can use encrypted DNS and/or a VPN (or Private Relay). None of these have bearing on using an application firewall or pf on macOS.
I think people are unaware of the difference between Apple Intelligence and Siri - they even have the same colour glow now. Also, can you always tell if it's Siri or Apple Intelligence handling a request?
The only privacy screen on macOS and iOS is during oob or after OS updates, and it does not make a distinction. As the OP post highlights, there is no way to avoid said telemetry from being sent or configure it in Settings. So all this is not only shady but quite illegal.
The only privacy screen on macOS and iOS is during oob or after OS updates, and it does not make a distinction. As the OP post highlights, there is no way to avoid said telemetry from being sent or configure it in Settings. So all this is not only shady but quite illegal.
Apple Intelligence currently handles no requests. ChatGPT handles some, but asks for confirmation first with the default settings.
I would like to make a broader comment: perhaps as users we should ask what AI features even make sense to use? Starting last year I experimented heavily with Google Gemini interacting with Google WorkPlace apps. The technology was cool, and is even much better now, but I came to the conclusion I don't really need it.
For an iPhone local AI, I wrote an app for myself (although I think there are maybe 10 other people who use it) that chats with Apple's local model (that is fairly good) and switches to a Secure Enclave model on their servers and from the documentation it looks like using the cloud model is private and secure.
Even better now, I signed up for ProtonMail's optional Luma LLM Chat system with integrated private web search tools. It is surprisingly good, and I trust Proton that it is private.
Almost the only thing I frequently use commercial LLMs for now is a few times a week using gemini-cli for coding, and NotebookLM a few times a month, plus occasional Gemini use, but I pay for Luma (powered by Mistral models) so I routinely use it for AI search use cases.
Just because technology is incredibly cool, this doesn't mean that we have to use it if real productivity gains are slim or non-existent.
For an iPhone local AI, I wrote an app for myself (although I think there are maybe 10 other people who use it) that chats with Apple's local model (that is fairly good) and switches to a Secure Enclave model on their servers and from the documentation it looks like using the cloud model is private and secure.
Even better now, I signed up for ProtonMail's optional Luma LLM Chat system with integrated private web search tools. It is surprisingly good, and I trust Proton that it is private.
Almost the only thing I frequently use commercial LLMs for now is a few times a week using gemini-cli for coding, and NotebookLM a few times a month, plus occasional Gemini use, but I pay for Luma (powered by Mistral models) so I routinely use it for AI search use cases.
Just because technology is incredibly cool, this doesn't mean that we have to use it if real productivity gains are slim or non-existent.
I'm somewhat disappointed that the authors did not realize that apple-relay.cloudflare.com is not PCC, but is publicly known to be part of iCloud Private Relay.
That made me immediately think that there were other parts of the article which were not properly investigated, and likely a fair bit of sensationalization.
That made me immediately think that there were other parts of the article which were not properly investigated, and likely a fair bit of sensationalization.
PCC uses 3rd party ohttp relays also.
doesn't disable Siri
"Siri is collecting all your data!"
insert company advertisement
"Siri is collecting all your data!"
insert company advertisement
Apple already slurps everything you do with their "privacy proxy" services, so what more risk do you want? Apple defaults for forwarding all dns and web pages via their proxies, they just can't seem to figure out what to do with it.
It's unclear what you're referring to. Their DNS (outside of Safari private browsing) and Private Relay aren't enabled by default. The latter is set up using blind signatures and oblivious DNS, so they can't figure out what it is (much less what to do with it). You're free to disable both and let your ISP do whatever they want with your browsing habits.
If you're talking about apple private relay, they're only proxying your packets, not intercepting any TLS connections. Therefore it's a bit misleading to characterize that as "Apple already slurps everything you do".
If apple can figure out how to carve out a privacy/individual-focused ai model that avoids the classic centralized data-harvest/training they could runaway with a large portion of the market.
So many people understand the benefits ai can bring to our lives with automations, search, etc but so many are also aware of the dangers lurking beneath the surface and causes anti-ai sentiment (for good reason).
They can be mutually exclusive but aren't generally.
So many people understand the benefits ai can bring to our lives with automations, search, etc but so many are also aware of the dangers lurking beneath the surface and causes anti-ai sentiment (for good reason).
They can be mutually exclusive but aren't generally.
All anti-AI sentiment I've seen on the internet is based on fears it'll take their jobs and comes in the form of almost-entirely-fake concerns about it supposedly using up all our water. Most people don't care about privacy very much.
At some point it became normalised to post your face, real name and home address online multiple times a day.
Privacy, for most people, is already gone
Privacy, for most people, is already gone
Do you really think so?
There have been privacy focused alternatives to Gmail since Gmail was invented and it didn’t stop everyone from letting Google index every single thing they sent or received.
Time and time again we have seen privacy nightmare products and services run away with the market. I just don’t think that people vote with their wallets if they even care at all
There have been privacy focused alternatives to Gmail since Gmail was invented and it didn’t stop everyone from letting Google index every single thing they sent or received.
Time and time again we have seen privacy nightmare products and services run away with the market. I just don’t think that people vote with their wallets if they even care at all
> they just can't seem to figure out what to do with it.
That's the funny thing about Siri. It has since Day 1 insisted on being exclusively online-only, processing on the server - even for commands that the pre-Siri Voice Control could do fully locally on an iPhone 3Gs such as "Call Steve" or "Turn Wi-Fi on"
That decision always surprised me, and it's surprising that Siri's never improved given that unlike Apple Intelligence it is not limited to on-device nor to PCC.
For all the scorn directed lately at the "Apple Intelligence" team for not shipping anything they promised, I have to hand it to JG's team for at least building a local processing capability plus having PCC instead of just one server-side thing and a "Trust Me Bro" like Siri.
If we're lucky, they'll build new Siri stuff from scratch using local + PCC, and finally push the old Siri stuff into the scrapyard where it belongs.
That's the funny thing about Siri. It has since Day 1 insisted on being exclusively online-only, processing on the server - even for commands that the pre-Siri Voice Control could do fully locally on an iPhone 3Gs such as "Call Steve" or "Turn Wi-Fi on"
That decision always surprised me, and it's surprising that Siri's never improved given that unlike Apple Intelligence it is not limited to on-device nor to PCC.
For all the scorn directed lately at the "Apple Intelligence" team for not shipping anything they promised, I have to hand it to JG's team for at least building a local processing capability plus having PCC instead of just one server-side thing and a "Trust Me Bro" like Siri.
If we're lucky, they'll build new Siri stuff from scratch using local + PCC, and finally push the old Siri stuff into the scrapyard where it belongs.
Siri has supported an offline subset of commands since iOS 15 came out in 2021.
I just tested this with Wi-Fi and cellular data disabled. Calling someone works perfectly and asking to turn Wi-Fi on presents the relevant toggle.
I just tested this with Wi-Fi and cellular data disabled. Calling someone works perfectly and asking to turn Wi-Fi on presents the relevant toggle.
Sorry, so it was only 11 years for them to restore the functionality that Siri in effect "took away" in 2010.
The only way to get it back for those 11 years was to disable Siri entirely, meaning you had to be "cloud dependent" or stick to that unchanging set of offline-only features exclusively for a decade. Not a lot for the Siri team to be proud of there.
The only way to get it back for those 11 years was to disable Siri entirely, meaning you had to be "cloud dependent" or stick to that unchanging set of offline-only features exclusively for a decade. Not a lot for the Siri team to be proud of there.
Siri doesn’t have any of the new AI features, the prompts they’re using have been around for years, and private cloud compute has always been about Apple Intelligence generative features.