This is good and clever work, and I applaud the author for looking at a boring feature and seeing in it a potential attack vector that probably wouldn't have occurred to most people. Kudos! That said, I think the security implications are pretty minimal.
(FWIW, iOS and MacOS do the exact same thing, opening captive.apple.com and showing whatever it redirects to if you're on a captive portal network.)
This behavior does incur a security risk, but using public wifi networks is basically impossible without doing this check either automatically or manually, and most users would be completely bewildered if the OS did nothing to prompt them when they needed to click through a page to make the internet work.
Moreover, if you can MITM the network and they're not tunneling their connection, you have lots of great ways to send them hostile code already! You can use classic SSL stripping to just send them whatever you want! (Granted, a lot of traffic goes straight to HTTPS these days, and browsers are getting wiser about this with HSTS and things like the new automatic HTTPS upgrade in Safari).
If you're paranoid enough not to want to run untrusted javascript (fair enough), you shouldn't be connecting to weird public wifi networks anyway.
A very similar thing happened when the folks at flathub tried to package MultiMC. Unfortunately the author threatened to sue for trademark infringement and they backed down.
Wait, this just looks like cryptographically signing pictures. What's so scary about that? You can do it today from your terminal if you feel like it.
I guess I agree that the 'trusted computing' stuff it seems like they're trying to do is a little scary, but the tech isn't really there yet, at least not on the desktop (look at the fiasco that is Intel SGX) and it's happening with or without whatever this CAI thing is.
I guess a world where your iPhone's camera sends signed frames to the processor's secure enclave which processes them and signs them with a key signed by Apple is... a little different from today? They do basically this for Face ID today.
If it's a simple proxy tunneling HTTPS traffic to Google, Apple probably doesn't know anything about the content of the queries, and Google doesn't know who sent them. If each kept records, they could get together and combine them to get the hashed URLs, but still a much better situation than directly querying a single endpoint.
(FWIW, iOS and MacOS do the exact same thing, opening captive.apple.com and showing whatever it redirects to if you're on a captive portal network.)
This behavior does incur a security risk, but using public wifi networks is basically impossible without doing this check either automatically or manually, and most users would be completely bewildered if the OS did nothing to prompt them when they needed to click through a page to make the internet work.
Moreover, if you can MITM the network and they're not tunneling their connection, you have lots of great ways to send them hostile code already! You can use classic SSL stripping to just send them whatever you want! (Granted, a lot of traffic goes straight to HTTPS these days, and browsers are getting wiser about this with HSTS and things like the new automatic HTTPS upgrade in Safari).
If you're paranoid enough not to want to run untrusted javascript (fair enough), you shouldn't be connecting to weird public wifi networks anyway.