Comcast, Mozilla strike privacy deal to encrypt DNS lookups in Firefox(arstechnica.com)
arstechnica.com
Comcast, Mozilla strike privacy deal to encrypt DNS lookups in Firefox
https://arstechnica.com/tech-policy/2020/06/comcast-mozilla-strike-privacy-deal-to-encrypt-dns-lookups-in-firefox/
210 comments
> Comcast sniffs / records / tracks their user's DNS traffic
Actually not only does Comcast say they don't do that (https://www.xfinity.com/privacy/policy/dns) but now has signed a contract to this effect as well, thereby meeting the same level of commitment as the other TRR operators. This means IMO that Mozilla is doing a good job leading the industry on DNS privacy and convincing many of the merits of a strong pro-privacy philosophy.
(disclosure: I work for Comcast and have been working on encrypted DNS)
Actually not only does Comcast say they don't do that (https://www.xfinity.com/privacy/policy/dns) but now has signed a contract to this effect as well, thereby meeting the same level of commitment as the other TRR operators. This means IMO that Mozilla is doing a good job leading the industry on DNS privacy and convincing many of the merits of a strong pro-privacy philosophy.
(disclosure: I work for Comcast and have been working on encrypted DNS)
> Actually not only does Comcast say they don't do that...
Just like they said they didn't forcibly reset BitTorrent connections (until they did).
Just like they said they didn't silently institute bandwidth caps (until they did).
Just like they said they didn't hijack NXDOMAIN responses (until they did).
Just like they said they didn't intercept plain-text HTTP connections and inject their own traffic into them (until they did).
---
With all due respect, I have personally had contracts with Comcast in the past and have experienced firsthand how well they honor those -- and I am certainly not the only one!
Surely you can understand why, to me and many others, their little agreement with Mozilla doesn't really mean a damn thing?
---
(I know that none of this is your fault and it's obviously nothing personal! I'm sure you're a smart, decent person but I'm also sure that you are well aware of your employer's reputation, their past "misdeeds", and, of course, the generally unfavorable opinion that many, many customers have of them.)
Just like they said they didn't forcibly reset BitTorrent connections (until they did).
Just like they said they didn't silently institute bandwidth caps (until they did).
Just like they said they didn't hijack NXDOMAIN responses (until they did).
Just like they said they didn't intercept plain-text HTTP connections and inject their own traffic into them (until they did).
---
With all due respect, I have personally had contracts with Comcast in the past and have experienced firsthand how well they honor those -- and I am certainly not the only one!
Surely you can understand why, to me and many others, their little agreement with Mozilla doesn't really mean a damn thing?
---
(I know that none of this is your fault and it's obviously nothing personal! I'm sure you're a smart, decent person but I'm also sure that you are well aware of your employer's reputation, their past "misdeeds", and, of course, the generally unfavorable opinion that many, many customers have of them.)
After the NXDomain redirection stuff, they invested a ton in doing DNS right, deploying DNSSEC and Anycast. They are a major participant in the IETF, DNS OARC, and many other industry working groups, if you attend these things, go talk to the Comcast folks!
Their limited use of HTTP interception is a published RFC, and I've thought about ways to get around this, and for the use cases they use it for, it seems like the only viable option.
I knew a few of folks who were involved in the BT RST thing, the whole org learned a lot from that, and internal opinions changed.
Their limited use of HTTP interception is a published RFC, and I've thought about ways to get around this, and for the use cases they use it for, it seems like the only viable option.
I knew a few of folks who were involved in the BT RST thing, the whole org learned a lot from that, and internal opinions changed.
Why does it matter, at all, that Comcast documented their traffic interception system in an RFC?
> With all due respect, I have personally had contracts with Comcast in the past and have experienced firsthand how well they honor those -- and I am certainly not the only one!
Consumer contracts? Because Mozilla having a business contract with Comcast is certainly not the same as you having a consumer contract - Mozilla has the resources to drag Comcast to court should they be found to ignore the agreement.
Consumer contracts? Because Mozilla having a business contract with Comcast is certainly not the same as you having a consumer contract - Mozilla has the resources to drag Comcast to court should they be found to ignore the agreement.
This is a wildly bad take in my opinion.
Comcast has proven themselves to be uninterested in adhering to their contractual obligations. You bet your ass they are 1) figuring out how to work around their contract with Mozilla without attracting legal attention, and 2) making contingency plans for winning any resulting lawsuit.
Comcast has proven themselves to be uninterested in adhering to their contractual obligations. You bet your ass they are 1) figuring out how to work around their contract with Mozilla without attracting legal attention, and 2) making contingency plans for winning any resulting lawsuit.
IANAL but I'm not even sure there would be a lawsuit, based on this policy document linked from the article: https://wiki.mozilla.org/Security/DOH-resolver-policy#Enforc...
It looks like the punishment for violating mozilla policies would simply be to remove Comcast from the trusted provider group...maybe. Not very impressive.
It looks like the punishment for violating mozilla policies would simply be to remove Comcast from the trusted provider group...maybe. Not very impressive.
> Mozilla has the resources to drag Comcast to court
They do not. Look at Mozilla's 1099 for proof.
They do not. Look at Mozilla's 1099 for proof.
Mozilla took Verizon to court.
Link? When was this? I'm only aware of Verizon suing Mozilla in 2017.
Where can we read the contract?
The policy linked by the article is here: https://wiki.mozilla.org/Security/DOH-resolver-policy#Enforc...
This just leads to a wiki page listing two TRR's, where someone admits that whatever "privacy policies" TRR's provide are not "contracts". Then someone replies that "there are legal contracts between Mozilla and those two listed providers". That does not mean those contracts relate to protecting user privacy. If you dig, there will be nothing there. There is nothing in any contract to protect any user. Neither Mozilla nor TRR's are "on the hook" for protecting user privacy. What I mean by that is that if some user's privacy is breached, there is absolutely zero liability accruing to Mozilla or TRR's.
https://wiki.mozilla.org/index.php?title=User_talk:Wthayer&a...
You, the end-user, will not get to see Mozilla's contracts. Policies are not contracts.
There seems to be some common misunderstandings about "policies" and tech companies are exploiting them. There is nothing legally binding in a policy and in the case the company deviates from the policy, there is no way for an affected user to "enforce" the policy she thought was being adhered to. This is of course assuming anyone outside the company actually discovers that a policy is being violated. Usually policy violations are non-detectable from outside the company.
https://wiki.mozilla.org/index.php?title=User_talk:Wthayer&a...
You, the end-user, will not get to see Mozilla's contracts. Policies are not contracts.
There seems to be some common misunderstandings about "policies" and tech companies are exploiting them. There is nothing legally binding in a policy and in the case the company deviates from the policy, there is no way for an affected user to "enforce" the policy she thought was being adhered to. This is of course assuming anyone outside the company actually discovers that a policy is being violated. Usually policy violations are non-detectable from outside the company.
When pigs fly!
Can you imagine Mozilla CEO depleting her own compensation in order to sue a company the size of Comcast? What exactly are the claimed damages in this hypothetical lawsuit? The damages to Mozilla are _____ ? (This PR piece in Ars suggests the deal is solely to protect users. No financial details.) How about the damages suffered by users? (Whoops, they are not parties to the agreement.)
Quite the vivid imagination!
Maybe the "white knight" narrative really does work on some people.
Can you imagine Mozilla CEO depleting her own compensation in order to sue a company the size of Comcast? What exactly are the claimed damages in this hypothetical lawsuit? The damages to Mozilla are _____ ? (This PR piece in Ars suggests the deal is solely to protect users. No financial details.) How about the damages suffered by users? (Whoops, they are not parties to the agreement.)
Quite the vivid imagination!
Maybe the "white knight" narrative really does work on some people.
Hypothetically, I imagine they'd sue for damage to their brand. Mozilla has been pushing Firefox as the privacy browser, and if Comcast were to break the deal it would hurt users' trust in Mozilla. I agree it is unclear what the consideration from each party is, though.
Comcast does not appear to be using the Mozilla brand. Am I missing something?
"We, Mozilla, are a privacy-first company. Firefox is powered by DNS services that respect your privacy. You can trust that your browsing history is safe with us."
> "BREAKING: Firefox default DNS provider tracks what sites you visit"
^damage to reputation
> "BREAKING: Firefox default DNS provider tracks what sites you visit"
^damage to reputation
On top of that, I'm totally mystified by what would cause this sudden change of heart from Comcast. Why do they want to provide this "service" to users, if they supposedly don't get anything out of it? If they're not profiting off the data, why not just let Firefox users connect to Cloudflare for DNS, as they're already doing to access 50% of the popular sites out there?
Why fight Mozilla to let them provide a service which is only going to cost them money to run?
Why fight Mozilla to let them provide a service which is only going to cost them money to run?
Respectfully, Comcast has an ATROCIOUS privacy record. Full stop. A quick search came up with [1] [2] [3].
Your employer actively and repeatedly abuses the privacy and trust of its customers. It also lobbies for damaging policies.
I do not trust Comcast. Firefox associating itself with Comcast makes me trust Firefox significantly less.
[1] https://oag.ca.gov/news/press-releases/attorney-general-kama...
[2] https://www.king5.com/article/news/local/comcast-fined-9-mil...
[3] https://consumerist.com/2016/04/01/comcast-says-fcc-privacy-...
Your employer actively and repeatedly abuses the privacy and trust of its customers. It also lobbies for damaging policies.
I do not trust Comcast. Firefox associating itself with Comcast makes me trust Firefox significantly less.
[1] https://oag.ca.gov/news/press-releases/attorney-general-kama...
[2] https://www.king5.com/article/news/local/comcast-fined-9-mil...
[3] https://consumerist.com/2016/04/01/comcast-says-fcc-privacy-...
> Your employer... It also lobbies for damaging policies.
To be sure, jlivingood isn't just some enterprise grunt with an opinion, but VP of Technology Policy & Standards.
To be sure, jlivingood isn't just some enterprise grunt with an opinion, but VP of Technology Policy & Standards.
Do you have a link for this? This is a pretty big difference in disclosure.
https://blog.mozilla.org/blog/2020/06/25/comcasts-xfinity-in...
Compare the username to the "Vice President, Technology Policy and Standards at Comcast Cable" in this blog post.
Compare the username to the "Vice President, Technology Policy and Standards at Comcast Cable" in this blog post.
Thanks!
To save people the click.
User name in question: jlivingood
> Jason Livingood, Vice President, Technology Policy and Standards at Comcast Cable.
To save people the click.
User name in question: jlivingood
> Jason Livingood, Vice President, Technology Policy and Standards at Comcast Cable.
[deleted]
It is entirely possible for one group (or incentive) within Comcast's organization to work towards that goal, and another group or incentive within Comcast to work against it.
Some things that make me trust this claim:
- That privacy policy
- That contract
- Association with trustworthy brands like Mozilla
- Work on encrypted DNS
- Consumer trust could theoretically be financially valuable to them
- Basic morals of Comcast employees
Some things that make me distrust it:
- Cultivated an infamous litany of consumer abuses
- Lobbied for regulations to harm consumers
- Engaged in regulatory capture with the FTC/FCC
- Have continually profited from their abusive business practices, and continue to profit in spite of atrocious levels of consumer trust
I believe that Comcast only wants to respect user privacy insofar as it helps them maintain a facade of trustworthiness and retain customers. If they ever get the opportunity to back out of that privacy-conscious posturing and turn it into money, I think history has proven that the winner of those two internal groups is clearly going to be the money side not the moral side.
Some things that make me trust this claim:
- That privacy policy
- That contract
- Association with trustworthy brands like Mozilla
- Work on encrypted DNS
- Consumer trust could theoretically be financially valuable to them
- Basic morals of Comcast employees
Some things that make me distrust it:
- Cultivated an infamous litany of consumer abuses
- Lobbied for regulations to harm consumers
- Engaged in regulatory capture with the FTC/FCC
- Have continually profited from their abusive business practices, and continue to profit in spite of atrocious levels of consumer trust
I believe that Comcast only wants to respect user privacy insofar as it helps them maintain a facade of trustworthiness and retain customers. If they ever get the opportunity to back out of that privacy-conscious posturing and turn it into money, I think history has proven that the winner of those two internal groups is clearly going to be the money side not the moral side.
I am a Comcast employee myself, but these opinions are my own:
I appreciate your comment as you clearly and accurately frame that good people work there, Comcast earned its reputation with its actions, and that there may be competing interests / morals at play.
I can’t predict the future to say what wins out, but I feel like there are competing interests at play internal to the org. There are the business/markets, and then there is technology, products and experience pieces too.
an aside: I am somewhat disappointed to see a HN thread so populated with groupthink “Comcast is bad!” I’d expect that on theVerge comment board or YouTube comments. sigh
Anyways, I hear that the wake up call was probably about the time that the time warner merger fell through. I haven’t worked for them that long. It takes time to shift an org as large as Comcast to come to terms with the public’s resentment. I can say that, in my experience on the inside, the fundamental attitudes are very consumer friendly. I’m on the tech/product side of the house and we just want to make great stuff. Reliable, scalable entertainment and home automation kind of things, ya know? Comcast strives to be an admired company. So yeah, guess its work is cut out for it! Everything is designed today with accessibility and privacy at the outset. There’s a massive amount of truly brilliant engineers, developers, QA folks etc. working there. Not saying your distrust in Comcast wasn’t earned, however I believe conditions are not what they once were, thus some negative sentiments may be outdated today.
I appreciate your comment as you clearly and accurately frame that good people work there, Comcast earned its reputation with its actions, and that there may be competing interests / morals at play.
I can’t predict the future to say what wins out, but I feel like there are competing interests at play internal to the org. There are the business/markets, and then there is technology, products and experience pieces too.
an aside: I am somewhat disappointed to see a HN thread so populated with groupthink “Comcast is bad!” I’d expect that on theVerge comment board or YouTube comments. sigh
Anyways, I hear that the wake up call was probably about the time that the time warner merger fell through. I haven’t worked for them that long. It takes time to shift an org as large as Comcast to come to terms with the public’s resentment. I can say that, in my experience on the inside, the fundamental attitudes are very consumer friendly. I’m on the tech/product side of the house and we just want to make great stuff. Reliable, scalable entertainment and home automation kind of things, ya know? Comcast strives to be an admired company. So yeah, guess its work is cut out for it! Everything is designed today with accessibility and privacy at the outset. There’s a massive amount of truly brilliant engineers, developers, QA folks etc. working there. Not saying your distrust in Comcast wasn’t earned, however I believe conditions are not what they once were, thus some negative sentiments may be outdated today.
I am sure there are some good people working at your company, and you may be one of them. But that is not really relevant here. It is the actions of the enterprise as a whole that count.
> I am somewhat disappointed to see a HN thread so populated with groupthink “Comcast is bad!”.
What I see in many comments is specific reference to cases where Comcast betrayed the public's trust. You can call it groupthink, but I get the impression the bad rep is well-earned.
Trust leaves on horseback, and comes back on foot. More work to do to convince those who doubt you. Happy to hear you are working on that.
> I am somewhat disappointed to see a HN thread so populated with groupthink “Comcast is bad!”.
What I see in many comments is specific reference to cases where Comcast betrayed the public's trust. You can call it groupthink, but I get the impression the bad rep is well-earned.
Trust leaves on horseback, and comes back on foot. More work to do to convince those who doubt you. Happy to hear you are working on that.
> an aside: I am somewhat disappointed to see a HN thread so populated with groupthink “Comcast is bad!” I’d expect that on theVerge comment board or YouTube comments. sigh
Is this really surprising? You do go on to acknowledge the “public resentment.” In the tech industry—HN’s specialty—Comcast certainly seems to be widely regarded as an evil beast with no conscience.
That’s an entirely anecdotal observation, but many of the comments in this HN thread are not: they frequently link to evidence, while Comcast employees chime in to say, “internally, it’s not like that anymore.” It’s great to hear that’s what it looks like on the inside, but given Comcast’s history and HN’s general skepticism, that’s going to be a hard sell without solid evidence.
Is this really surprising? You do go on to acknowledge the “public resentment.” In the tech industry—HN’s specialty—Comcast certainly seems to be widely regarded as an evil beast with no conscience.
That’s an entirely anecdotal observation, but many of the comments in this HN thread are not: they frequently link to evidence, while Comcast employees chime in to say, “internally, it’s not like that anymore.” It’s great to hear that’s what it looks like on the inside, but given Comcast’s history and HN’s general skepticism, that’s going to be a hard sell without solid evidence.
Comcast has had a pretty poor track record w/r/t their statements of what they do or don't do versus reality, and this goes back well over a decade (see: denials of Sandvine deployment).
I personally was impacted by this behavior after having my internet service deactivated due to going over bandwidth caps (which reportedly didn't yet exist) and it was one of the most Kafka-esque corporate experiences in my life. My internet was being deactivated because I violated a policy which they repeatedly stated didn't exist. Several months later those caps actually became policy.
I ask this with all sincerity: why should we believe Comcast?
I personally was impacted by this behavior after having my internet service deactivated due to going over bandwidth caps (which reportedly didn't yet exist) and it was one of the most Kafka-esque corporate experiences in my life. My internet was being deactivated because I violated a policy which they repeatedly stated didn't exist. Several months later those caps actually became policy.
I ask this with all sincerity: why should we believe Comcast?
Given your disclosure presumably you know if it's true or not.
It's a weird way to phrase "I work on this, we don't capture any information about customers site visits" or "I work on this we don't capture any information about domain lookups" or whatever.
When a customer gets a contract with Comcast are you saying the contract includes that Comcast will not filter/log their domain lookups in any way?
To others: what's the penalty of they breech that contract? Do they actual have anything at risk?
In UK ISPs couldn't make such a contract as the gov obliges DNS filtering of some domains, AIUI.
It's a weird way to phrase "I work on this, we don't capture any information about customers site visits" or "I work on this we don't capture any information about domain lookups" or whatever.
When a customer gets a contract with Comcast are you saying the contract includes that Comcast will not filter/log their domain lookups in any way?
To others: what's the penalty of they breech that contract? Do they actual have anything at risk?
In UK ISPs couldn't make such a contract as the gov obliges DNS filtering of some domains, AIUI.
UK ISPs could offer to operate a Mozilla TRR DoH server for two reasons:
1. The deal major "as seen on TV" ISPs struck was that they would offer a configurable child protection style filtering to their users. Mozilla permits users to opt in to filtering, you just aren't allowed to filter by default, so an ISP provided DoH server which can be configured explicitly to filter would meet this requirement. NextDNS offers this, yet is in Mozilla's programme. If you just pick NextDNS from the drop-down in Firefox you get no filtering, if you sign up and pay them (or take the free offer) you get filtering of your choice, and DoH, with instructions on how to tell your Firefox about this (basically paste a per-user URL into a preferences window, the nice thing about DoH compared to plain DNS or even DoT is that in a URL your user identifier will be encrypted, improving privacy)
2. The government did not legislate a requirement. They've been burned before on the difference between public appeals to think of the children (generally broadly accepted by the populace, no legal fallout) and censorship laws (likely to be destroyed in the courts because it turns out people don't like being told what to read). All those famous ISPs chose to voluntarily censor the Internet (mustn't let kids see porn) and then since they had the capability to censor courts told them to also obey Hollywood's instructions (no Pirate Bay either).
A small ISP like Andrews & Arnold isn't censored. During sign-up it says "Do you need child friendly filtering as part of this product?" or something. If you click "Yes" it says sorry they don't want you as a customer, good bye and the sign-up process is over.
1. The deal major "as seen on TV" ISPs struck was that they would offer a configurable child protection style filtering to their users. Mozilla permits users to opt in to filtering, you just aren't allowed to filter by default, so an ISP provided DoH server which can be configured explicitly to filter would meet this requirement. NextDNS offers this, yet is in Mozilla's programme. If you just pick NextDNS from the drop-down in Firefox you get no filtering, if you sign up and pay them (or take the free offer) you get filtering of your choice, and DoH, with instructions on how to tell your Firefox about this (basically paste a per-user URL into a preferences window, the nice thing about DoH compared to plain DNS or even DoT is that in a URL your user identifier will be encrypted, improving privacy)
2. The government did not legislate a requirement. They've been burned before on the difference between public appeals to think of the children (generally broadly accepted by the populace, no legal fallout) and censorship laws (likely to be destroyed in the courts because it turns out people don't like being told what to read). All those famous ISPs chose to voluntarily censor the Internet (mustn't let kids see porn) and then since they had the capability to censor courts told them to also obey Hollywood's instructions (no Pirate Bay either).
A small ISP like Andrews & Arnold isn't censored. During sign-up it says "Do you need child friendly filtering as part of this product?" or something. If you click "Yes" it says sorry they don't want you as a customer, good bye and the sign-up process is over.
I chose "oblige" carefully. Basically the ISPs AIUI/IIRC were given the option to voluntarily abide by a blocklist of domains - like thepiratebay - or have legislation made to force them to comply.
I'd heard A&A were an outlier here but didn't know they actively stopped users from choosing their service if they want filtering. That seems weirdly fascist: like a supermarket that won't let you choose not to have Coca Cola on your shopping list, if you don't want it you have to actively remove it yourself.
Porn, yes to some extent, but super-violence, torture, malware, gambling, prostitution, ... these are all things I choose to attempt not to pipe in to my home via OpenDNS/pihole/uBlock/direct instruction to local users!
I'd heard A&A were an outlier here but didn't know they actively stopped users from choosing their service if they want filtering. That seems weirdly fascist: like a supermarket that won't let you choose not to have Coca Cola on your shopping list, if you don't want it you have to actively remove it yourself.
Porn, yes to some extent, but super-violence, torture, malware, gambling, prostitution, ... these are all things I choose to attempt not to pipe in to my home via OpenDNS/pihole/uBlock/direct instruction to local users!
I don't agree that choosing not to offer a product or service which is popular but which you believe is a terrible idea is "weirdly fascist".
I would suggest instead that demanding other people figure out whatever weird quirks you have and then cater to them under the guise of being "child friendly" is at best weirdly fascist.
And it seems you at least reluctantly agree this that can't work even if you wish A&A would try to do it anyway, since all the systems you listed involve you explicitly configuring what you want blocked, allowing you to take responsibility for the inevitable under/overblocking. This approach works fine† with A&A since it doesn't ask anything of them.
† Well, as "fine" as can be expected, no worse than at other providers.
I would suggest instead that demanding other people figure out whatever weird quirks you have and then cater to them under the guise of being "child friendly" is at best weirdly fascist.
And it seems you at least reluctantly agree this that can't work even if you wish A&A would try to do it anyway, since all the systems you listed involve you explicitly configuring what you want blocked, allowing you to take responsibility for the inevitable under/overblocking. This approach works fine† with A&A since it doesn't ask anything of them.
† Well, as "fine" as can be expected, no worse than at other providers.
If Comcast sells DNS data now, they open themselves up to penalties from the both FTC and Mozilla. FTC because they enforce privacy policies, and Mozilla because of the contract they have.
I would say this Mozilla changing the overall ecosystem for the better.
I would say this Mozilla changing the overall ecosystem for the better.
Do we know what the actual penalties are? I have trouble believing that they are of any substance.
Additionally, I think it's safe to say that Comcast has years and years of experience in finding "loopholes" and/or other "workarounds" in its agreements.
> I would say this Mozilla changing the overall ecosystem for the better.
You obviously have much more faith in Comcast than I do. Let's hope you're right.
Additionally, I think it's safe to say that Comcast has years and years of experience in finding "loopholes" and/or other "workarounds" in its agreements.
> I would say this Mozilla changing the overall ecosystem for the better.
You obviously have much more faith in Comcast than I do. Let's hope you're right.
The FTC has no teeth and no one at Comcast is losing any sleep over potential FTC penalties. Ajit Pai is more likely encourage meaningful enforcement of this agreement than anyone at the FTC. Which is to say, I don’t have much faith.
I imagine Comcast would not sell DNS data; far better to simply use it internally with, for example:
- NBC
- Telemundo
- USA Network
- SyFy
- Universal Pictures
- Sky Group
- NBC
- Telemundo
- USA Network
- SyFy
- Universal Pictures
- Sky Group
What good is a contract between Mozilla and Comcast to me if Mozilla is unwilling to notice/care/sue when Comcast breaks that contract because Mozilla has a financial incentive to turn a blind eye?
That’s great that they’ve said they don’t publicly and that they’re now going to be contractually obligated not to... but that’s kind of irrelevant if they are actually sniffing, tracking, and/or recording user DNS traffic.
And the fact that Comcast doesn’t allow users to change the DNS settings of xfinity routers leads me to believe they have some monetary incentive to go in and disable that functionality in their equipment.
And the fact that Comcast doesn’t allow users to change the DNS settings of xfinity routers leads me to believe they have some monetary incentive to go in and disable that functionality in their equipment.
The link provided only concerns logging of DNS requests on Comcast's DNS servers themselves. It doesn't say anything about recording packets to UDP port 53 in general. If Comcast were recording every packet I sent to 8.8.8.8 they wouldn't be breaking the letter of the policy.
Hopefully this is exactly what DoH should be addressing.
Can users enforce that contract if it is breached? Maybe users should be named explicitly as third party beneficiaries.
Mozilla is a company and needs to compensate its CEO and staff. The CEO received $2,458,350 in 2018.^1 Everything Mozilla does cannot be solely for users' benefit. Mozilla, the people behind it, have their own financial interests.
1. https://assets.mozilla.net/annualreport/2018/mozilla-2018-fo...
Mozilla is a company and needs to compensate its CEO and staff. The CEO received $2,458,350 in 2018.^1 Everything Mozilla does cannot be solely for users' benefit. Mozilla, the people behind it, have their own financial interests.
1. https://assets.mozilla.net/annualreport/2018/mozilla-2018-fo...
You work(ed) on IPv6 as well, right? You get all the fun projects. :-)
> Comcast sniffs / records / tracks their user's DNS traffic
what if the are your dns server?
what if the are your dns server?
I don't understand why Mozilla should care or get involved at all into what Comcast thinks of them.
Mozilla introduce a privacy feature in a free, open-source browser. Comcast bitches about it because it prevents them from doing shenanigans, essentially incriminating themselves and proving that the feature is both working as expected and necessary.
Why does Mozilla need to care about Comcast's opinion on this, and try to work out an "agreement" with them? It's equivalent to antivirus software actually working, and then the antivirus developer does a deal with the virus makers because the antivirus is actually effective and the virus makers are complaining.
I am really disappointed in Mozilla's recent features, poor focus and nonsensical management.
Mozilla introduce a privacy feature in a free, open-source browser. Comcast bitches about it because it prevents them from doing shenanigans, essentially incriminating themselves and proving that the feature is both working as expected and necessary.
Why does Mozilla need to care about Comcast's opinion on this, and try to work out an "agreement" with them? It's equivalent to antivirus software actually working, and then the antivirus developer does a deal with the virus makers because the antivirus is actually effective and the virus makers are complaining.
I am really disappointed in Mozilla's recent features, poor focus and nonsensical management.
> Why does Mozilla need to care about Comcast's opinion on this, and try to work out an "agreement" with them?
Money.
Money.
You are being downvoted, but are not wrong. A lot of people confuse Mozilla the foundation with Mozilla the corporation, which is the profit seeking branch and the entity which holds the IP. They are different utilities with different goals. They hold similar contracts with Google and you would all do well to at very least become aware of their financial prospectus.
Which is why I insist on running my own fork off FF, stripping and adding stuff as I see fit, and compiling it myself.
Those who don't are subjected to the whims of either the foundation or corproation of Mozilla.
Those who don't are subjected to the whims of either the foundation or corproation of Mozilla.
> Why does Mozilla need to care about Comcast's opinion on this
Because Comcast has Washington in its pocket?
Because Comcast has Washington in its pocket?
> I'm really confused why Mozilla would agree to this.
If it's anything like their CloudFlare deals, then Mozilla did this because they were able to secure contracts that provide additional privacy protections for Mozilla's customers that the parent company doesn't normally provide to end-users.
In theory, those contracts should be enforceable in court. Whether or not you think the companies Mozilla contracts with will find ways to snoop on the private data anyway is a different story.
If it's anything like their CloudFlare deals, then Mozilla did this because they were able to secure contracts that provide additional privacy protections for Mozilla's customers that the parent company doesn't normally provide to end-users.
In theory, those contracts should be enforceable in court. Whether or not you think the companies Mozilla contracts with will find ways to snoop on the private data anyway is a different story.
Who are Mozilla's customers? Google are to a close approximation the only ones who pay them??
Customers = users. Mozilla offers paid features in Pocket and Firefox Private Network, and are looking to offer more premium tiers of services. Specifically, the customers who use or pay for FPN have additional privacy protections that CloudFlare doesn't offer to its own customers, despite FPN running on their network.
The optimist would say s/customer/donor but I'm not an optimist.
Also: People don't want Mozilla to send all DoH traffic to one company, so Mozilla has set a list of things they require a company to do so that they'll use their DoH servers automatically. Apparently, Comcast has agreed to said list. (We'll have to see if details on oversight and enforcement are published)
> People don't want Mozilla to send all DoH traffic to one company
Good point, CloudFlare have earned a good reputation, but a monoculture is still something to avoid.
Who would be your ideal DNS provider? Perhaps someone like the EFF? FSF? Mozilla themselves?
Good point, CloudFlare have earned a good reputation, but a monoculture is still something to avoid.
Who would be your ideal DNS provider? Perhaps someone like the EFF? FSF? Mozilla themselves?
The browser is not the place for configuring DNS settings.
This is the Mozilla we know. While touting their privacy record, they keep running head first into the biggest privacy debacles imaginable.
Certifying an ISP that ran a NXDOMAIN hijack scheme while no doubt also selling DNS data for DoH is beyond the pale.
Certifying an ISP that ran a NXDOMAIN hijack scheme while no doubt also selling DNS data for DoH is beyond the pale.
>Now, Comcast and Mozilla come to some sort of agreement which effectively restores Comcast's access to their customer's DNS traffic?
They could (and can) do that regardless of DNS. Most websites and other services are uniquely identifiable by their IP(-range).
Encrypted SNI is still a draft so not applicable here.
They could (and can) do that regardless of DNS. Most websites and other services are uniquely identifiable by their IP(-range).
Encrypted SNI is still a draft so not applicable here.
> Most websites and other services are uniquely identifiable by their IP(-range).
You are very correct about that:
"What can you learn from an IP?" https://irtf.org/anrw/2019/slides-anrw19-final44.pdf
Basically Encrypted Client Hello (the new name for eSNI) and DoH/DoT are only useful for websites hosted on global CDN providers like Akamai, Cloudflare and so on.
You are very correct about that:
"What can you learn from an IP?" https://irtf.org/anrw/2019/slides-anrw19-final44.pdf
Basically Encrypted Client Hello (the new name for eSNI) and DoH/DoT are only useful for websites hosted on global CDN providers like Akamai, Cloudflare and so on.
From the article:
> Comcast's version of DNS over HTTPS (DoH) will be turned on by default for Firefox users on Comcast's broadband network
> Comcast is the first ISP to join Firefox's Trusted Recursive Resolver (TRR) program
> Cloudflare and NextDNS were already in Mozilla's program
> which requires encrypted-DNS providers to meet privacy and transparency criteria and pledge not to block or filter domains by default
I believe Firefox must not turn on Comcast's DoH as default even for Comcast ISP users. They ought to show some kind of first time prompt.
> Comcast's version of DNS over HTTPS (DoH) will be turned on by default for Firefox users on Comcast's broadband network
> Comcast is the first ISP to join Firefox's Trusted Recursive Resolver (TRR) program
> Cloudflare and NextDNS were already in Mozilla's program
> which requires encrypted-DNS providers to meet privacy and transparency criteria and pledge not to block or filter domains by default
I believe Firefox must not turn on Comcast's DoH as default even for Comcast ISP users. They ought to show some kind of first time prompt.
It would also be a good idea to split your encrypted DNS lookups between as many providers as possible so as to both minimize its value to any one entity and also minimize the amount of data a single bad-faith provider would get.
Mozilla has been untrustworthy from a privacy angle since the day they instituted silent spyware in the form of telemetry into their browser.
If I’m going to use a spyware browser, I may as well use Chrome, because it’s way more secure thanks to Google’s industry-leading security team.
If I’m going to use a spyware browser, I may as well use Chrome, because it’s way more secure thanks to Google’s industry-leading security team.
welp, thats it lads. DoH was a fun experiment until Mozilla decided to let the fox back into the henhouse. You can be damn sure "the foundation" got a nice fat cheque for their hard work letting predatory surveillance capitalism continue unfettered.
s/foundation/corporation/
Mozilla loves partnerships. They don't have to be revenue streams. They just preferably have to be hostile to user freedom.
Mozilla needs to fix the internal incentives that lead to this situation.
Mozilla needs to fix the internal incentives that lead to this situation.
Considering that Comcast sniffs, intercepts, and injects into HTTP web sites for their customer notification system(data cap overages and such) this just screams suspicious to me even if it seems like it is meant to be a good announcement. I am not sure how I am supposed to trust that they will do the right thing for their customers.
There is nothing that Comcast can do that would increase my opinion of them re privacy. In my security regime ISPs like them are on the other side. Last-mile ISPs are unsecured public networks that shouldn't be trusted any more than free airport wifi. I want them blind to everything I (and my client) does online. Encrypt everything. Route DNS to trusted non-profit entities. Serve me the encrypted data I request but otherwise I don't want to even have a conversation with Comcast.
> Route DNS to trusted non-profit entities.
I'm sure you know this, but some readers might not. DNS is totally insecure. Even if you change your DNS server from the default to 1.1.1.1 or whatever, your ISP can and does still read and/or intercept these requests. This sort of interference is absolutely trivial to implement, even at scale. Don't think it isn't happening to you.
I'm sure you know this, but some readers might not. DNS is totally insecure. Even if you change your DNS server from the default to 1.1.1.1 or whatever, your ISP can and does still read and/or intercept these requests. This sort of interference is absolutely trivial to implement, even at scale. Don't think it isn't happening to you.
... which is exactly why DoH is gaining attention.
But I keep wondering: Can't the ISP trivially correlate the accessed IP addresses with their corresponding sites even without DNS query data?
But I keep wondering: Can't the ISP trivially correlate the accessed IP addresses with their corresponding sites even without DNS query data?
Only for sites with dedicated IPs. If they're hosted on some sort of cloud service then the ISP has to sniff the SNI data. And with ESNI coming to encrypt it that hole will be plugged soon.
That just means moving from the ISP in a prime position for snooping to various CDNs being in that prime position.
You traded one master for another.
You traded one master for another.
>You traded one master for another.
No. By definition, last mile ISP sees 100% of net-bound traffic. "Various CDNs" itself already represents a dilution of that view, and are not universal themselves. It's an inherent improvement even outside of other factors. But there are other factors, including a decrease in the level of natural monopoly. Last-mile ISPs often have zero effective competition, and even with one or two there are often high change over costs, longer term contracts involved, etc. The closer you get to the net's core however, the more bandwidth there is and the more players there are and in turn vastly more competition potential. That's not a guarantee sure, but it absolutely makes a difference. There's also a limiting factor principle at work: even if you do trust a given ISP, how does that help you avoid CDNs anyway?
It's the same reason that spinning up your own instance of an algo VPN on some VPS and funneling all your home and mobile traffic through that may have practical benefits. Sure in principle the VPS (or data center if you go on your own metal) provider could try spying as well. But competition there is fierce, the average technical level of users is higher, major business interests are involved in reputation, and swapping to another provider is utterly trivial. The incentives and business models for the likes of Amazon/DigitalOcean/Google/Microsoft/OVH/Scaleway/Vultr/[...] in their compute offerings are completely different from the likes of AT&T/Charter/Comcast/T-mobile/Verizon. So it is in fact reasonable to expect a difference in the level of shenanigans too, and hey, if not you can easily move, which also in turn makes it much easier as a practical matter to retaliate (sue), which virtuously further decreases the likelihood of shenanigans.
No. By definition, last mile ISP sees 100% of net-bound traffic. "Various CDNs" itself already represents a dilution of that view, and are not universal themselves. It's an inherent improvement even outside of other factors. But there are other factors, including a decrease in the level of natural monopoly. Last-mile ISPs often have zero effective competition, and even with one or two there are often high change over costs, longer term contracts involved, etc. The closer you get to the net's core however, the more bandwidth there is and the more players there are and in turn vastly more competition potential. That's not a guarantee sure, but it absolutely makes a difference. There's also a limiting factor principle at work: even if you do trust a given ISP, how does that help you avoid CDNs anyway?
It's the same reason that spinning up your own instance of an algo VPN on some VPS and funneling all your home and mobile traffic through that may have practical benefits. Sure in principle the VPS (or data center if you go on your own metal) provider could try spying as well. But competition there is fierce, the average technical level of users is higher, major business interests are involved in reputation, and swapping to another provider is utterly trivial. The incentives and business models for the likes of Amazon/DigitalOcean/Google/Microsoft/OVH/Scaleway/Vultr/[...] in their compute offerings are completely different from the likes of AT&T/Charter/Comcast/T-mobile/Verizon. So it is in fact reasonable to expect a difference in the level of shenanigans too, and hey, if not you can easily move, which also in turn makes it much easier as a practical matter to retaliate (sue), which virtuously further decreases the likelihood of shenanigans.
You have a contractual relation with your ISP and they're in your jurisdiction so at least in theory you have legal recourse.
Advocating for ESNI on the other hand means argueing for more centralization towards entities which are far more removed from you where you have little recourse. So as far as incentives go they may be more beholden to some law enforcement agency than you the non-customer.
There are difference, but it does not appear to be an obvious improvement to me.
Advocating for ESNI on the other hand means argueing for more centralization towards entities which are far more removed from you where you have little recourse. So as far as incentives go they may be more beholden to some law enforcement agency than you the non-customer.
There are difference, but it does not appear to be an obvious improvement to me.
I think you bring up good points but your takeaway is 180 off.
Different jurisdiction means less likely to be answerable to your local government should they be oppressive.
The fact you don't have a contractual relationship with a CDN is a good thing because it becomes trivial simply to stop using them, should the need arise. Don't like Cloudflare? block them. Your choice of websites will be reduced greatly but you still have an operational internet connection.
Different jurisdiction means less likely to be answerable to your local government should they be oppressive.
The fact you don't have a contractual relationship with a CDN is a good thing because it becomes trivial simply to stop using them, should the need arise. Don't like Cloudflare? block them. Your choice of websites will be reduced greatly but you still have an operational internet connection.
Only if the site is using a CDN. I said some sort of "cloud" service, which (IMO, it's an admittedly fuzzy term for someone else's computer) includes shared hosting providers. All together there are a LOT of shared hosts out there, not just AWS, Azure, and GCP. So collating all your habits becomes much harder.
I found this study interesting:
"What can you learn from an IP?" https://irtf.org/anrw/2019/slides-anrw19-final44.pdf
So essentially ESNI/DoH are only useful for websites on global CDN providers? Why would Mozilla be interested in enhancing those companies profits?
"What can you learn from an IP?" https://irtf.org/anrw/2019/slides-anrw19-final44.pdf
So essentially ESNI/DoH are only useful for websites on global CDN providers? Why would Mozilla be interested in enhancing those companies profits?
Or any other shared hosting. I never mentioned global CDNs, just "cloud" (somebody else's computer with other clients) hosting.
Also don't forget about SNI. This exposes the domain you're connecting to over TLS. Yes, I know eSNI is a thing, but it's new and so unlikely to be deployed much.
Trivial in the individual case is not trivial at scale.
>> Even if you change your DNS server from the default to 1.1.1.1 or whatever, your ISP can and does still read and/or intercept these requests.
Not if VPNs/firewalls are properly implmented. Plugging DNS leaks is security 101.
Not if VPNs/firewalls are properly implmented. Plugging DNS leaks is security 101.
I get that VPNs are considered more trustworthy by some (although so many also have shoddy records/ownership), but it's still 1 extra party to trust vs. just your DNS provider.
If your VPN is only being used for DNS queries, then they don't have much useful data there. I've recently seen instructions for setting this up on free cloud instances from AWS/Azure/etc. If the only traffic is DNS wrapped in VPN, it should stay within the free tier (especially if you add local caching, but even that shouldn't be necessary)
I'm very suspicious of Comcast too. They've had hostile policies in their Internet management for as long as I can remember, going back to the days they'd forge RST packets because they didn't like customers using BitTorrent.
OTOH as the article says, 'Joining Mozilla's program means that Comcast agreed that it won't "retain, sell, or transfer to any third party (except as may be required by law) any personal information, IP addresses, or other user identifiers, or user query patterns from the DNS queries sent from the Firefox browser'.
I assume Mozilla will audit and keep Comcast honest here, or at least try. I just am left wondering what loophole Comcast has found.
OTOH as the article says, 'Joining Mozilla's program means that Comcast agreed that it won't "retain, sell, or transfer to any third party (except as may be required by law) any personal information, IP addresses, or other user identifiers, or user query patterns from the DNS queries sent from the Firefox browser'.
I assume Mozilla will audit and keep Comcast honest here, or at least try. I just am left wondering what loophole Comcast has found.
I hit commit on that feature. Sorry.
It's not just Comcast. Every ISP that uses Akamai's software to power their ISP has this ability and possibly uses it, just not in obvious ways. And given that almost every ISP in the US, let alone the world, uses this software, well.. that's just how it is.
Though, it's http only. You can always switch to https and they can not do anything. There is no key injection or anything going on thankfully. Also, encrypted DNS should make it moot as well.
It's not just Comcast. Every ISP that uses Akamai's software to power their ISP has this ability and possibly uses it, just not in obvious ways. And given that almost every ISP in the US, let alone the world, uses this software, well.. that's just how it is.
Though, it's http only. You can always switch to https and they can not do anything. There is no key injection or anything going on thankfully. Also, encrypted DNS should make it moot as well.
How would encrypting DNS help me avoid Comcast MITMing my HTTP traffic to inject bandwidth cap notifications? Doesn't the system just inject a script tag into the appropriate place in the HTTP response?
HTTPS Everywhere + encrypted DNS blocks a huge chunk of what they can see without expending effort on you in particular
FYI if you block this notification (it requires the script tag to load and you to click an I AGREE TO PAY PER GB bullshit thing), you can no longer do UDP traffic and your TCP connections start getting reset.
I found this out the hard way, because I browse nearly all HTTPS sites, and uBlock blocks the injected malware script on the few plaintext sites and never really noticed.
Your IP changes into a shared Comcast-run squid proxy one, all TCP ports are no longer available other than 80/443 filtered through squid, all UDP is no longer available.
I found this out the hard way, because I browse nearly all HTTPS sites, and uBlock blocks the injected malware script on the few plaintext sites and never really noticed.
Your IP changes into a shared Comcast-run squid proxy one, all TCP ports are no longer available other than 80/443 filtered through squid, all UDP is no longer available.
that is not what I was asking. GP claimed that encrypted DNS would stop comcast from injecting notifications into HTTP traffic, I want to know how that would work, in the hopes that my assumptions about the system are wrong.
You make a good point. I worked on the code before encrypted DNS was a thing (or anything I knew about) so I'm going off of theory, not first hand experience.
When a request is sent for a http web page it is ran through the layer 4 proxy. In there is a user profile where http injection can occur. It works by injecting JavaScript into the end of the web page.
If the dns request is encrypted all of the handshaking goes through tls bypassing the proxy's view of this data for everything except disconnected http body data. However, it could be that as years have gone by it's been updated to take in http data without any sort of head and then it would work again. It's probably as simple as running some regex looking for </html>.
So, me in my half awake state this morning didn't really think it through. In previous versions of the software this wouldn't be supported, but in hindsight it's not a terribly difficult problem to fix, so Comcast probably does support HTTP injection even when using encrypted DNS by now. My apologies.
When a request is sent for a http web page it is ran through the layer 4 proxy. In there is a user profile where http injection can occur. It works by injecting JavaScript into the end of the web page.
If the dns request is encrypted all of the handshaking goes through tls bypassing the proxy's view of this data for everything except disconnected http body data. However, it could be that as years have gone by it's been updated to take in http data without any sort of head and then it would work again. It's probably as simple as running some regex looking for </html>.
So, me in my half awake state this morning didn't really think it through. In previous versions of the software this wouldn't be supported, but in hindsight it's not a terribly difficult problem to fix, so Comcast probably does support HTTP injection even when using encrypted DNS by now. My apologies.
I don't care what their intent is. Interception and injection from a third party is bad.
With technology there are many ways to achieve the goal of notifying a customer without foul play.
With technology there are many ways to achieve the goal of notifying a customer without foul play.
Agreed, if it was cloudflare I would have at least given them the benefits of a doubt, but not Comcast.
Cloudflare is harmful to independent CDN's. They hide the originating i.p. address (no other does it) to the nameserver in the name of "privacy" so you can only have as much granularity as the nearest cloudflare server to user (anti-competitive). That is unless you buy an ipv4 block (because everyone is still on ipv4) and set up anycast.
For non 1.1.1.1 dns users I can just set up varnish and they'll be served via GEO ip lookup so the domain gets pointed to the nearest ip address. This is much cheaper. I'm not going to buy an ipv4 block just for cloudflare dns users.
Their privacy claim is a lie because your webserver is going to be exposed to the end user i.p. address anyways after resolving from the nameserver.
For non 1.1.1.1 dns users I can just set up varnish and they'll be served via GEO ip lookup so the domain gets pointed to the nearest ip address. This is much cheaper. I'm not going to buy an ipv4 block just for cloudflare dns users.
Their privacy claim is a lie because your webserver is going to be exposed to the end user i.p. address anyways after resolving from the nameserver.
Not forwarding EDNS client subnet is a requirement for Mozilla TRR partners. NextDNS also doesn't forward EDNS subnet client since it is a partner and soon Comcast will be joining that list. Although not currently a member of the program Quad9 also by default doesn't forward EDNS subnet info.
I host my name server's myself. If Mozilla is really doing this, they're misguided. All this does is make it impossible to serve requests from nearest webserver. You are getting the ip address anyways, so why do this? This means if I get someone from netherlands I'd have to redirect their requests from www.example.com to nl.example.com or buy an ipv4 block, set up anycast and then serve from the closest ipaddress/server. The end result is same. I'll always get the end user's ip address unless they use a VPN or something.
This is a stupid decision by Mozilla. Too bad Firefox users when visiting websites making their own CDN's without anycast/country level redirects will see much slower sites.
This is a stupid decision by Mozilla. Too bad Firefox users when visiting websites making their own CDN's without anycast/country level redirects will see much slower sites.
What you're claiming is false.
Cloudflare has over 200 PoPs; in your own name servers, you can use the Cloudflare Resolver's IP (which will be a "close to the user" IP, not 1.1.1.1) to do geotargeting and serve from your closest IP address/server.
Cloudflare has over 200 PoPs; in your own name servers, you can use the Cloudflare Resolver's IP (which will be a "close to the user" IP, not 1.1.1.1) to do geotargeting and serve from your closest IP address/server.
>What you're claiming is false.
Cloudflare has over 200 PoPs; in your own name servers, you can use the Cloudflare Resolver's IP (which will be a "close to the user" IP, not 1.1.1.1) to do geotargeting and serve from your closest IP address/server.
What if my server is closer than cloudflare? Why is cloudflare artificially limiting?
What if my server is closer than cloudflare? Why is cloudflare artificially limiting?
I use cloudflare precisely because I don't want clients hitting the server directly. That's its entire purpose. For both caching and anti-ddos reasons.
rydre is specifically talking about the perspective of someone running a non-cloudflare CDN or possibly a site that does their own CDN with DNS rather than anycast because they explicitly don't want to use cloudflare for whatever reason. They are not talking about someone just hosting a site.
You will always get the IP because you host the nameserver and the web server. In many cases, the nameserver is hosted by a third party. Doesn’t what Mozilla and Cloudflare have done prevent the nameserver from receiving the originating IP, thus preventing large nameservers from generating user traffic histories? This is an honest question. I’m not fully familiar with the process of running a nameserver.
EDIT: I associated the name Cloudflare with the main product. I forgot the context of this being about DNS and therefore my comment is about Cloudflare CDN not 1.1.1.1
> Cloudflare is harmful to independent CDN's.
Cloudflare is a competitor to CDNs. You don't need to use a CDN with Cloudflare, they proxy your content fully and do caching along the way as needed.
> Their privacy claim is a lie because your webserver is going to be exposed to the end user i.p. address anyways after resolving from the nameserver.
It's not a lie. Cloudflare is the nameserver, and the CDN. So after resolution the end user still has just a Cloudflare IP.
> Cloudflare is harmful to independent CDN's.
Cloudflare is a competitor to CDNs. You don't need to use a CDN with Cloudflare, they proxy your content fully and do caching along the way as needed.
> Their privacy claim is a lie because your webserver is going to be exposed to the end user i.p. address anyways after resolving from the nameserver.
It's not a lie. Cloudflare is the nameserver, and the CDN. So after resolution the end user still has just a Cloudflare IP.
> It's not a lie. Cloudflare is the nameserver, and the CDN. So after resolution the end user still has just a Cloudflare IP.
> In 2011 Google wrote an IETF draft to send Client IP information using the EDNS0 extension and this is usually called ‘edns-client-subnet’. As a DNS client, it means that a truncated version of your IP address will be added into the DNS request. The DNS server will use this truncated IP address to make a more informed decision in how it responds so that you can be connected to the most optimal server. This standard is promoted by the Faster Internet initiative and already adopted by some leading vendors.
Because it is designed to keep privacy, the sender has the freedom to limit the client IP information. Instead of sending a full IP address, the DNS server is able to send partial information such as /24 only. For instance, if your IP address is 66.214.81.22, the DNS server will only expose the first three octets, so 66–214–81. Armed with the real IP address of the querying device, the DNS server can now come up with a much more accurate response. With this more intelligent routing, customers have a better Internet experience with lower latency and faster speeds. Best of all, this integration is being done using an open standard that is available for any company to integrate into their own platform.
source: https://engineering.salesforce.com/why-is-edns-important-for...
Cloudflare 1.1.1.1 for consumers kills EDNS "edns-client-subnet" and instead offers the ip of the nearest cloudflare server to the user even if the website is not using cloudflare. This means your website can not ever serve content faster then cloudflare even if you could potentially be faster.
This is the reason why many internet archives do not allow access to cloudflare client dns (1.1.1.1) users as a form of protest.
BTW, Cloudflare allows you to get informed about the end user's ip via a "x-forwarded-for" header.
> In 2011 Google wrote an IETF draft to send Client IP information using the EDNS0 extension and this is usually called ‘edns-client-subnet’. As a DNS client, it means that a truncated version of your IP address will be added into the DNS request. The DNS server will use this truncated IP address to make a more informed decision in how it responds so that you can be connected to the most optimal server. This standard is promoted by the Faster Internet initiative and already adopted by some leading vendors.
Because it is designed to keep privacy, the sender has the freedom to limit the client IP information. Instead of sending a full IP address, the DNS server is able to send partial information such as /24 only. For instance, if your IP address is 66.214.81.22, the DNS server will only expose the first three octets, so 66–214–81. Armed with the real IP address of the querying device, the DNS server can now come up with a much more accurate response. With this more intelligent routing, customers have a better Internet experience with lower latency and faster speeds. Best of all, this integration is being done using an open standard that is available for any company to integrate into their own platform.
source: https://engineering.salesforce.com/why-is-edns-important-for...
Cloudflare 1.1.1.1 for consumers kills EDNS "edns-client-subnet" and instead offers the ip of the nearest cloudflare server to the user even if the website is not using cloudflare. This means your website can not ever serve content faster then cloudflare even if you could potentially be faster.
This is the reason why many internet archives do not allow access to cloudflare client dns (1.1.1.1) users as a form of protest.
BTW, Cloudflare allows you to get informed about the end user's ip via a "x-forwarded-for" header.
There is only a single "archive" that does not allow access to Cloudflare DNS users - not many.
It is also exceedingly unlikely that you have greater density of anycast PoPs than Cloudflare's 200+. In your case, you have zero...
It is also exceedingly unlikely that you have greater density of anycast PoPs than Cloudflare's 200+. In your case, you have zero...
Akamai has more than 200 pops and do geodns to stear traffic.
If I compare cloudflare DNS vs Google DNS, I can see a difference of ~50ms between the Akamai POPs offered.
https://pastebin.com/raw/xFQb4pVF
If I compare cloudflare DNS vs Google DNS, I can see a difference of ~50ms between the Akamai POPs offered.
https://pastebin.com/raw/xFQb4pVF
Even archive.today has given up on that crusade; I noticed a few days ago that they don't block me anymore (I use Cloudflare DNS) so they have to have stopped within the past couple weeks.
So now AFAIK the number of sites that block DNS resolvers which do not forward edns-client-subnet is zero. As it should be.
So now AFAIK the number of sites that block DNS resolvers which do not forward edns-client-subnet is zero. As it should be.
They continue to attempt to try to associate your connections/use dns cookies. CtrlF 'pixel' when you are visiting one of their pages (not frontpage)
They also attempt to correlate .onion traffic.
They also attempt to correlate .onion traffic.
[deleted]
I think you’re confusing how CloudFlare can hide the server IP from the user (to protect against DDoS) versus how CloudFlare DNS hides the client’s IP from the name server, even though the Client IP is exposed to the web server by Cloudflare, and of course standard Cloudflare is irrelevant if the requested site isn’t using the Cloudflare service in the first place.
I also wonder what the RIAA and MPAA think of this? I know using a browser isn't ideal for piracy, but at the same time those two organizations are one of several major reasons why ISPs snoop on their customers.
does RIAA/MPAA snooping rely on DNS traffic at all? I was under the impression that the only thing RIAA/MPAA caught people for was the act of sharing, usually in the form of torrent seeding, and this was done by third parties they contracted out to who would provide a list of IPs they caught seeding. Then the list of IPs were resolved to ISPs who were sent subpoenas where they thought they had a good chance of ISP cooperation. Is there some legal, warrantless wiretapping I'm not aware of?
Usually they primarily go after torrents, but some websites have caught their ire over the years.
> Considering that Comcast sniffs, intercepts, and injects into HTTP web sites
You think that is bad ? If you get a connection from India's 'premier' public telco BSNL, you'll be treated to ad-injections for random malware straight into your HTTP page.
"Your govt. welcomes your appreciation for its 'top-notch' services."
You think that is bad ? If you get a connection from India's 'premier' public telco BSNL, you'll be treated to ad-injections for random malware straight into your HTTP page.
"Your govt. welcomes your appreciation for its 'top-notch' services."
> Mozilla in November accused ISPs of lying to Congress in order to spread confusion about encrypted DNS. Mozilla's letter to Congress criticized Comcast
> NCTA cable lobby that Comcast belongs to wrote a letter to Congress objecting to Google's plans for encrypted DNS. Comcast gave members of Congress a lobbying presentation that claimed the encrypted-DNS plan would "centraliz[e] a majority of worldwide DNS data with Google". Comcast's lobbying presentation also complained about Mozilla's plan for Firefox.
Compromise, 2020 style: Comcast retains access to it's users DNS data and Mozilla doesn't dogpiled by NCTA-purchased legislators.
> NCTA cable lobby that Comcast belongs to wrote a letter to Congress objecting to Google's plans for encrypted DNS. Comcast gave members of Congress a lobbying presentation that claimed the encrypted-DNS plan would "centraliz[e] a majority of worldwide DNS data with Google". Comcast's lobbying presentation also complained about Mozilla's plan for Firefox.
Compromise, 2020 style: Comcast retains access to it's users DNS data and Mozilla doesn't dogpiled by NCTA-purchased legislators.
This says to me they've cleared the "but what about encrypted DNS in Firefox?" excuse from the boards so they can focus all their lobbying power fighting the only other encrypted DNS implementation (in Chromium.)
Comcast's anti-DoH argument doesn't work with Mozilla as an adversary. The basis of it is squarely anti-Google so having any other reputed organization backing DoH against them sinks that argument.
This is potentially a very evil move and Mozilla is not only complicit but actually aiding. Concerning.
Comcast's anti-DoH argument doesn't work with Mozilla as an adversary. The basis of it is squarely anti-Google so having any other reputed organization backing DoH against them sinks that argument.
This is potentially a very evil move and Mozilla is not only complicit but actually aiding. Concerning.
>Comcast told Ars yesterday that "Firefox users on Xfinity should automatically default to Xfinity resolvers under Mozilla's Trusted Recursive Resolver program, unless they have manually chosen a different resolver, or if DoH is disabled.
How would this work? Is the detection done once, everytime firefox starts, or everytime the network changes? Would you ever get into a situation where you're not using comcast, but are still using comcast dns? eg. you have VPN enabled or your laptop moved to somewhere else.
>Joining Mozilla's program means that Comcast agreed that it won't "retain, sell, or transfer to any third party (except as may be required by law) any personal information, IP addresses, or other user identifiers, or user query patterns from the DNS queries sent from the Firefox browser," along with other requirements.
And how is this enforced? If comcast breaches the agreement, is anyone going to sue them for punitive damages? Given the current state of the US legal system (eg. what happened equifax after the breach), these assurances are worthless to me.
How would this work? Is the detection done once, everytime firefox starts, or everytime the network changes? Would you ever get into a situation where you're not using comcast, but are still using comcast dns? eg. you have VPN enabled or your laptop moved to somewhere else.
>Joining Mozilla's program means that Comcast agreed that it won't "retain, sell, or transfer to any third party (except as may be required by law) any personal information, IP addresses, or other user identifiers, or user query patterns from the DNS queries sent from the Firefox browser," along with other requirements.
And how is this enforced? If comcast breaches the agreement, is anyone going to sue them for punitive damages? Given the current state of the US legal system (eg. what happened equifax after the breach), these assurances are worthless to me.
My understanding is that Comcast signs a legally-binding contract with Mozilla which imposes the requirements on them [0]. This obviously isn't perfect protection, but it substantially increases the risk of failing to adhere to the requirements. Mozilla claims "We intend to publicly document violations of this Policy and take additional actions if necessary." [1]. Presumably the additional actions include suing for damages pursuant to the breach of contract.
[0] https://blog.mozilla.org/netpolicy/2020/02/25/the-facts-mozi... [1] https://wiki.mozilla.org/Security/DOH-resolver-policy#Enforc...
[0] https://blog.mozilla.org/netpolicy/2020/02/25/the-facts-mozi... [1] https://wiki.mozilla.org/Security/DOH-resolver-policy#Enforc...
Surely damages will be approximately zero? There has to be something else to sway Comcast's executives to abide by the contract, surely. Like the CEO agrees to forfeit an amount equal to their previous years total earnings, from all sources, ... that would be an interesting contract!
If Comcast breaks the contract then Mozilla will simply change the default back to Cloudflare DNS.
Out of the pot into the fire.
24 years ago a group of Stanford students started Architext. They took a few million from Kleiner Perkins, called themselves Excite, and started a search engine and internet provider. They were a good, ethical, well ran technology company. Over the years bits and pieces were chopped up and merged and acquired and spun off based on what generated shareholder value. Parts of that old soul live in on now in the current Comcast.
The same thing will happen to Cloudflare. Matthew Prince will move on, or retire, or get hit by a bus. The board will be taken over by an activist investor. It will get merged with ExxonTacoBell, which also now owns the 2nd largest ad network. They will figure out the data gold mine the company built under total ethical pretenses, and the stock price will triple. There isn't a damn thing a single current Cloudflare employee can do to stop it except stop participating in the centralization of the internet behind a single MitM proxy.
24 years ago a group of Stanford students started Architext. They took a few million from Kleiner Perkins, called themselves Excite, and started a search engine and internet provider. They were a good, ethical, well ran technology company. Over the years bits and pieces were chopped up and merged and acquired and spun off based on what generated shareholder value. Parts of that old soul live in on now in the current Comcast.
The same thing will happen to Cloudflare. Matthew Prince will move on, or retire, or get hit by a bus. The board will be taken over by an activist investor. It will get merged with ExxonTacoBell, which also now owns the 2nd largest ad network. They will figure out the data gold mine the company built under total ethical pretenses, and the stock price will triple. There isn't a damn thing a single current Cloudflare employee can do to stop it except stop participating in the centralization of the internet behind a single MitM proxy.
I sure hope that Mozilla writes contracts so that they can't be transferred by sale or merger. That's the most basic protection from your friendly counterparty joining your sworn enemy.
> Presumably the additional actions include suing for damages pursuant to the breach of contract.
Given that the sky is blue, and Comcast is Comcast, Mozilla should have some more funding pretty soon.
Given that the sky is blue, and Comcast is Comcast, Mozilla should have some more funding pretty soon.
I think it's more likely that Mozilla should be dragged into a prolonged and expensive lawsuit that Comcast has the legal might and connections to win pretty soon.
> How would this work?
A 1st draft of the steering mechanism just posted today for comment at https://tools.ietf.org/id/draft-rescorla-doh-cdisco-00.txt
A 1st draft of the steering mechanism just posted today for comment at https://tools.ietf.org/id/draft-rescorla-doh-cdisco-00.txt
I'm confused. For me, a major selling point of DoH is it hides DNS queries from your ISP, which has detailed personal information about you. And if you're locked into Comcast, you're operating with completely eroded trust from the get-go.
Clearly, DNS statistics are extremely valuable to Comcast, or they would not have engaged with Mozilla to get back the data, nor would they have raised hell with Congress.
I would not have expected an organization like Mozilla to sign a data deal with Comcast, even if Comcast is now theoretically restricted on how they use the data.
This is a weak move.
Clearly, DNS statistics are extremely valuable to Comcast, or they would not have engaged with Mozilla to get back the data, nor would they have raised hell with Congress.
I would not have expected an organization like Mozilla to sign a data deal with Comcast, even if Comcast is now theoretically restricted on how they use the data.
This is a weak move.
Mozilla cannot enable one provider by default. People already complained that Cloudflare was initially the only choice.
Users at the moment are expected to choose their provider anyway.
This deal is about Mozilla picking Comcast by default for Comcast customers. This is essentially as if they'd be using the network's default, because Comcast is the network's default already, being what people get via DHCP.
They can always choose a different provider. And Mozilla apparently struck a privacy deal with them too.
Users at the moment are expected to choose their provider anyway.
This deal is about Mozilla picking Comcast by default for Comcast customers. This is essentially as if they'd be using the network's default, because Comcast is the network's default already, being what people get via DHCP.
They can always choose a different provider. And Mozilla apparently struck a privacy deal with them too.
I understand the arrangement. From a Comcast user’s perspective, very little has changed, depending on how much trust you assign to a “we promise” privacy agreement. Are Comcast users better off than default? Yes. But decoupling DNS from ISPs which sit in such a privileged position is, for me, 85% of the threat model.
I’d like to read more about how the choice will be presented to users, beyond about:config. I’d also like to understand more the community’s reaction to Cloudflare default.
What if there was a round robin setup between neutral operators? Pairing Comcast users to Comcast just seems like a wtf move.
I’d like to read more about how the choice will be presented to users, beyond about:config. I’d also like to understand more the community’s reaction to Cloudflare default.
What if there was a round robin setup between neutral operators? Pairing Comcast users to Comcast just seems like a wtf move.
Having just done a Firefox install recently, the UX when prompted to enable DoH on first run did not set the expectation of choosing a provider.
Is there something you can point to that speaks to that expectation?
Is there something you can point to that speaks to that expectation?
Everyone is complaining:
- No DOH: "DNS is trivial to snoop"
- DOH with Cloudflair: "DNS is not longer distributed"
- DOH with ISPs: "Great, the ISPs get the data again"
Well, guess what? With the current changes we get rid of the arbitrary snooper problem while preserving DNS as a distributed service: not bad, not amazing, but strictly better than before this all began!
Of course ISPs are sketch, and Comcast in particular, but I rather have multiple providers to play off against each other. The upcoming IETF draft they mention would also restore the ability of network admins to adjust the default DOH (just like with regular DNS)---also good.
Actual anatomized and distributed DNS querying would require vastly different technology that anything being proposed in these comments.
- No DOH: "DNS is trivial to snoop"
- DOH with Cloudflair: "DNS is not longer distributed"
- DOH with ISPs: "Great, the ISPs get the data again"
Well, guess what? With the current changes we get rid of the arbitrary snooper problem while preserving DNS as a distributed service: not bad, not amazing, but strictly better than before this all began!
Of course ISPs are sketch, and Comcast in particular, but I rather have multiple providers to play off against each other. The upcoming IETF draft they mention would also restore the ability of network admins to adjust the default DOH (just like with regular DNS)---also good.
Actual anatomized and distributed DNS querying would require vastly different technology that anything being proposed in these comments.
> Actual anatomized and distributed DNS querying would require vastly different technology that anything being proposed in these comments.
I hope the everything should be solved with blockchain! crowd don't get any ideas.
I hope the everything should be solved with blockchain! crowd don't get any ideas.
I don't know how we can have privacy and Comcast in one sentence. We have a saying where I come from that translates roughly to "Putting the Wolf to Guard the Sheep".
If you don't have any other option but to be with comcast my recommendation is to run Pi-Hole + DoH.
If you don't have any other option but to be with comcast my recommendation is to run Pi-Hole + DoH.
I do run PiHole. Any tips on how to do the `DoH` side of the equation?
edit: this looks to do the trick: https://docs.pi-hole.net/guides/dns-over-https/
edit: this looks to do the trick: https://docs.pi-hole.net/guides/dns-over-https/
Sorry, just saw the reply. Yes, cloudflared is what I do at the moment.
Tin foil hat time but I can't help feeling a lot of things promoted as privacy solutions like VPNs and DoH are just aggregating data in a handful of locations so it is easier to intercept. Sure they have privacy policies but are they worth the paper they are written on when state actors are bound by a totally different set of rules?
I recently changed my local dnssec resolver to forward to quad9 and cloudflare using DoT because I was sick of the high latency with DNS resolving on boot. I would forward to my own DoT server if there was some authentication built into it and I could deny other traffic. But I have gone from dns requests being aggregated at my ISP for easy inspection my the democratically elected government of my own country, to a my own dns resolver which while it isn't aggregated is still easy enough to intercept under warrant for local law enforcement (which I generally support) to aggregating my queries in a few logs which are likely in foreign countries where I have no say in how they are used or abused. I am not sure what problem we are trying to solve with this technology.
I recently changed my local dnssec resolver to forward to quad9 and cloudflare using DoT because I was sick of the high latency with DNS resolving on boot. I would forward to my own DoT server if there was some authentication built into it and I could deny other traffic. But I have gone from dns requests being aggregated at my ISP for easy inspection my the democratically elected government of my own country, to a my own dns resolver which while it isn't aggregated is still easy enough to intercept under warrant for local law enforcement (which I generally support) to aggregating my queries in a few logs which are likely in foreign countries where I have no say in how they are used or abused. I am not sure what problem we are trying to solve with this technology.
Indeed. As it stands, trust is merely being shifted from one set of parties (ISPs) to another set (big DNS/CDN providers). Apparently that's attractive enough to a lot of people who prefer to send their queries to a foreign company than to their own ISP, but has it changed anything fundamental?
The hope is that it's a change away from an untrustworthy provider, toward a trusted provider. Not sure if it counts as being a fundamental change, but it seems worth doing.
If you're in the US, you can't trust your ISP (most of them farm DNS data), and shaking your fist at the clouds won't change that. If you don't Cloudflare or Quad9, stand up your own DoH server somewhere; it's trivial to do.
Actually, most Quad9 users (in so far as we know, since we only know from those who contact us) are already running their own local caching resolver, and many of them are doing QNAME minimization, which is excellent. They use us to keep their cache misses from going in the clear, and because many of them have small enough user populations that their resolver still exposes individual user behavior.
At home I've got a pihole handling my DNS, including using DoH to Cloudflare.
I assume that this configuration is superior to whatever FF is doing natively, and I should disable FF's DoH support?
I assume that this configuration is superior to whatever FF is doing natively, and I should disable FF's DoH support?
> and I should disable FF's DoH support?
yeah there is no reason (I can think of) why you need FF DoH with your setup. In fact if you were to enable DoH in FF it would bypass your pie-hole - so you most certainly want to avoid that.
my setup looks pretty much like yours with some additional /etc/hosts blocking[1] on the client just to avoid the round-trip to the pie-hole. it's also a double insulation (but it's more of a performance reason than bc of paranoia). I found that switching off ipv6 dns resolution in FF (`network.dns.disableIPv6` in about:config) has tremendously sped up my DNS lookups in FF (though I haven't had time to analyze why).
in case you're worried about homograph phishing attacks you could also add a regex to your pie-hole's dnsmasq (not sure what piehole uses but I know it has a fork of dnsmask that supports regex) so that punicode domains (any domains matching "xn--") are sinkholed to 0.0.0.0 as well.
[1] https://github.com/StevenBlack/hosts
yeah there is no reason (I can think of) why you need FF DoH with your setup. In fact if you were to enable DoH in FF it would bypass your pie-hole - so you most certainly want to avoid that.
my setup looks pretty much like yours with some additional /etc/hosts blocking[1] on the client just to avoid the round-trip to the pie-hole. it's also a double insulation (but it's more of a performance reason than bc of paranoia). I found that switching off ipv6 dns resolution in FF (`network.dns.disableIPv6` in about:config) has tremendously sped up my DNS lookups in FF (though I haven't had time to analyze why).
in case you're worried about homograph phishing attacks you could also add a regex to your pie-hole's dnsmasq (not sure what piehole uses but I know it has a fork of dnsmask that supports regex) so that punicode domains (any domains matching "xn--") are sinkholed to 0.0.0.0 as well.
[1] https://github.com/StevenBlack/hosts
If you do not disable Firefox's DoH support, it will by pass your Pi-Hole entirely. So you'd lose all the benefits of that and be limited to just the protections Firefox provides (which are great, to be clear. Just not as good as a well-sourced Pi0Hole)
This is why I absolutely despise DoH. SysAdmins have no direct control over it. In my organization we have blocked direct IP access from userspace VLAN's to all known public DNS servers thus forcing all clients to rely on the company DNS servers, which is not the most ideal way to do things.
Don't you just set a canary domain - https://support.mozilla.org/en-US/kb/canary-domain-use-appli... - and then it's disabled for your network?
Again not the most ideal way to do things and Mozilla is doing a different approach to Chrome and Edge. and also a concern is that malware can use DoH to retrieve data without logging suspicious DNS queries on Firewall DNS logs which are monitored to highlight of new domains that have not been pre-approved.
DNS should be something that is handled by the OS. I favor DoT which is secure and practical over DoH.
DNS should be something that is handled by the OS. I favor DoT which is secure and practical over DoH.
Actually, in that case, adding the canary domain to your existing Microsoft DNS servers probably IS the most ideal way to disable Firefox's DoH support.
Alternatively, you can roll out a Group Policy or use Mozilla's "Enterprise" policies to do it.
Hopefully you're also blocking 53/TCP and 53/UDP outbound (except from your internal DNS servers).
Alternatively, you can roll out a Group Policy or use Mozilla's "Enterprise" policies to do it.
Hopefully you're also blocking 53/TCP and 53/UDP outbound (except from your internal DNS servers).
> malware can use DoH to retrieve data without logging suspicious DNS queries on Firewall DNS logs
Malware can already query IPs of its choice to learn about other IPs it should contact. DoH doesn't let it do anything new.
Malware can already query IPs of its choice to learn about other IPs it should contact. DoH doesn't let it do anything new.
In a corporate network, it's pretty common to block all outgoing DNS traffic (53/TCP and 53/UDP), except from the company's DNS servers.
In that case, DoH does let malware do something new -- block the company's existing DNS policies, quert logging, and security monitoring!
In that case, DoH does let malware do something new -- block the company's existing DNS policies, quert logging, and security monitoring!
DoH is a protocol for using HTTPS to learn what IPs to talk to.
Malware does not need DoH to do this. They can simply run an ordinary HTTPS server with a self-signed cert on an arbitrary IP, with a simple JSON-based or whatever protocol, and have support for that in their client.
Malware does not need DoH to do this. They can simply run an ordinary HTTPS server with a self-signed cert on an arbitrary IP, with a simple JSON-based or whatever protocol, and have support for that in their client.
> Malware does not need DoH to do this.
Yes, you're right, of course.
There are any number of things that malware can do. Most of it doesn't, however, and can either be stopped completely or, at the least, detected quite easily using some basic techniques.
Yes, you're right, of course.
There are any number of things that malware can do. Most of it doesn't, however, and can either be stopped completely or, at the least, detected quite easily using some basic techniques.
Not really.
Malware could always use some proprietary wrapping in TLS to hide name resolution.
Domain fronting through Azure is still a thing for instance.
Malware could always use some proprietary wrapping in TLS to hide name resolution.
Domain fronting through Azure is still a thing for instance.
[deleted]
Why do you want them to rely on the company DNS servers?
1. Internal names won't resolve if a client is using, for example, 1.1 as their DNS server (breaking, among other things, logging on to an Active Directory domain!)
2. Many companies have established DNS logging and monitoring in place for security.
2. Many companies have established DNS logging and monitoring in place for security.
Active Directory, Internal Domains, monitoring, blocking, etc.
Policy.
Not sending my private browsing data/DNS history to a third party (Cloudflare, Google etc.)
Not sending my private browsing data/DNS history to a third party (Cloudflare, Google etc.)
Strictly speaking you should run DoH to your pihole, to prevent local-network attacks, but that's a fairly minor issue and supposedly (epistemic status: outright hearsay) DoH is intentionally difficult to configure correctly (ie to a non-mozilla-approved server).
No need to disable Firefox's DoH support, just point it at your pihole as the DoH provider.
How do you do that without accessing each client, like how do I disable DoH on my home network for all devices without having to access all devices?
Well, there's the use-application-dns.net canary domain (if it resolves to a response code other than NOERROR or to NOERROR but lacks both A and AAAA records), Firefox will fall back to using the OS DNS. Otherwise it's up to the application, just like with DNS it's always been the application's choice whether to use the OS-provided DNS or to use their own (eg via proxy or VPN).
In short, you can't. But you never could stop an application from tunneling DNS over some other channel before, so that's nothing new.
In short, you can't. But you never could stop an application from tunneling DNS over some other channel before, so that's nothing new.
It’s basically the same minus the filtering.
Cloudflare are getting all your browsing info from DNS.
Your ISP from SNI.
Cloudflare are getting all your browsing info from DNS.
Your ISP from SNI.
This is a net increase in privacy for most people on Comcast networks, I'm glad to see Mozilla striking a deal like this -- especially with the privacy agreements Comcast is signing. But you should still switch your encrypted DNS provider to someone else like Cloudflare or similar.
In short, good move, but you personally can make better moves than trusting Comcast.
In short, good move, but you personally can make better moves than trusting Comcast.
> "Adding ISPs in the TRR program paves the way for providing customers with the security of trusted DNS resolution, while also offering the benefits of a resolver provided by their ISP such as parental control services and better optimized, localized results," the announcement said.
What? No! Why would DNS have "optimized, localized results"?
What? No! Why would DNS have "optimized, localized results"?
> Why would DNS have "optimized, localized results"?
Any content that is CDN-based (which is most content) dynamically responds to DNS queries based on network and geographic location - to support CDN localization. In this way, Akamai for example knows the end user is in Boston on a Comcast network and will send the recursive DNS server a dynamic response that points to a directly-connected local-to-Boston content server.
Any content that is CDN-based (which is most content) dynamically responds to DNS queries based on network and geographic location - to support CDN localization. In this way, Akamai for example knows the end user is in Boston on a Comcast network and will send the recursive DNS server a dynamic response that points to a directly-connected local-to-Boston content server.
Any serious CDN is doing that with anycast, not with geodns which has tons of drawbacks.
That's not the case - many use hybrid approaches, and GeoDNS is still used heavily in the industry. Anycast sucks too.
You're right. This is really cool tech.
https://www.cloudflare.com/learning/cdn/glossary/anycast-net...
oh, I didn't know that. thanks!!
Consider a hostname that can map to different, widely geographically separated, IPs. You probably want the one with the lowest latency, which is likely to be the closest-located one. Not guaranteed, of course.
You should just use anycast for that instead of trying to shoehorn it in with DNS trickery.
It's interesting that the DNS-based solution is considered "trickery", when I don't really know anyone except for very networking-focused people who can explain how anycast works to achieve the same thing. While BGP is definitely not magic, it feels way more magic to me than DNS.
The DNS-based solution, in comparison, seems way simpler to explain: get general location of IP of requester, send back the IP of a server in a DC closest to that location.
The DNS-based solution, in comparison, seems way simpler to explain: get general location of IP of requester, send back the IP of a server in a DC closest to that location.
I'm only calling it "trickery" because DNS isn't supposed to work that way. The entire systems structure of authoritative resolvers to recursive resolvers to cachers depends on a record being the same regardless of external factors like location or source IP. Meanwhile, anycast was literally designed for this purpose. And it's been around since 1993.[1]
>explain how anycast works
Advertise your IP space in multiple locations. Traffic will come to the location closest[2] to it. There's nothing particularly complicated about it.
How the routing end of it works is up to the specific protocol you're using and whether this is external or internal... so it depends on BGP on the Internet (in the case we're talking about here), but you can also anycast on your internal network, and the routing will be handled by OSPF/IS-IS/whatever.
Further, effectively every large service online today uses anycast clusters. Do you think 8.8.8.8 goes to a single server? Cloudflare IPs? Google IPs? It's all anycast.
[1]https://tools.ietf.org/html/rfc1546
[2]"closest" is subjective, it's not necessarily the geographically closest, but the "best" path. This is decided by the specific protocol you're using, but generally you can assign costs/metrics to various routes to prioritize/deprioritize them. This is your netops team's problem, not yours, so for all intents and purposes you'll always get the "best" path.
>explain how anycast works
Advertise your IP space in multiple locations. Traffic will come to the location closest[2] to it. There's nothing particularly complicated about it.
How the routing end of it works is up to the specific protocol you're using and whether this is external or internal... so it depends on BGP on the Internet (in the case we're talking about here), but you can also anycast on your internal network, and the routing will be handled by OSPF/IS-IS/whatever.
Further, effectively every large service online today uses anycast clusters. Do you think 8.8.8.8 goes to a single server? Cloudflare IPs? Google IPs? It's all anycast.
[1]https://tools.ietf.org/html/rfc1546
[2]"closest" is subjective, it's not necessarily the geographically closest, but the "best" path. This is decided by the specific protocol you're using, but generally you can assign costs/metrics to various routes to prioritize/deprioritize them. This is your netops team's problem, not yours, so for all intents and purposes you'll always get the "best" path.
> I don't really know anyone except for very networking-focused people who can explain how anycast works to achieve the same thing.
I'm not familiar with the crap the IETF et al have smeared on it, but in principle, it's very simple: each network vertex maintains a mapping from (blocks of) addresses to edges (and the neighbors on the other ends of each edge) moving closer to that address, such that the directed edges for any given address form a rooted directed graph (ie a tree) where the root is the host with that address. Anycast just relaxes the requirement that the latent directed graph for a anycast address form a single tree with a single root, and instead allows multiple trees and roots forming a exact cover of the network. Updates still try to find the neighbor with the shortest round-trip time, but now there can be multiple network vertices with RTT=0 (ie multiple servers with the same anycast address).
I'm not familiar with the crap the IETF et al have smeared on it, but in principle, it's very simple: each network vertex maintains a mapping from (blocks of) addresses to edges (and the neighbors on the other ends of each edge) moving closer to that address, such that the directed edges for any given address form a rooted directed graph (ie a tree) where the root is the host with that address. Anycast just relaxes the requirement that the latent directed graph for a anycast address form a single tree with a single root, and instead allows multiple trees and roots forming a exact cover of the network. Updates still try to find the neighbor with the shortest round-trip time, but now there can be multiple network vertices with RTT=0 (ie multiple servers with the same anycast address).
> but in principle, it's very simple
Er, very simple provided that you're already doing doing unicast routing, that is. There are plenty of complexities, they just almost all already exist even without any anycast addresses.
Er, very simple provided that you're already doing doing unicast routing, that is. There are plenty of complexities, they just almost all already exist even without any anycast addresses.
Exactly, if you're already doing routing/advertisements anycast just boils down to "advertise from multiple places at once" and it basically just works. If you're not doing your own routing this isn't a conversation for you.
Running any connection/TCP based service on an anycast IP seems to require a large and effective network operations team.
Dealing with BGP route flapping, single connection traffic being split between different servers, is a difficult problem that requires extensive relationships among other network operators.
Implementing EDNS Client Subnet, on the other hand is pretty simple.
Dealing with BGP route flapping, single connection traffic being split between different servers, is a difficult problem that requires extensive relationships among other network operators.
Implementing EDNS Client Subnet, on the other hand is pretty simple.
The joke is you have to use anycast if you want any redundancy in your network. If you advertise your address space on a single upstream you're praying that they never have any issues.
>BGP route flapping
So what? Your traffic will go over to one of the other locations you're advertising on, and you keep ticking along merrily, with some extra latency.
>single connection traffic being split between different servers
That shouldn't happen. Keep in mind traffic just gets driven to your router, from there you load balance as you see fit. Unless you mean split between different ingress points -- TCP will happily renegotiate with very little overhead, if you're getting more serious issues your application sucks and should should be more resilient to connection state changes (if this is causing you problems, you're probably also dropping users who switch mobile to wifi...). Not to mention that these kinds of path cost updates are pretty rare on the real internet, you're not going to see this more often than once a month maybe.
>BGP route flapping
So what? Your traffic will go over to one of the other locations you're advertising on, and you keep ticking along merrily, with some extra latency.
>single connection traffic being split between different servers
That shouldn't happen. Keep in mind traffic just gets driven to your router, from there you load balance as you see fit. Unless you mean split between different ingress points -- TCP will happily renegotiate with very little overhead, if you're getting more serious issues your application sucks and should should be more resilient to connection state changes (if this is causing you problems, you're probably also dropping users who switch mobile to wifi...). Not to mention that these kinds of path cost updates are pretty rare on the real internet, you're not going to see this more often than once a month maybe.
I think you are overestimating the resilience of TCP, yes it will probably eventually renegotiate, but you create a horrible user experience.
Have a read of some of the cloudflare engineering blogs, they give some sense of the technical challenges of deploying anycast.
I wouldnt recommend a company deploy connection based anycast services, unless they are prepared to spend a lot on the engineering challenges.
> Since packets will follow the shortest path, if a particular path is withdrawn then packets will find their way to the next shortest available route. For simple protocols like UDP that don't maintain state, Anycast is ideal and it has been used widely to load balance DNS for some time. At CloudFlare, we've done a significant amount of engineering to allow TCP to run across Anycast without flapping. This involves carefully adjusting routes in order to get optimal routing and also adjusting the way we handle protocol negotiation itself. While more complex to maintain than a Unicast network, the benefit is we can lose an entire data center and packets flow to the next closest facility without anyone noticing and hiccup.
https://blog.cloudflare.com/cloudflares-architecture-elimina...
> WARP was experiencing so many failures because devices were switching servers much more often than we expected. If you recall, our ECMP router configuration uses a combination of (Source IP, Source Port, Destination IP, Destination Port) to match a packet to a server. Destination IP doesn’t generally change, WARP clients are always connecting to the same Anycast addresses. Similarly, Destination Port doesn’t change, we always listen on the same port for WARP traffic. The other two values, Source IP and Source Port, were changing much more frequently than we had planned.
https://blog.cloudflare.com/warp-technical-challenges/
Have a read of some of the cloudflare engineering blogs, they give some sense of the technical challenges of deploying anycast.
I wouldnt recommend a company deploy connection based anycast services, unless they are prepared to spend a lot on the engineering challenges.
> Since packets will follow the shortest path, if a particular path is withdrawn then packets will find their way to the next shortest available route. For simple protocols like UDP that don't maintain state, Anycast is ideal and it has been used widely to load balance DNS for some time. At CloudFlare, we've done a significant amount of engineering to allow TCP to run across Anycast without flapping. This involves carefully adjusting routes in order to get optimal routing and also adjusting the way we handle protocol negotiation itself. While more complex to maintain than a Unicast network, the benefit is we can lose an entire data center and packets flow to the next closest facility without anyone noticing and hiccup.
https://blog.cloudflare.com/cloudflares-architecture-elimina...
> WARP was experiencing so many failures because devices were switching servers much more often than we expected. If you recall, our ECMP router configuration uses a combination of (Source IP, Source Port, Destination IP, Destination Port) to match a packet to a server. Destination IP doesn’t generally change, WARP clients are always connecting to the same Anycast addresses. Similarly, Destination Port doesn’t change, we always listen on the same port for WARP traffic. The other two values, Source IP and Source Port, were changing much more frequently than we had planned.
https://blog.cloudflare.com/warp-technical-challenges/
Is this the same Comcast that:
1) created an RFC to inject JavaScript into non-TLS pages 2) created an RFC to intercept failed DNS lookups / connections with their own error page
Both of which I reported to the FCC as MITM attacks. Comcast followed up referenced the bunk RFCs saying "it's fine."
1) created an RFC to inject JavaScript into non-TLS pages 2) created an RFC to intercept failed DNS lookups / connections with their own error page
Both of which I reported to the FCC as MITM attacks. Comcast followed up referenced the bunk RFCs saying "it's fine."
Weird that a network or OS level concern is being moved to the application layer. But considering that trust in all the other layers has been lost, from the ISP to the OS (specifically Windows), maybe this makes sense.
The whole newroling stack needs encryption etc.
This is one reason why you see everything moving of the layers.
The other is, that Http servers are widely available, tested and easily scalable
Would be Comcast able to intercept the TLS SNI requests anyway and see at least were the traffic is directed?
"Let's encrypt DNS queries but send them to the ISP which can associate the query with subscriber info!"
Regardless of how good this deal actually is, I have a knee-jerk reaction to anything regarding Comcast.
I would not trust them with anything whatsoever.
Mozilla even if they made a good decision here seems to be taking a step backwards just by virtue of associating with Comcast in any slight form whatsoever.
I would not trust them with anything whatsoever.
Mozilla even if they made a good decision here seems to be taking a step backwards just by virtue of associating with Comcast in any slight form whatsoever.
How does FF know my DNS is a Comcast-provided one? Is there an IP list kept inside of browsers and updated?
Comcast's ASNs and networks are documented in ARIN's WHOIS database and various route registries. Hell, Comcast probably publishes a list on their own web site.
So, yeah, Mozilla can easily determine if a user is on the Comcast network just from their IP address.
Also, while Comcast actually has a bunch of DNS servers spread across the country, I believe that nowadays they're mostly "promoting" the use of 75.75.75.75 and 75.75.76.76 (which, AFAIK, are anycasted and direct end users to their "local" DNS servers).
So, yeah, Mozilla can easily determine if a user is on the Comcast network just from their IP address.
Also, while Comcast actually has a bunch of DNS servers spread across the country, I believe that nowadays they're mostly "promoting" the use of 75.75.75.75 and 75.75.76.76 (which, AFAIK, are anycasted and direct end users to their "local" DNS servers).
> So, yeah, Mozilla can easily determine if a user is on the Comcast network just from their IP address.
I mean, how can the determine it's a Comcast DNS server configured on my network? I might be a Comcast customer w/ a custom DNS server configured. If it's a fixed IP check, I suppose that list is in the browser.
I mean, how can the determine it's a Comcast DNS server configured on my network? I might be a Comcast customer w/ a custom DNS server configured. If it's a fixed IP check, I suppose that list is in the browser.
What makes you think they are doing that? To jlgaddis's point, they are likely checking what IP address you are coming from over the public Internet. They can see who operates it, and knows that it's Comcast. Then they change the options in your browser.
A draft RFC was published today: https://news.ycombinator.com/item?id=23644068
I’ve never understood the purpose of DOH. It doesn’t really hide your traffic from any party, does it?
It encrypts your DNS traffic over the public wire in a way that only the DOH endpoint operator can decrypt, preventing plaintext interception/modification attacks by unauthorized malicious actors positioned between you and the DOH endpoint
It represents your DNS traffic over the wire as encrypted HTTPS traffic, which decreases the effectiveness of deep packet inspection and traffic shaping systems operated by some network providers.
When hosted at heavily-used CDN endpoints that receive other (non-DOH) HTTPS traffic, it requires a network provider who wishes for whatever reason to block your DNS traffic to block all HTTPS traffic to all CDN endpoints.
It represents your DNS traffic over the wire as encrypted HTTPS traffic, which decreases the effectiveness of deep packet inspection and traffic shaping systems operated by some network providers.
When hosted at heavily-used CDN endpoints that receive other (non-DOH) HTTPS traffic, it requires a network provider who wishes for whatever reason to block your DNS traffic to block all HTTPS traffic to all CDN endpoints.
OK sure but what good is that when my next TCP/UDP activity after a dns lookup is to actually connect to that host? The upstream ISP knows exactly where you are going right? They can store and reverse that info and do with it as they wish.
An IP address is often less specific than a hostname, and will become less useful over time due to IPv4 address space exhaustion and concentration of internet services among a small number of cloud providers. Widespread use of DOH therefore makes it harder for ISPs and middleboxes to interfere without collateral damage. It's far from perfect, but it'll help.
You might want to read this study on that topic:
"What can you learn from an IP?" https://irtf.org/anrw/2019/slides-anrw19-final44.pdf
"What can you learn from an IP?" https://irtf.org/anrw/2019/slides-anrw19-final44.pdf
Interesting reference, thanks. I’m surprised there are so many site-unique IPs. Fingerprinting is less compelling in the case of blocking (I think?) but is certainly still a privacy problem.
Ultimately, all security is about raising the cost for attackers, and I think it’s a good thing that DOH will make middleboxes more expensive and less accurate. It would be a mistake to pitch it as being even close to a perfect solution to any problem, though.
Ultimately, all security is about raising the cost for attackers, and I think it’s a good thing that DOH will make middleboxes more expensive and less accurate. It would be a mistake to pitch it as being even close to a perfect solution to any problem, though.
Provided we get eSNI everywhere too, or it is easy for middlemen to sniff out your actual hostname even though the IP may be shared with thousands/millions of other hosts.
Yep, I’m assuming that will happen eventually.
That's unclear to me. By running DoH, Comcast gets to spy Comcast customers. But Comcast of course can already see what IP addresses talk with Comcast customers, so what changes really? That the spying might become marginally easier maybe?
[deleted]
If you can run your own local, recursive resolver, you should.
Why can’t mozilla roll out a encrypted dns service in a form of a browser setting or plugin? Does the browser not have control over how the dns is resolved ?
So Mozilla wanted encrypted DNS Comcast did not. Now they made a deal. Comcast agreed in writing not to monitor DNS. So what did Mozilla give up to make this deal? Mozilla wont encrypt DNS on Comcast connections? Something does not add up.
* Comcast sniffs / records / tracks their user's DNS traffic
* Mozilla announced they would enable DoH by default, to protect end user's DNS data from shady ISPs like Comcast
* Comcast then raised hell about Mozilla's decision (presumably because they would no longer have access to this data)
* Now, Comcast and Mozilla come to some sort of agreement which effectively restores Comcast's access to their customer's DNS traffic?
---
I'm really confused why Mozilla would agree to this. I really hope this isn't one of the ways they're exploring to "diversify" their revenue streams but, in the last few years, Mozilla has made a lot of decisions that I don't agree with so I suppose I really wouldn't be all that surprised.