Firefox 87 trims HTTP Referrers by default to protect user privacy(blog.mozilla.org)
blog.mozilla.org
Firefox 87 trims HTTP Referrers by default to protect user privacy
https://blog.mozilla.org/security/2021/03/22/firefox-87-trims-http-referrers-by-default-to-protect-user-privacy/
262 comments
Your FYI is also useful for the contrapositive of that feeling, which we often see on HN: "Look how Firefox is leading privacy improvements and how much more privacy forward it is than Chrome".
You see, when Firefox improves privacy it's for the good of users, but when Chrome improves privacy it's because Google can track you just fine despite the change and they want to give competing ad networks a kicking.
/s ?
/s ?
I mean, that can be true, no? Google does have many ways of tracking you and blocking others from doing it while still doing it themselves certainly helps them maintain their moat.
AMP is a pretty good example of this. Good for Google, reduces tracking (by anyone but Google), bad for everyone else.
>You see, when Firefox improves privacy it's for the good of users, but when Chrome improves privacy it's because Google can track you just fine despite the change and they want to give competing ad networks a kicking.
/s ?
Given that Firefox improving privacy generally doesn't draw antitrust scrutiny, where is the problem with that statement?
>The questions from Justice Department investigators have touched on how Chrome policies, including those related to cookies, affect the ad and news industries, four people said.
Investigators are asking whether Google is using Chrome, which has 60% global market share, to reduce competition by preventing rival ad companies from tracking users through cookies while leaving loopholes for it to gather data with cookies, analytics tools and other sources, the sources added.
https://www.reuters.com/article/us-tech-antitrust-google-exc...
If Google were truly improving privacy, one would expect their changes to decrease their own ability to track users, not just their competitors.
/s ?
Given that Firefox improving privacy generally doesn't draw antitrust scrutiny, where is the problem with that statement?
>The questions from Justice Department investigators have touched on how Chrome policies, including those related to cookies, affect the ad and news industries, four people said.
Investigators are asking whether Google is using Chrome, which has 60% global market share, to reduce competition by preventing rival ad companies from tracking users through cookies while leaving loopholes for it to gather data with cookies, analytics tools and other sources, the sources added.
https://www.reuters.com/article/us-tech-antitrust-google-exc...
If Google were truly improving privacy, one would expect their changes to decrease their own ability to track users, not just their competitors.
[deleted]
Both browsers are funded mainly by the same source, so in the end changes in any of them is Google’s move anyway.
True, but this case contains a little more complexity. Namely:
1. Chrome, as the dominant browser, drives developer behaviour (devs need to change their websites when Chrome breaks compat). Firefox doesn't have this luxury, so some features like this must necessarily be lead by Chrome in the monopolistic browser world we live in.
2. Referers allow websites to individually track users better, which is bad for user privacy if you are concerned about entities such as Facebook, etc. tracking you. However, those entities are Google's competitors; for Google, Chrome does the tracking, so they have no need of the Referer.
Removing it is "good" for user privacy w.r.t. Google's competitors, and simultaneously good for Google (disadvantaging such competitors).
1. Chrome, as the dominant browser, drives developer behaviour (devs need to change their websites when Chrome breaks compat). Firefox doesn't have this luxury, so some features like this must necessarily be lead by Chrome in the monopolistic browser world we live in.
2. Referers allow websites to individually track users better, which is bad for user privacy if you are concerned about entities such as Facebook, etc. tracking you. However, those entities are Google's competitors; for Google, Chrome does the tracking, so they have no need of the Referer.
Removing it is "good" for user privacy w.r.t. Google's competitors, and simultaneously good for Google (disadvantaging such competitors).
It seems that the solution to the “complex” scenario you’re describing is simple: use Firefox instead of Chrome.
It’s only simple in the sense that it takes few words to say, but then world peace is also simple: just stop having wars.
Do you really think the two are in any way comparable?
It's a web browser.
It's a web browser.
It's an analogy.
Still easier said than done. There are still many websites that break in weird ways because the developers don’t care about Firefox.
I myself religiously use Firefox by default and only use Chrome if I bump into some websites don’t work in Firefox. But my kid’s teachers often send links that my kid needs to use for school work. And a lot of them are educational websites that don’t work in Firefox. I can’t be home all day to provide tech support. So having Firefox as the default browser is simply a no-go. Guess what wins, between Chrome and Firefox?
I myself religiously use Firefox by default and only use Chrome if I bump into some websites don’t work in Firefox. But my kid’s teachers often send links that my kid needs to use for school work. And a lot of them are educational websites that don’t work in Firefox. I can’t be home all day to provide tech support. So having Firefox as the default browser is simply a no-go. Guess what wins, between Chrome and Firefox?
> Still easier said than done. There are still many websites that break in weird ways because the developers don’t care about Firefox.
I read a lot of people saying this, but I use Firefox and I don't run on this issues, except when I land by accident on some weird dark pages (mainly porn pages)
As developer, I develop on Firefox, and sometimes do a quick test on Chromiun/Edge/Chrome to see if something it's break or bugged on Chromiun/Edge/Chrome web engines.
I read a lot of people saying this, but I use Firefox and I don't run on this issues, except when I land by accident on some weird dark pages (mainly porn pages)
As developer, I develop on Firefox, and sometimes do a quick test on Chromiun/Edge/Chrome to see if something it's break or bugged on Chromiun/Edge/Chrome web engines.
Literally nothing about the scenario described above will change, except that it will give Firefox the option to make the change before Chrome does.
Your answer takes Google's POV into account (the "why"), but the GP was mentioning the end result of firms' intentions (the "what"). In other words, it doesn't matter why google needs to or wants to implement some features. What matters is that those features are implemented in Chrome and yet they don't get praised on HN. But if Mozilla adds the same features, HN folks go crazy about it.
I'm a happy Firefox user too, but I don't want to believe that Firefox is the only browser that cares about user privacy.
I'm a happy Firefox user too, but I don't want to believe that Firefox is the only browser that cares about user privacy.
> I don't want to believe that Firefox is the only browser that cares about user privacy.
I don't either, but I don't believe HN users praising Chrome is going to help with that situation...
I don't either, but I don't believe HN users praising Chrome is going to help with that situation...
I think the parent comment's point is that Firefox is worth celebrating because they're making this (and other changes) with no ulterior motive other than championing user privacy. But with Chrome, the user privacy improvement is a biproduct of Google making profit-oriented decisions, and not the focal point of their change.
I'll happily celebrate Google making a privacy-oriented change that's actually intended and not a bi-product, but this isn't such a case.
I'll happily celebrate Google making a privacy-oriented change that's actually intended and not a bi-product, but this isn't such a case.
We can guess Google's motivation but no source.
> for Google, Chrome does the tracking, so they have no need of the Referer.
Source?
Source?
[deleted]
[deleted]
Afaik, Chrome does not the tracking. Or do you have some source?
In the event that such behavior would occur, it can be detected by sniffing your packets, and people would very soon speak about it.
In the event that such behavior would occur, it can be detected by sniffing your packets, and people would very soon speak about it.
Chrome does a lot of tracking. This seems obvious enough that I would reply by asking: do you have a source demonstrating that it does not track?
However there are many many many sources out there documenting Chrome's comprehensive tracking:
- Google's own ToS is probably too long to read, but it lists ways in which you consent to be tracked by the browser
- Google's proposals to the W3C for adding new ad-tracking tech to web standards go into detail on Chrome's own mechanisms for doing this tracking as a PoC example [0]
- Chrome isn't technically available on iOS; they are forced to use Safari's engine under the hood of iOS Chrome, which restricts in some ways the amount of tracking the browser can do. Despite this, the list of data tracked by iOS Chrome is listed on the AppStore download page and IT IS LONG.
Finally, you mention packet sniffing. In actual fact, direct packet sniffing has been made non-trivial by cert-pinning. This doesn't however prevent checking the frequency and destination of app phone-home requests, and yes, Chrome does it a lot. You can see this yourself if you use common user firewall tools such as LittleSnitch/LuLu/OpenSnitch/etc.
[0] https://github.com/WICG/floc#qualifying-users-for-whom-a-coh...
However there are many many many sources out there documenting Chrome's comprehensive tracking:
- Google's own ToS is probably too long to read, but it lists ways in which you consent to be tracked by the browser
- Google's proposals to the W3C for adding new ad-tracking tech to web standards go into detail on Chrome's own mechanisms for doing this tracking as a PoC example [0]
- Chrome isn't technically available on iOS; they are forced to use Safari's engine under the hood of iOS Chrome, which restricts in some ways the amount of tracking the browser can do. Despite this, the list of data tracked by iOS Chrome is listed on the AppStore download page and IT IS LONG.
Finally, you mention packet sniffing. In actual fact, direct packet sniffing has been made non-trivial by cert-pinning. This doesn't however prevent checking the frequency and destination of app phone-home requests, and yes, Chrome does it a lot. You can see this yourself if you use common user firewall tools such as LittleSnitch/LuLu/OpenSnitch/etc.
[0] https://github.com/WICG/floc#qualifying-users-for-whom-a-coh...
Ok it looks like you're right that by default a lot of tracking is done.
However the link you provided shows that in "Google Activity Controls" and "Google Ad Settings" the user has the possibility to completely opt-out of this tracking.
If privacy-minded users are fully aware of all of these options, that may be fine for them? (spoiler: I was NOT aware of all!)
However the link you provided shows that in "Google Activity Controls" and "Google Ad Settings" the user has the possibility to completely opt-out of this tracking.
If privacy-minded users are fully aware of all of these options, that may be fine for them? (spoiler: I was NOT aware of all!)
The link provides (pretty convoluted) details on how to opt out of one very specific type of tracking that Chrome does.
There are other options in Chrome's settings that may allow you to further opt out of other types of tracking, if you can discover them.
These settings aren't documented simply or centrally by Google and can change with each release (releases are extremely frequent).
Then there are types of tracking that can't be disabled as Google classifies them as "legitimate interest" or required for certain functions of the browser (one particular example here that comes up often is the collection of wifi ssid data for geo apis).
Anything that's on by default will always be unmanageable even by most technical users. We need a browser that doesn't track by default. Otherwise, you must typically assume you are being tracked (this goes for Firefox as well, though I think we can at least assume less data is tracked in firefox and is slightly easier to opt out)
There are other options in Chrome's settings that may allow you to further opt out of other types of tracking, if you can discover them.
These settings aren't documented simply or centrally by Google and can change with each release (releases are extremely frequent).
Then there are types of tracking that can't be disabled as Google classifies them as "legitimate interest" or required for certain functions of the browser (one particular example here that comes up often is the collection of wifi ssid data for geo apis).
Anything that's on by default will always be unmanageable even by most technical users. We need a browser that doesn't track by default. Otherwise, you must typically assume you are being tracked (this goes for Firefox as well, though I think we can at least assume less data is tracked in firefox and is slightly easier to opt out)
Someone shared this with me earlier today, funny enough [1]. They track you quite thoroughly and link the collected data, including browsing history, to your identity. It's actually much worse than just browsing data, though [2].
1. https://www.forbes.com/sites/zakdoffman/2021/03/20/stop-usin...
2. https://specials-images.forbesimg.com/imageserve/6055073241c...
1. https://www.forbes.com/sites/zakdoffman/2021/03/20/stop-usin...
2. https://specials-images.forbesimg.com/imageserve/6055073241c...
I think it's the classic "help enhance user experience" opt-out (usually prompted when you first open Chrome) that was weirdly not done here.
It's accessible even afterwards on chrome://settings/security
It's accessible even afterwards on chrome://settings/security
> Removing it is "good" for user privacy w.r.t. Google's competitors, and simultaneously good for Google (disadvantaging such competitors).
And how does Mozilla make money?
And how does Mozilla make money?
Google paying it.
Out of Chrome and Firefox, one is one big level of indirection further away from Google's influence than the other.
Out of Chrome and Firefox, one is one big level of indirection further away from Google's influence than the other.
No. Mozilla Corp is a separate entity from the Mozilla Foundation, and Google pays Mozilla Corp. (The arrangement with Google is exactly why Mozilla Corp is a distinct thing from the Mozilla Foundation: it means profits to Mozilla Corp can't influence the Mozilla Foundation.)
Mozilla Foundation is purely a donation-fed nonprofit, and most things you associate with "Mozilla" are made by the Mozilla Foundation. In fact, even Firefox itself — the open source project — is made by the Mozilla Foundation.
Mozilla Corp is just a company that employs engineers to work on Firefox. (I.e. their job is to be ordinary FOSS contributors to the project.) Those engineers might have a profit motive to do things Mozilla Corp investors/shareholders like, but that's fine, because those engineers mostly aren't part of the Firefox steering committee. The Firefox steering committee (along with all other Mozilla projects) is composed of people who are employed by the nonprofit Mozilla Foundation; and/or FOSS people external to Mozilla as a whole.
(ETA: fixed the names. I originally called Mozilla Corp "Firefox Corp" — and it really basically is, as Mozilla Corp doesn't employ people to work on anything other than Firefox. IMHO "Firefox Corp" would be a strictly-better name for it.)
Mozilla Foundation is purely a donation-fed nonprofit, and most things you associate with "Mozilla" are made by the Mozilla Foundation. In fact, even Firefox itself — the open source project — is made by the Mozilla Foundation.
Mozilla Corp is just a company that employs engineers to work on Firefox. (I.e. their job is to be ordinary FOSS contributors to the project.) Those engineers might have a profit motive to do things Mozilla Corp investors/shareholders like, but that's fine, because those engineers mostly aren't part of the Firefox steering committee. The Firefox steering committee (along with all other Mozilla projects) is composed of people who are employed by the nonprofit Mozilla Foundation; and/or FOSS people external to Mozilla as a whole.
(ETA: fixed the names. I originally called Mozilla Corp "Firefox Corp" — and it really basically is, as Mozilla Corp doesn't employ people to work on anything other than Firefox. IMHO "Firefox Corp" would be a strictly-better name for it.)
This comment misses one glaring point: Mozilla Foundation does not fund Firefox development.
There's three entities:
- Mozilla Foundation: accepts donations and spends them on web initiatives, does not spend them on Firefox
- Mozilla Corporation: is owned by Mozilla Foundation, employs Firefox devs, has search deals with Google
- Firefox software: is developed by Mozilla Corporation, is funded by search deals
There's three entities:
- Mozilla Foundation: accepts donations and spends them on web initiatives, does not spend them on Firefox
- Mozilla Corporation: is owned by Mozilla Foundation, employs Firefox devs, has search deals with Google
- Firefox software: is developed by Mozilla Corporation, is funded by search deals
You're close, but there's no "Firefox Corp".
From Wikipedia: "Mozilla [...] is a free software community founded in 1998 by members of Netscape. [...] The community is supported institutionally by the not-for-profit Mozilla Foundation and its tax-paying subsidiary, the Mozilla Corporation."
From Wikipedia: "Mozilla [...] is a free software community founded in 1998 by members of Netscape. [...] The community is supported institutionally by the not-for-profit Mozilla Foundation and its tax-paying subsidiary, the Mozilla Corporation."
A distinction without a difference, since the same people lead both organizations.
Through royalties and donations[0]. You say this like something nefarious is going on. But also consider that Mozilla has a net income of 90 million/yr[1]. The income levels between these two companies are extremely different and thus they can operate in different ways and under different conditions and saying "how does x make money" and not taking this into account is just a bad question. Their goal isn't to be Google and to be a trillion dollar company.
[0] https://www.investopedia.com/articles/investing/041315/how-m...
[1] https://en.wikipedia.org/wiki/Mozilla_Corporation
[0] https://www.investopedia.com/articles/investing/041315/how-m...
[1] https://en.wikipedia.org/wiki/Mozilla_Corporation
This is technically correct as the commenter asked "How does Mozilla make money?", but I suspect they were really enquiring how Firefox is funded.
Many people are a bit shocked to learn Mozilla Foundation does not fund Firefox. Firefox is funded by search deals (mostly by Google); Mozilla spends donations elsewhere.
Many people are a bit shocked to learn Mozilla Foundation does not fund Firefox. Firefox is funded by search deals (mostly by Google); Mozilla spends donations elsewhere.
Google consistenly tries to make it harder for their competitors to grab user data.
It's also useful for recognizing how preconceived notions and biases influence upvoting behavior and by extension what gains visibility on the front page, further reinforcing those biases.
At the time I'm writing this, there are 740 upvotes and 187 comments on this story vs. 12 upvotes and 0 comments for the top submission when Chrome made the same change [1]. Without the FYI, I would have assumed this was Firefox leading the way in privacy because I was simply unaware that Chrome had done it.
[1] https://news.ycombinator.com/item?id=24008516
At the time I'm writing this, there are 740 upvotes and 187 comments on this story vs. 12 upvotes and 0 comments for the top submission when Chrome made the same change [1]. Without the FYI, I would have assumed this was Firefox leading the way in privacy because I was simply unaware that Chrome had done it.
[1] https://news.ycombinator.com/item?id=24008516
I didn't realize this, thanks for pointing it out.
I thought, wait, but how is this feature in my own app still working with chrome then! Aha, Chrome only trims referrer for cross-origin requests.
Is FF the same?
Looks like... yes! "…will also trim path and query information for all cross-origin requests."
I thought, wait, but how is this feature in my own app still working with chrome then! Aha, Chrome only trims referrer for cross-origin requests.
Is FF the same?
Looks like... yes! "…will also trim path and query information for all cross-origin requests."
Good for both Firefox and Chrome.
Does this mean Gmail can stop rewriting links in emails (ostensibly to derefer)?
Does this mean Gmail can stop rewriting links in emails (ostensibly to derefer)?
I'm pretty sure they could have done this years ago as most browsers supported setting the policy explicitly for years. This only changes the default policy.
The only reason I can see Google still doing rewriting is to protect very old browsers and laziness.
The only reason I can see Google still doing rewriting is to protect very old browsers and laziness.
Are there any extensions that can rewrite these in the meantime?
I wrote Intercept Redirect to skip these redirect services. It doesn't change the URLs that appear on sites but intercepts the requests and instantly redirects them to the intended URL. It also requests the are minimum of permissions and collects no data.
Seems to work well, you should directly link to the Firefox extension
Thanks for trying it out and leaving a review. It's available for most browsers at https://intercept-redirect.bjornstar.com
Not using gmail works
does gmail rewrite links? AFAIK it's not rewriting any of the links in my emails. Maybe I have some setting or environment where it doesn't
[deleted]
why tf should removing Referer break sites? who uses Referer?! the only thing that Referer brought were privacy leaks.
who uses Referer?!
How can you tell it's Monday morning on HN?
Someone starts pontificating, "I don't use this, so there is no conceivable reason that anyone else on the planet should ever need this ever!"
See also: Half of StackOverflow questions.
How can you tell it's Monday morning on HN?
Someone starts pontificating, "I don't use this, so there is no conceivable reason that anyone else on the planet should ever need this ever!"
See also: Half of StackOverflow questions.
And likewise, the other sign it's Monday morning is when someone starts pontificating "this exists for some reason, so there must be someone using it for non-malicious purposes".
Which very well might be true, but that was exactly the question: who?
Which very well might be true, but that was exactly the question: who?
I use it to understand which site or link the user used to find an outdated link on our site. It's very helpful to know how he got there because often the references to that obsolete link are all purged in the code but somewhere deep inside the database or CMS somebody still put a link to the obsolete page.
well you can turn the header off pretty easily and you won't see many sites are breaking (alexa top 100), of course some wierd sites think it's important to use it, but because of the madness of referer there are so many things that will leak your privacy extremly, it came to the point that it's necessary to have extensions to remove the header or at least inject noreferer into a tags.
Loads of cashback systems use it. You get directed to Lieferando or whatever and the "click" is registered using the referrer.
there are other ways to do the same, like a URL parameter. Referrer affects the privacy of everyone and the entire web. The fact that cashback systems for some small subset of internet users will be affected (and they have alternatives like URL parameters, they did not need to use referrer) is irrelevant.
We use them to prevent other websites to hotlink our dynamic map tiles. Not a big protection of course but useful anyway.
I recently had to enable it for some login to work, unfortunately can't remember which website.
I've had to enable it for Atlassian's login to work.
I turned it off once and the most common breakages were usually authentication that involves some kind of OpenID delegation.
Yeah, we (Azure AD) got a couple of "login is broken" escalations when Chrome made this change. A couple vendors use it to validate a request is coming from the upstream IDP, to block drive by attacks I assume.
Some websites would whitelist referrers to prevent hotlinking resources from other websites (incurring potentially high bandwidth costs).
But overall I'm all for getting rid of it. It's a remnant of the early web that doesn't make a lot of sense these days, and creates more problems than it solves.
EDIT: Although I just realized that strict-origin-when-cross-origin (the new policy) would still let the hotlinking detection use case work, since you typically only need domain information for that. I initially thought that they would use same-origin or something like that. It'll teach me to comment before I read the linked story.
But overall I'm all for getting rid of it. It's a remnant of the early web that doesn't make a lot of sense these days, and creates more problems than it solves.
EDIT: Although I just realized that strict-origin-when-cross-origin (the new policy) would still let the hotlinking detection use case work, since you typically only need domain information for that. I initially thought that they would use same-origin or something like that. It'll teach me to comment before I read the linked story.
VictorOps uses referrers as a "security measure" on login. So I just have a separate profile of firefox without my antireferrer addon
> VictorOps uses referrers as a "security measure" on login.
Using the referrer header as a security and privacy measure is prevalent among service providers. Vimeo even hilariously charges for this feature which is trivially bypassed; and I'd reckon their paying customers aren't even aware of it: https://vimeo.zendesk.com/hc/en-us/articles/224819527-Changi...
Talk about smoke and mirrors.
Using the referrer header as a security and privacy measure is prevalent among service providers. Vimeo even hilariously charges for this feature which is trivially bypassed; and I'd reckon their paying customers aren't even aware of it: https://vimeo.zendesk.com/hc/en-us/articles/224819527-Changi...
Talk about smoke and mirrors.
Maybe they advertise it as a security feature but it's still a good way to prevent unauthorized hotlinking that burns your vimeo bandwidth, costing you money (assuming it does indeed prevent random websites from embedding your videos).
This reminds me of some vague memories of a time long ago where some websites had a secure login page that asks for a username&pw but every other page after just checked the referall header. It wasn't a very secure system and people shared the header values needed to bypass logging in.
That's basically a degenerate form of capability-based security. Sharing the referrer header is delegating access rights. Of course, that's not actually a property you want in this case.
I've started to see quite a bit of that on websites dealing with transfer of medical data. I figured it was probably some sort of an effort to handicap MITM potential of impostor websites that were trying to steal credentials. My theory is that it might be some sort of a tripwire for the website to detect fishiness and lock the account.
Twitter does this and more
I can't think of a single use case where a user would want a target site to know where they came from. Put that in the URL or something less nefarious if you want to communicate state.
I believe jwz rather famously does. Let's see:
https://www.jwz.org/blog/2021/03/cursed-keyboard-image/
https://www.jwz.org/blog/2021/03/cursed-keyboard-image/
His use only needs the domain, so should be unaffected? That said, something from my browser, probably ETP strict mode, appears to be stripping out the referrer entirely already anyway.
I'm not following that on the assumption is he the guy that shows HN users the goatse image?
If so then please mark your link up as NSFW/NSFL.
If so then please mark your link up as NSFW/NSFL.
jwz has got a rather ... interesting use case for it relating to HN
And slashdot before that.
Ah jwz did this for /. too? If he did then it seems like he has switched it off, because I cannot reproduce it
I just tried to find references to it, but couldn’t. I’m wondering if my memory is broken. Slashdot is old though, we used to get worked up about bugs in X11 screensavers, and GNAA trollers.
It was a somewhat common “security” feature to check referer, especially in the days before you could rely on CSP.
The first thing I thought of was when you go to a URL but it requires a login so it redirects you to the login page but then redirects you back to your original page after you login. Pretty sure it’s easily remedied without referer but that was the first thing I thought about.
Some ad networks use(d to use) referers for brand safety when resource serving an ad is embedded iframe. For example if the parent URL contains "osama-bin-laden-killed", don't display ad about nice family moments together (or anything else, likely)
Or keep track of your internet history to have a more accurate profile of you. This is not a good thing and it's precisely why it's a privacy leak.
Some sites use referrer as their "cookie session". One site use Cloudflare Anti-DDOS referrer as their cookie. Removing that referrer will cause the browser to infinity loop in CF Anti-DDOS because the site is expecting that referrer before entering the site.
I hate sharing Amazon shopping links because Amazon packs much referrer as possible in the link. Amazon uses the referrer to track who am I sharing to and use it for data for them to sells.
I hate sharing Amazon shopping links because Amazon packs much referrer as possible in the link. Amazon uses the referrer to track who am I sharing to and use it for data for them to sells.
Are you conflating Referer with URL parameters? (Amazon calls them "ref tags" (referrer tags) but that's not HTTP Referer. It's not even possible to do what you are imagining with HTTP Referer, since Referer is only inside a browser session, not across from you to your friend.
Dang it, I didn't realize it was specific to HTTP. Thank you for pointing it out, all I see referrers (I somehow blank on HTTP) and I thought it was about the referrer tags.
Referer is an important way to maintain the security of an OAuth flow.
I don't know how commonly it would break that flow, but it certainly could depending on implementation details that are not narrowly prescribed by the OAuth2 spec.
I don't know how commonly it would break that flow, but it certainly could depending on implementation details that are not narrowly prescribed by the OAuth2 spec.
> Referer is an important way to maintain the security of an OAuth flow.
How? Referrer can be easily spoofed.
How? Referrer can be easily spoofed.
The scenario is that the user is being phished with an uncompromised, modern browser. The idea is that if the attacker doesn't have control of the browser, they can't spoof the referer.
no referer is an important way to screw your security in an oauth2 flow. with oauth2 a correct configuration would be Referrer-Policy: no-referrer and enforcing a secure state param and csrf.
Isn't it also sort of useful information for site owners even if they are not maliciously slurping your private information?
For example you might not know that important site X linked to you and is driving Y% of your audience. You are not necessarily interested in the individuals following the links when you discover that.
For example you might not know that important site X linked to you and is driving Y% of your audience. You are not necessarily interested in the individuals following the links when you discover that.
This change only trims the referrer string, it doesn't remove the origin domain info. You can still tell that "important site X linked to you and is driving Y% of your audience." You just wouldn't be able to get any more granular than that.
And to note the obvious... the companies that you might think would be most opposed to it (e.g. facebook, google) really don't care because their page tracking hooks are omnipresent so they're not missing out on any precious meta data.
Yeah, I was gonna say; isn't this part of the spec? Seems like it is; at least in the latest editor's draft: https://w3c.github.io/webappsec-referrer-policy/#default-ref...
That doesn't appear to be accurate in Chrome, in 89 I am getting no-referrer-when-downgrade.
For example, I clicked a link on old reddit and saw the full referer being sent to another origin.
For example, I clicked a link on old reddit and saw the full referer being sent to another origin.
Ah, that explains why the old "come from Google" trick for bypassing paywalls stopped working a while ago.
This doesn't change my opinion. This is like saying, "The soviet union gulags their people too and did it first. It's fine."
Whataboutism re: corporate control of web standards.
Whataboutism re: corporate control of web standards.
That's probably because if you run Chrome, you're already Google's executable on your system. I mean they can track you using gazillion of other, more precise meanings and this is something they shut it to keep the pie for themselves. I do not use Chrome anymore, switched to Firefox a year and a half ago - best decision ever.
Oh, I'll really miss occasionally peeking at AWStats and discovering weird pages pointing at my weird pages :(
This subtle aspect of web had been always strangely appealing for me: people leaving trails in access logs and building real "footpaths" network of synapses between HTML documents, across origins. Sad to watch it dying, however beneficial and understandable it is.
I feel it didn't have to be this way: maybe if GET wasn't so widely misused recently and generally everybody knew what to not put in URL and acted accordingly, we could have preserved such nice things.
This subtle aspect of web had been always strangely appealing for me: people leaving trails in access logs and building real "footpaths" network of synapses between HTML documents, across origins. Sad to watch it dying, however beneficial and understandable it is.
I feel it didn't have to be this way: maybe if GET wasn't so widely misused recently and generally everybody knew what to not put in URL and acted accordingly, we could have preserved such nice things.
You'll still get the source domain name, you just won't be able to see the full path. Of course how useful it is will depend on the source website, if it's "news.ycombinator.com" it won't be too difficult to find the post, if it's "facebook.com" it'll be a tad more complicated.
Yes, I came to lament AWStats as well. I remember when google started stripping search terms in their referring urls. So then you didn't know what search terms were pointing to your site.
Google's answer was to just get your site verified and use their search console tools. Annoying, but OK. But then other search engines followed suit which left awstats with a lot of missing data. Now with simplified referring urls... it will mostly be a fancy page counter :/
Google's answer was to just get your site verified and use their search console tools. Annoying, but OK. But then other search engines followed suit which left awstats with a lot of missing data. Now with simplified referring urls... it will mostly be a fancy page counter :/
Similar here, and as I have written recently: https://news.ycombinator.com/item?id=26532433
I feel that this could easily be a user toggle-able option near the url bar - and that people could choose to send referrer and search parameters to sites they like - and I could understand certain medical terms and what-not maybe people wanting to keep more hidden.
I'm all for the browser taking control of this and giving the users options - love to be able to have the web site detect this and add a box saying 'hey would you mind sharing referral and or search term with us"
It really helps when trying to figure out if you should be adding more content for 'term-mainly-X-demo' might be searching or for sure should be adding more content for other terms people are definitely searching for.
I would imagine that many people would like to hide referrer for fbook, most of my sites I would think people would be willing to help with providing such info.
as far as webmastering and stats to try to help visitors - it appears google is keeping more and more data to themselves, and now firefox is making it less transparent too.
I see there is a link to the referrer policy for your site, but nothing about begging the user to include the info when visiting - oh well.
awstats and similar have helped troubleshoot and helped expand many sites for many years and now another nudge to install third party slow-ware google-stats.. I just, sigh.
I feel that this could easily be a user toggle-able option near the url bar - and that people could choose to send referrer and search parameters to sites they like - and I could understand certain medical terms and what-not maybe people wanting to keep more hidden.
I'm all for the browser taking control of this and giving the users options - love to be able to have the web site detect this and add a box saying 'hey would you mind sharing referral and or search term with us"
It really helps when trying to figure out if you should be adding more content for 'term-mainly-X-demo' might be searching or for sure should be adding more content for other terms people are definitely searching for.
I would imagine that many people would like to hide referrer for fbook, most of my sites I would think people would be willing to help with providing such info.
as far as webmastering and stats to try to help visitors - it appears google is keeping more and more data to themselves, and now firefox is making it less transparent too.
I see there is a link to the referrer policy for your site, but nothing about begging the user to include the info when visiting - oh well.
awstats and similar have helped troubleshoot and helped expand many sites for many years and now another nudge to install third party slow-ware google-stats.. I just, sigh.
It's not really important enough to be a visible element on the bar. It would be a good compromise to bury an opt-in somewhere in settings though. Firefox is doing the right thing here by making it default and uncomplicated.
I feel that you are right.
I'd love it to become a right click on the page and 'share all requested info, or selected info with this site" perhaps.
I don't mind right click - save page as.. but feel the secondary menu needed when right clicking on a tab, then hover down to move tab, then another flyout menu for new window is too much. Wish the 'move tab to new window' was just right there one level up.
I feel that people would right click and allow info share right there if requested by some sites. Buried in the about: settings or whatever would never get used.
I don't mind right click - save page as.. but feel the secondary menu needed when right clicking on a tab, then hover down to move tab, then another flyout menu for new window is too much. Wish the 'move tab to new window' was just right there one level up.
I feel that people would right click and allow info share right there if requested by some sites. Buried in the about: settings or whatever would never get used.
> I feel that this could easily be a user toggle-able option near the url bar...
No ordinary user wants to deal with that much complexity.
No ordinary user wants to deal with that much complexity.
But why have we decided lately to fall for the tyranny of what "ordinary users" want?
Why can't we have a full featured Firefox hidden behind a setting, so this mythical "ordinary user" can have a dumbed down interface and the rest of us can have a real browser?
Why can't we have a full featured Firefox hidden behind a setting, so this mythical "ordinary user" can have a dumbed down interface and the rest of us can have a real browser?
> But why have we decided lately to fall for the tyranny of what "ordinary users" want?
For the same reason bank robbers rob banks. That's where the money is. The source of the funding that pays for development of these tools is driving this.
I'm not defending that situation, that's just the way it is. "Hackers" working on what they find technically elegant and useful are not driving these sorts of changes.
For the same reason bank robbers rob banks. That's where the money is. The source of the funding that pays for development of these tools is driving this.
I'm not defending that situation, that's just the way it is. "Hackers" working on what they find technically elegant and useful are not driving these sorts of changes.
[deleted]
"mythical" ordinary user? Have you been living in the world recently?
Been living here for many years already.
Even worked a lot on the help desk, taught in vocational education and held a couple of training courses.
At least around here my evaluation is that ordinary users are a lot more capable than we give them credit for.
Or maybe I'm just insanely good at helping people, or I misunderstand or something else, you choose ;-)
Even worked a lot on the help desk, taught in vocational education and held a couple of training courses.
At least around here my evaluation is that ordinary users are a lot more capable than we give them credit for.
Or maybe I'm just insanely good at helping people, or I misunderstand or something else, you choose ;-)
I suspect this is going to be a huge boon for services like ahrefs that crawls the web and provides information on which URLs on the internet link to which other URLs.
Yeah, the big sites will still get to see the data though, so if you just become an employee, you can probably still get it.
Why? What happened here is a change to the default value when https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Re... value if it's left unspecified. Set it to what you want, and this change doesn't affect you?
It's the policies of the external sites that matter to the parent poster, not their own.
yes, for content-only sites it makes perfect sense. For sites hosting private data requiring login, (especially if there's some sensitive information in the URL) it poses a risk.
Totally agree. As almost no one uses anything like webmentions, this was the way to discover who was linking to your site. (And because the Google link tool didn’t seem to return much actual information.)
From a glance at MDN [1] and specs [3] it seems at least we can still opt-in our sites to suggest visitors user agent to pass full referrer address along outgoing links; there seems to be three ways to do so:
(Funny all three have different spelling, and that in effect they direct value of a single HTTP header with inherently erroneous spelling.)
[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Re... [2] my guess using `<meta http-equiv="Referrer-Policy" content="no-referrer-when-downgrade">` should do the same, but is not explicitly mentioned anywhere. [3] https://w3c.github.io/webappsec-referrer-policy/
# 1. Apache conf
Header set Referrer-Policy "no-referrer-when-downgrade"
<!-- 2. meta HTML head with same effect [2] -->
<meta name="referrer" content="no-referrer-when-downgrade">
<!-- 3. HTML anchor attribute -->
<a href="https://…" referrerpolicy="no-referrer-when-downgrade">
(Funny all three have different spelling, and that in effect they direct value of a single HTTP header with inherently erroneous spelling.)
[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Re... [2] my guess using `<meta http-equiv="Referrer-Policy" content="no-referrer-when-downgrade">` should do the same, but is not explicitly mentioned anywhere. [3] https://w3c.github.io/webappsec-referrer-policy/
Note that it only works if it's set on the source page, for obvious reasons. So if you set these headers on your page the browser will send the referrer when browsing away from your page, but not when your page is the target.
Bring it back! https://news.ycombinator.com/item?id=26401955
Absolutely, I loved seeing spikes in my traffic and going to the reddit/HN thread that caused it. I guess you still can manually figure it out using a search engine maybe, but like you said the smaller weird ones will go under the cover probably.
Exactly. I am torn on this feature. On one hand this is the correct default for privacy, there are a number of websites with somewhat sensitive information in the URL. But relying on a search engine or webmaster tools to crawl the entire web to see where your traffic is coming from is a real shame. Especially because it will have an inevitable delay.
It is a fun feature, but privacy generally trumps such things since they are constantly being used by various entities for bad purposes.
The next step is not knowing where your visitors are coming from and having to only buy google ads rather then creating affiliate relationships. It is too bad FireFox is playing into this, it is not helping anyone's privacy. It only lines Google's pockets.
You’ll still know the site sending traffic your way, but not the exact page or query. For most affiliates that’s fine and sites like HN could set the Referrer-Policy header to allow sending page-level details since they know that they don’t include anything sensitive.
What the changed default does is mean that things fail safely when someone doesn’t think about this at all and builds a site with sensitive info in the URL. In that case, outbound link targets might receive things like usernames or email addresses, information in URL path components, etc. which isn’t intended to be public. Those sites are often poorly supported so the default changing means that the work shifts to a fraction of the web community which is best prepared to deal with the trivial extra work required to set a policy.
What the changed default does is mean that things fail safely when someone doesn’t think about this at all and builds a site with sensitive info in the URL. In that case, outbound link targets might receive things like usernames or email addresses, information in URL path components, etc. which isn’t intended to be public. Those sites are often poorly supported so the default changing means that the work shifts to a fraction of the web community which is best prepared to deal with the trivial extra work required to set a policy.
Using and needing the exact url to form an affiliate relationship sounds like a very rare situation.
And I don't know how you can say it doesn't help privacy with a straight face.
And I don't know how you can say it doesn't help privacy with a straight face.
It makes total sense that you have a website but have no idea where people are coming from to get to it. Why know what is working for your website so that you can improve upon it?
No idea?? You know the domain, which is usually enough by itself. And 99% of the time you can quickly search the domain to find what links to you.
So then there is no reason to remove most of the referrer. Because you have an ip address, url, and the ability to probably find the rest of the url. I would prefer to have the whole url and no ip address because a users ip address is useless to me while the url with the link on it is valuable. So they are taking away the wrong thing.
It depends on how the site is laid out. Sometimes the url has information that shouldn't be shared, and truncating the url fixes that with very little downside.
And it would be nice if IP addresses could be hidden but that's a very different topic.
And it would be nice if IP addresses could be hidden but that's a very different topic.
Then you would cut off the querystring. Removing the querystring would chop off any sessionID making it impossible to log back in.
Affiliates can still use things like personalised discount codes, script-based tracking, etc. There's still numerous ways for affiliates to exist.
Here's a practical reason for doing this.
https://shkspr.mobi/blog/2018/01/mailchimp-leaks-your-email-...
A few years ago, I discovered that referrers from MailChimp let you unsubscribe people from lists, and see their email addresses.
https://shkspr.mobi/blog/2018/01/mailchimp-leaks-your-email-...
A few years ago, I discovered that referrers from MailChimp let you unsubscribe people from lists, and see their email addresses.
Very cool discovery. This is an excellent example of why putting sensitive data inside URLs can be problematic and should be done with great caution.
It should be added that the eventual elimination of referers won’t make URLs safe. There are countless other ways they can leak.
It should be added that the eventual elimination of referers won’t make URLs safe. There are countless other ways they can leak.
I think there'd be a bit less panic in the comments if the title/headline reflected that the change (as I understand it) applies cross-origin and cross-scheme (http > tls). So if you're preventing hot-linking of assets, this should not affect you (or; you have some control over it via policy):
> this new stricter referrer policy will not only trim information for requests going from HTTPS to HTTP, but will also trim path and query information for all cross-origin requests.
Seems like a fairly balanced way to protect privacy along with preserving utility?
> this new stricter referrer policy will not only trim information for requests going from HTTPS to HTTP, but will also trim path and query information for all cross-origin requests.
Seems like a fairly balanced way to protect privacy along with preserving utility?
For anyone who wonders how http referer could ever be a good idea consider the following:
I remember when my dad studied to become a teacher. As one of their assignments they had to create a webside. As someone who had recently given up farming I think he wrote about farm animals and linked to some other pages about small scale poultry and similar topics.
One day he got a mail from the "webmaster" of one of the sites he linked to that he would have to update his links soon. I remember being really surprised that someone knew my dad had linked to them.
Being only 16 or 17 or something I only knew simple html, basic and vb but I knew that html links were one way.
I don't think I realized until later what had really happened: this person had looked at their server logs to see where their customers came from, looked up the page and found the email address.
Of course this also highlights why the referer is so problematic.
I remember when my dad studied to become a teacher. As one of their assignments they had to create a webside. As someone who had recently given up farming I think he wrote about farm animals and linked to some other pages about small scale poultry and similar topics.
One day he got a mail from the "webmaster" of one of the sites he linked to that he would have to update his links soon. I remember being really surprised that someone knew my dad had linked to them.
Being only 16 or 17 or something I only knew simple html, basic and vb but I knew that html links were one way.
I don't think I realized until later what had really happened: this person had looked at their server logs to see where their customers came from, looked up the page and found the email address.
Of course this also highlights why the referer is so problematic.
> One day he got a mail from the "webmaster" of one of the sites he linked to that he would have to update his links soon. I remember being really surprised that someone knew my dad had linked to them.
These days 100% of such emails I receive are from spammers trying to steal some google juice.
These days 100% of such emails I receive are from spammers trying to steal some google juice.
You'll still be able to see which site visitors came from, just not the specific page.
That reminds me that in 1999, looking through the referrer logs, I realized that if the link came in from an Outlook email, Outlook+IE would report the subject of the referring email as the referrer (iirc with user name, something like “mailbox://user@site/subject-of-the-email”).
So we started looking for those more seriously in my company, and got quite a bit of interesting Intel from potential investors, competitors we knew about, and some we weren’t even aware of.
It was just the subject and user, but was often surprisingly informative.
So we started looking for those more seriously in my company, and got quite a bit of interesting Intel from potential investors, competitors we knew about, and some we weren’t even aware of.
It was just the subject and user, but was often surprisingly informative.
Do you, or anyone, have [links to] more information about exactly what Outlook does when requesting a tracking pixel to display in an email.
I can't sniff it (Wireshark) as I don't have that sort of access to a network with Outlook on. I've googled but didn't find anything useful.
I can't sniff it (Wireshark) as I don't have that sort of access to a network with Outlook on. I've googled but didn't find anything useful.
It was surely fixed in Outlook 2007, perhaps even 2003 and maybe even 2000.
Nice, I'm using SmartReferer (https://addons.mozilla.org/en-US/firefox/addon/smart-referer...), I'm always happy to drop an extension when Mozilla natively implements the same feature
Smart Referer is even stricter than this new policy if you don't use the recommended 'Lax' setting. It will block referers even on the same domain but to different subdomains, if you want it to.
Doesn't this just push people to more tracking cookies? How are sites supposed to know what sources are driving their traffic? Whether visitors are coming via email campaigns, google, etc?
A popular approach seems to be to use customized URLs. Which is of course always an option, though ClearURLs takes care of a few common options.
Besides it's up to a user whether they want to tell you where they came from, you aren't supposed to know anything, you are allowed to.
Besides it's up to a user whether they want to tell you where they came from, you aren't supposed to know anything, you are allowed to.
Thanks for the pointer -- not come across Clear URLs before.
https://addons.mozilla.org/en-GB/firefox/addon/clearurls/ in case anyone else has also not seen it.
https://addons.mozilla.org/en-GB/firefox/addon/clearurls/ in case anyone else has also not seen it.
Back in the days, there'd be a jsessionid in the url :-)
Why should they know this? I am keen that they don't know this.
If I walk into a shop, the shopkeeper doesn't know if something in the window caught my eye, if my friend recommended the helpful staff, if I saw the advert they placed on a billboard, or if I just came in because it's raining.
If I walk into a shop, the shopkeeper doesn't know if something in the window caught my eye, if my friend recommended the helpful staff, if I saw the advert they placed on a billboard, or if I just came in because it's raining.
> If I walk into a shop, the shopkeeper doesn't know if something in the window caught my eye, if my friend recommended the helpful staff, if I saw the advert they placed on a billboard, or if I just came in because it's raining.
A lot of places will attach unique discount codes to advertisements to get an idea on where you came from.
For example, on a TV commercial it might say use promo code COOL123 to save 10% or if you saw that billboard it might say to use NICE123 instead. Then there's radio, newspapers and so on each with their own unique code that offer the same 10% discount.
And since a discount is applied chances are you'll use it because not too many folks would purposely avoid the discount to hide where you discovered the shopkeeper from.
You often see this being done online too, but there's also things like UTM tags or unique URLs that let you do the same thing without discount codes. It's not to make more money as a shopkeeper (avoiding the discount), it's just easier to set up. Using a UTM tag is a matter of creating a link with a few query parameters. Creating a specific discount code or a unique URL for a specific promotion takes a bit of extra leg work.
I'm not sure where I stand on the movement to remove all forms of referral tracking. Both referral headers and UTM tags can be spoofed or removed through extensions so using them as any type of source of truth was a bad idea anyways.
However, as someone who sells digital products I like knowing which specific video or blog post helped someone discover some of my paid content. But at the same time, the end game is really "is the needle moving forward?" so the specifics kind of don't matter. But in the short term while you're figuring things out, the extra info does help you focus on where to spend your time. Not just for more profit, but to provide better and more free content because it's what folks want.
A lot of places will attach unique discount codes to advertisements to get an idea on where you came from.
For example, on a TV commercial it might say use promo code COOL123 to save 10% or if you saw that billboard it might say to use NICE123 instead. Then there's radio, newspapers and so on each with their own unique code that offer the same 10% discount.
And since a discount is applied chances are you'll use it because not too many folks would purposely avoid the discount to hide where you discovered the shopkeeper from.
You often see this being done online too, but there's also things like UTM tags or unique URLs that let you do the same thing without discount codes. It's not to make more money as a shopkeeper (avoiding the discount), it's just easier to set up. Using a UTM tag is a matter of creating a link with a few query parameters. Creating a specific discount code or a unique URL for a specific promotion takes a bit of extra leg work.
I'm not sure where I stand on the movement to remove all forms of referral tracking. Both referral headers and UTM tags can be spoofed or removed through extensions so using them as any type of source of truth was a bad idea anyways.
However, as someone who sells digital products I like knowing which specific video or blog post helped someone discover some of my paid content. But at the same time, the end game is really "is the needle moving forward?" so the specifics kind of don't matter. But in the short term while you're figuring things out, the extra info does help you focus on where to spend your time. Not just for more profit, but to provide better and more free content because it's what folks want.
As a user with a certain combination of medical and other life issues, I'm neutral on the use of varied discount codes, as it's not personally identifying, just gives you the seller an idea of which media is a good buy. Ok, no worries.
Bur given the unique personal searches/browsing I could have just been doing, I sure the heck don't want my recent activity advertised to the next site I go to. Happy as a clam to shut down all the Referrer uses, even whatever login flows that might have to re-engineer themselves.
Bur given the unique personal searches/browsing I could have just been doing, I sure the heck don't want my recent activity advertised to the next site I go to. Happy as a clam to shut down all the Referrer uses, even whatever login flows that might have to re-engineer themselves.
We run a micro-business and we ask people how they heard of us (and use offer codes, but that was mentioned).
People are usually happy to tell us; but then we only use that information to run our business better, we don't sell it.
Another technique is referral bonus, "recommend a friend" type offers.
People are usually happy to tell us; but then we only use that information to run our business better, we don't sell it.
Another technique is referral bonus, "recommend a friend" type offers.
"the shopkeeper doesn't know if something in the window caught my eye"
Except that's already here, or in the making, i.e. with those advertising eye-tracking stories that make the rounds every 6 months or so.
Except that's already here, or in the making, i.e. with those advertising eye-tracking stories that make the rounds every 6 months or so.
If I walk into a shop, the shopkeeper doesn't know if something in the window caught my eye, if my friend recommended the helpful staff, if I saw the advert they placed on a billboard, or if I just came in because it's raining.
But the shopkeeper knows if you came in through the street entrance (maybe saw the window display), through the mall entrance (maybe saw the sandwich board), or through the communicating door to the next shop (maybe saw the merchandise).
But the shopkeeper knows if you came in through the street entrance (maybe saw the window display), through the mall entrance (maybe saw the sandwich board), or through the communicating door to the next shop (maybe saw the merchandise).
Only true for a small shop where the shopkeeper can see all entrances and is able to pay attention to that enough of the time.
Otherwise you need video cameras and AI... or perhaps spray every incoming customer with a source-identifying powder or smell - hmm, maybe that explains Abercrombie & Fitch!
Otherwise you need video cameras and AI... or perhaps spray every incoming customer with a source-identifying powder or smell - hmm, maybe that explains Abercrombie & Fitch!
Firefox still sends the origin (the domain name and subdomain) of the referring page, so you can still see the website the user is coming from.
Google has had a dereferral process in place for many years anyway, which prevents you from seeing the specific search engine results page the user is coming from. So this really doesn't change anything for those websites such as Google that already had these security features in place.
Google has had a dereferral process in place for many years anyway, which prevents you from seeing the specific search engine results page the user is coming from. So this really doesn't change anything for those websites such as Google that already had these security features in place.
> Google has had a dereferral process in place for many years anyway, which prevents you from seeing the specific search engine results page the user is coming from.
Is this really a security feature though? Or is it Google keeping the important information for themselves?
Is this really a security feature though? Or is it Google keeping the important information for themselves?
It's certainly has nothing to do with security. It's Google's way to force you to use Google Analytics if you want to track said informarion.
Google Analytics does not give you additional keyword information.
If you want keyword information from Google search results, the best way to get it is actually buying ads for those keywords. And in fact some marketers allocate ad budget to keywords they already rank for, specifically for that purpose.
If you want keyword information from Google search results, the best way to get it is actually buying ads for those keywords. And in fact some marketers allocate ad budget to keywords they already rank for, specifically for that purpose.
But to the grandparent's point, GA used to give you a ton more organic info at the keyword level. It is part of what drove the seo industry. The only way to get an abstract view of that keyword volume now, including on your brand terms, is to buy ads and run the organic reports.
Also, it prevents you from exfiltrating Google's ML model for search and build up your own.
How does this work in real life? How does say Tesco can find out if I’m shopping there because of a billboard ad, or because it’s conveniently located on my commute? How do they measure their ad campaign effectiveness?
Facial recognition in stores, your credit card, or your loyalty card tie your purchase to an ID, which then can be back-tied to all the Internet surveillance in your browser and on your phone, to attribute purchases to ads viewed on-line. It's a bit tougher with attributing real-life billboard ads.
Brave new world ;)
You can still see what site the referrer came from, just not the path + query string behind it.
As a visitor to your web site: not my problem and you've no right to know anyway what I visited before coming to your site. That fact that you can still see the referring domain is still too much, in my opinion.
In Firefox, "network.http.referer.spoofSource = true" takes care of that last bit. The only thing it breaks for me is codepen.io, which fortunately I don't use.
> How are sites supposed to know what sources are driving their traffic? Whether visitors are coming via email campaigns, google, etc?
Kindly, and non-intrusively, ask them? You know, just like in other aspects of life?
Kindly, and non-intrusively, ask them? You know, just like in other aspects of life?
I think the major problem here is that this information was sent *by default* and many websites didn't realize that outbound links where sharing the source URL. This could result in an exploitable security issue.
Now the website has to do something explicit to share information (beyond the hostname) to outbound cross-origin outbound links. It is much more likely that the website will pay attention to what information they are sharing in this case.
Now the website has to do something explicit to share information (beyond the hostname) to outbound cross-origin outbound links. It is much more likely that the website will pay attention to what information they are sharing in this case.
This made me wonder how to do it now, in Firefox 86. I found this helpful page: https://askubuntu.com/questions/797135/how-to-disable-http-r...
TL;DR about:config --> Network.http.sendRefererHeader --> change value from 2 to 0
TL;DR about:config --> Network.http.sendRefererHeader --> change value from 2 to 0
[deleted]
And so the dream of bidirectional hyperlinks finally dies. Turns out not only do we not need them, in most cases you really don't want them.
Yes. This makes surfing the web harder and will result in more centralization within proprietary walled gardens instead of federation via protocol mechanisms.
And just a reminder that it's always a good idea to set Referrer-Policy regardless on sites you own.
It's good advice, but bizarrely some browsers ignore Referrer-Policy, even when it requests "no-referrer", which basically increases the privacy level to the maximum possible - beyond for example the origin-only settings which the browser defaults are now tending towards (such as with this Firefox update).
In my testing, all recent versions of Safari on iOS were the worst offenders here. I talked more about this here: https://twitter.com/arbuge/status/1354805900268105743
You would think that Safari would jump at the chance to show no referrer, given Apple's attempts to position themselves as champions of user privacy, but for some reason the opposite is in fact true.
In my testing, all recent versions of Safari on iOS were the worst offenders here. I talked more about this here: https://twitter.com/arbuge/status/1354805900268105743
You would think that Safari would jump at the chance to show no referrer, given Apple's attempts to position themselves as champions of user privacy, but for some reason the opposite is in fact true.
I just realized the 'Referer' header is actually a misspelling in the http protocol.
Yup, here's an article you may enjoy https://httptoolkit.tech/blog/http-wtf/ and related HN discussion https://news.ycombinator.com/item?id=26343577
And I made up the HTTP status code "397 Tolerating" so that if you spell it "Referrer", browsers can correct you while still giving you the response you wanted :-p (see section 3 example):
https://pastebin.com/TPj9RwuZ
https://pastebin.com/TPj9RwuZ
Late correction: sorry, that should be "if your browser spells it 'Referrer', webservers can correct you...".
I am genuinely unsure whether your RFC was intended to be serious or not.
So am I! I did it as an April Fool’s joke (see the date and the author, Ben Dover). But I also think it would be legitimately useful and have long wished for a standard protocol for expressing that you’re “tolerating” a standards violation while still pointing it out.
We chose not to add referrer to ASPSecurityKit main site [0]. It's a static content site and I think it'd be useful to let other sites know which page (docs/guides/blog) on our site got them a visitor because the content is public anyway. We've applied it on the dashboard though, this same origin-when-cross-origin policy.
0: https://ASPSecurityKit.net
0: https://ASPSecurityKit.net
any chance that a few CORS implementations will break due to this ?
No.... ? Not as far as I could gather. Why would CORS break because of this change? AFAIK, CORS has nothing with the referrer header.
he's probably talking about referrer checks which are used as a mitigation against csrf
It only trims path and query information so it should be fine unless the site is overzealous and checks those.
Google largely killed this when they moved to https by default, but I missed reviewing what search terms visitors used to visit my site and then creating content to answer their actual questions, instead of guessing.
But the web was a much smaller place/time back then.
Oh, and seeing people search for my uncommon name...
But the web was a much smaller place/time back then.
Oh, and seeing people search for my uncommon name...
I hope they do User-Agent next.
Coincidentally, Mozilla's site is one that consistently blocks my user-agent (Lynx).
They've indicated they may: https://github.com/mozilla/standards-positions/issues/202#is...
Commenting in support of Chrome's proposal to freeze the user agent and provide UACH instead.
Commenting in support of Chrome's proposal to freeze the user agent and provide UACH instead.
UACH is worse than a header.
The server shouldn't get information about the client.
Unsurprisingly, UACH is spearheaded by an ad tracking company that benefits from a moat caused by minimally viable/workable browsers costing millions of dollars and dozens of developers to implement.
The server shouldn't get information about the client.
Unsurprisingly, UACH is spearheaded by an ad tracking company that benefits from a moat caused by minimally viable/workable browsers costing millions of dollars and dozens of developers to implement.
> UACH is worse than a header.
It's not. The header gives up all information by default. UA-CH works by request-only, and allows the browser to determine how much to send. This is easily controlled via native settings or browser extensions.
The API also requires a secure connection, and specifically forces the server to admit to any fingerprinting (or legitimate use-case of data otherwise).
It's not. The header gives up all information by default. UA-CH works by request-only, and allows the browser to determine how much to send. This is easily controlled via native settings or browser extensions.
The API also requires a secure connection, and specifically forces the server to admit to any fingerprinting (or legitimate use-case of data otherwise).
At least that might stop Twitter refusing to serve User-Agents it doesn't recognise.
There was a brief period of time last year when Twitch streams broke for me, turns out because my UA includes "FreeBSD" instead of an OS they know about. Eventually this was fixed.
How long until we see a CFAA prosecution based solely on providing a UA in the request that you're not "supposed" to use?
I'm surprised it took this long, and it's still not completely gone. I've never understood the history of why http referer exists (original intent) or why the user would benefit from sharing it.
The original HTTP specification really didn’t think of the web as an adversarial environment. The RFC that specified the ‘referer’ header [1] described it as having the following purpose:
[1] https://tools.ietf.org/html/rfc1945#section-10.13
[Referer]... allows the client to specify, for
the server's benefit, the address (URI) of the resource from which
the Request-URI was obtained. This allows a server to generate lists
of back-links to resources for interest, logging, optimized caching,
etc. It also allows obsolete or mistyped links to be traced for
maintenance.
[1] https://tools.ietf.org/html/rfc1945#section-10.13
I never send a referer header. This has zero effect on my "user experience".
Same experience here.
But why don't thet kill the referer header entirely? Mozilla's attempts at protecting user privacy seem very half-hearted.
But why don't thet kill the referer header entirely? Mozilla's attempts at protecting user privacy seem very half-hearted.
"Mozilla's attempts at protecting user privacy seem very half-hearted."
That is because they are. Mozilla cannot survive without internet advertising, and they sacrifice the privacy of Firefox users in exchange for funding from an advertising company, Google.
Anyone who is serious about internet privacy knows it requires some amount of vigilance. It necessitates some amount of inconvenience. It is not simply a matter of software selection. AFAIK, no third party today is going to take on full responisibility for any user's privacy. You never see Mozilla advocating for user vigilance, yet the fight for internet privacy is the user's, not Mozilla's. The message from Mozilla is something like "If you use Firefox, we have you covered." This encourages inaction more than action. That's good for companies like Google.
If Google paid a group of users the millions it pays the group working at Mozilla, I doubt those users would care one iota about "privacy". Their own, or anybody else's. Why bite the hand that feeds you.
I only send Host and Connection headers for GET and Host, Connection, Content-Length and Content-Type for POST. For me, this works beautifully for 100% of websites posted to HN, and elsewhere on the www. I can easily create exceptions in the forward proxy configs to send Referer to sites that require it. This never happens, IME.
That is because they are. Mozilla cannot survive without internet advertising, and they sacrifice the privacy of Firefox users in exchange for funding from an advertising company, Google.
Anyone who is serious about internet privacy knows it requires some amount of vigilance. It necessitates some amount of inconvenience. It is not simply a matter of software selection. AFAIK, no third party today is going to take on full responisibility for any user's privacy. You never see Mozilla advocating for user vigilance, yet the fight for internet privacy is the user's, not Mozilla's. The message from Mozilla is something like "If you use Firefox, we have you covered." This encourages inaction more than action. That's good for companies like Google.
If Google paid a group of users the millions it pays the group working at Mozilla, I doubt those users would care one iota about "privacy". Their own, or anybody else's. Why bite the hand that feeds you.
I only send Host and Connection headers for GET and Host, Connection, Content-Length and Content-Type for POST. For me, this works beautifully for 100% of websites posted to HN, and elsewhere on the www. I can easily create exceptions in the forward proxy configs to send Referer to sites that require it. This never happens, IME.
> But why don't thet kill the referer header entirely?
Compatibility, probably. There are some sites which fail to load or end up in redirect loops if the Referer header is missing. It's not just minor sites, either. The last time I tried blocking Referer headers—just the third-party ones—it broke Google Hangouts (both the Chrome extension and the web version). That was earlier this year.
Compatibility, probably. There are some sites which fail to load or end up in redirect loops if the Referer header is missing. It's not just minor sites, either. The last time I tried blocking Referer headers—just the third-party ones—it broke Google Hangouts (both the Chrome extension and the web version). That was earlier this year.
Would it really break that much to just get rid of referrer altogether? I would miss it on my personal site (self hosted, Foss analytics, no google analytics there), but it wouldn't actually break anything.
That's going to break lots of older sites using Referrer for navi state. These will now either have to use query params, cookies, or JS instead. Not to mention easy affiliation links.
As a user I love stricter privacy. As a developer working for a platform company whose content (video) is embedded by thousands of websites, I hate this particular change:
- more difficult to analyze weird / fraudulent embedders
- more difficult to debug issues ("what's the sample URL to repro? no sample URL in logs, only top-level of the domain, but I don't find our embed anywhere ¯\_(ツ)_/¯")
Funny thing: you can't just tell your embedding partners to change the embed code and use `referrerpolicy=...` on the iframe, to expose the full URL, because it's not GDPR-compliant apparently. So you need user's consent first. But how do you obtain user's consent before you render HTML on the server? :) ("GDPR wall" is not compliant either)
Life sucks, I guess. But it's for greater good, and I guess the companies will somehow survive.
- more difficult to analyze weird / fraudulent embedders
- more difficult to debug issues ("what's the sample URL to repro? no sample URL in logs, only top-level of the domain, but I don't find our embed anywhere ¯\_(ツ)_/¯")
Funny thing: you can't just tell your embedding partners to change the embed code and use `referrerpolicy=...` on the iframe, to expose the full URL, because it's not GDPR-compliant apparently. So you need user's consent first. But how do you obtain user's consent before you render HTML on the server? :) ("GDPR wall" is not compliant either)
Life sucks, I guess. But it's for greater good, and I guess the companies will somehow survive.
Wish i was able enough to help Web servers cull http and be https per default rather than offer complex alternatives that are often hard and multi step to implement.
The HTTP referer (a misspelling of referrer) was broken for a while, ever since spammers figured out they can abuse it.
I have sending the referrer header for years in firefox, if a site doesn't accept it I just go somewhere else.
[deleted]
This is going to suck... now 2% of my users won't have HTTPS referrers.
Good.
It's about time...
your privacy will become the joke
I've noticed recently the <Back button is sometimes disabled in Firefox. Related?
Not sure if it's related, but the
> Back button is sometimes disabled in Firefox
for me when I click on a Medium (or similar) link. However I think this is related to containers. It's really annoying, but I'd rather just avoid Medium that abandon Firefox.
> Back button is sometimes disabled in Firefox
for me when I click on a Medium (or similar) link. However I think this is related to containers. It's really annoying, but I'd rather just avoid Medium that abandon Firefox.
Good for firefox. although I can see how this may break vimeo kind of service that only allow embedding video from certain web site.
They still send the domain name on cross-origin requests, so the type of filtering Vimeo does will be unaffected as far as I can tell.
It's only when the target website needs the full source URL (with path and GET parameters) that you may encounter issues.
It's only when the target website needs the full source URL (with path and GET parameters) that you may encounter issues.
Does Firefox 87 still make my MBP a toaster when I turn on a Zoom call through Firefox?
Power usage for same streaming video call on Safari vs. Firefox.
https://i.imgur.com/I7T19d0.png
Power usage for same streaming video call on Safari vs. Firefox.
https://i.imgur.com/I7T19d0.png
Isn't this going to break campaign tracking for Adwords etc.
Not exactly what the actual risk is to privacy here does seem there is a lot of bandwagon jumping going on - a bit like "Elf & Safety" or the Data protection act is trotted out when an organisation wants an excuse not to do something.
Not exactly what the actual risk is to privacy here does seem there is a lot of bandwagon jumping going on - a bit like "Elf & Safety" or the Data protection act is trotted out when an organisation wants an excuse not to do something.
Referrers can leak information about which pages the user was navigating before, which linked websites have no business of knowing.
If you want to do tracking and have control over both pages, just use a different URL, a query string, or something like that.
If you want to do tracking and have control over both pages, just use a different URL, a query string, or something like that.
Will Firefox trim their own header additions when searching?
e.g. here's the page that Firefox generates when I try to search "Dragon Quest XI" in a Private Window on Amazon via the address bar:
https://www.amazon.com/s?k=dragon+quest+xi&link_code=qs&sour...
Note the 'mozilla-20' tag at the end.
e.g. here's the page that Firefox generates when I try to search "Dragon Quest XI" in a Private Window on Amazon via the address bar:
https://www.amazon.com/s?k=dragon+quest+xi&link_code=qs&sour...
Note the 'mozilla-20' tag at the end.
That's not the same as the referrer header; it's mostly invisible to the end user without inspecting the raw request data.
That's an affiliate link. It seems then Mozilla is injecting affiliate codes on Amazon searches.
What does the browser has to do with how amazon implements their search? Amazon probably parses the User Agent and puts mozilla there for some reason.
That seems rather unlikely. If it's not the browser itself, then it's more likely to be an extension inserting it.
I don't understand what Mozilla does to Firefox anymore. What does it mean a page "can" leak private data?
Is there any story about anyone affected by the issue? Does this issue even exist?
It is just breaking another piece of the open web.
It just seems like Mozilla not only surrendered in gaining browser market but also actively acts against open web.
I thought website creators and website users should be in charge of what they want to do. But it seems no. Now Mozilla decides what web standards can be broken.
I'd maybe applaud changes by Mozilla, but with all of these efforts it is not aimed in gaining more users. Firefox does not gain users with such actions. It does not make any sense what is the aim of Mozilla anymore with Firefox.
Is there any story about anyone affected by the issue? Does this issue even exist?
It is just breaking another piece of the open web.
It just seems like Mozilla not only surrendered in gaining browser market but also actively acts against open web.
I thought website creators and website users should be in charge of what they want to do. But it seems no. Now Mozilla decides what web standards can be broken.
I'd maybe applaud changes by Mozilla, but with all of these efforts it is not aimed in gaining more users. Firefox does not gain users with such actions. It does not make any sense what is the aim of Mozilla anymore with Firefox.
If the referrer header contains the full URL from which the user came from it could contain something the user deems private.
Imagine that the URL contains a search string that contains something the user might deem private, or the article is about something a state may deem to be illegal, etc.
This may not be PII but it could be deemed private by the user. The user should be in control of what information could be given to a website. Imagine a scenario where a person is a member of a group that is looking for equal rights in a country that is very much opposed to this. The group uses a tool that happens to make it clear who this group is. The group then links to an article about their cause or similar. Now that country could potentially link things together and demand information about everyone in that group.
It's amazing how little bits of information about you can get you in hot water.
This is a good move imo, and I'm glad that Mozilla is trying to plug these types of things. It's better for everyone.
As an aside, moving to a privacy oriented browser could very well get Firefox more users. Just like Apple's play to being a private and secure mobile OS is getting them users.
Imagine that the URL contains a search string that contains something the user might deem private, or the article is about something a state may deem to be illegal, etc.
This may not be PII but it could be deemed private by the user. The user should be in control of what information could be given to a website. Imagine a scenario where a person is a member of a group that is looking for equal rights in a country that is very much opposed to this. The group uses a tool that happens to make it clear who this group is. The group then links to an article about their cause or similar. Now that country could potentially link things together and demand information about everyone in that group.
It's amazing how little bits of information about you can get you in hot water.
This is a good move imo, and I'm glad that Mozilla is trying to plug these types of things. It's better for everyone.
As an aside, moving to a privacy oriented browser could very well get Firefox more users. Just like Apple's play to being a private and secure mobile OS is getting them users.
So where is the example?
I think you've been given plenty.
You should try to come into these conversations with a lot less attitude. Perhaps I'm reading your tone wrong, and if I am I'm sorry about that.
I always come into these discussions assuming I'm not the smartest person in the room because that means I have more to learn. Maybe give that a try?
You should try to come into these conversations with a lot less attitude. Perhaps I'm reading your tone wrong, and if I am I'm sorry about that.
I always come into these discussions assuming I'm not the smartest person in the room because that means I have more to learn. Maybe give that a try?
RFC-1945: Because the source of a link may be private information or may reveal an otherwise private information source, it is strongly recommended that the user be able to select whether or not the Referer field is sent. For example, a browser client could have a toggle switch for browsing openly/anonymously, which would respectively enable/disable the sending of Referer and From information.
So not sending referrer has always been fine, and I suspect that firefox will have an option to enable the sending somewhere in it.
So not sending referrer has always been fine, and I suspect that firefox will have an option to enable the sending somewhere in it.
Normal people are going to switch browser instead of trying to find the said option, because Firefox is "broken".
I would say that most "normal people" don't even know the browser sends a referrer header, and it doesn't actually do anything for them.
This is useful for site owners, and they have little control over what browser their users use.
And as someone else already pointed out, the other big browser (Chrome) has already done this.
This is useful for site owners, and they have little control over what browser their users use.
And as someone else already pointed out, the other big browser (Chrome) has already done this.
Why would this break anything? Which sites would break just because the referrer policy default setting changed? And why wouldn't sites have fixed that before, since Chrome pushed the same change last year?
Switch to what, exactly? Chrome already does this, according to another poster.
Normal websites are going to be unaffected in terms of user-facing functionality, regardless of whether they use the data gathered from HTTP Referer internally. On the other hand, badly designed websites that do depend on the HTTP Referer to provide functional user experience are going to lose their users in favour of normal websites, because they are "broken"—and always have been.
According to Wikipedia [1] the "Referer" (sic) HTTP header is an optional one so I don't see how they are breaking the open web by sending less data in it under some circumstances.
[1] https://en.wikipedia.org/wiki/HTTP_referer
[1] https://en.wikipedia.org/wiki/HTTP_referer
> Is there any story about anyone affected by the issue? Does this issue even exist
Yes. Yes it does.
Imagine, a web site that has a query parameter with a secret (password reset pages, email unsubscribe, custom feeds, etc). Any images or other assets used in the page will receive the full URL as the referrer, so they can see those secrets.
GitHub fixed this very issue a few years ago. They have a feature to get custom RSS feeds for user activity. There is a secret worry parameter in the URL, and any third party images were receiving that full URL.
Referrer-Policy header is already supported in many browsers.
Yes. Yes it does.
Imagine, a web site that has a query parameter with a secret (password reset pages, email unsubscribe, custom feeds, etc). Any images or other assets used in the page will receive the full URL as the referrer, so they can see those secrets.
GitHub fixed this very issue a few years ago. They have a feature to get custom RSS feeds for user activity. There is a secret worry parameter in the URL, and any third party images were receiving that full URL.
Referrer-Policy header is already supported in many browsers.
If it's a page that contain or receive secret should be using https where referrer is already stripped off
Firefox was using no-referrer-when-downgrade, which only strips refers when navigating from HTTPS to HTTP.
They are switching to strict-origin-when-cross-origin, which cuts the referrer down to just the origin when navigating to a different origin.
They are switching to strict-origin-when-cross-origin, which cuts the referrer down to just the origin when navigating to a different origin.
So where is a real life example?
[deleted]
https://developers.google.com/web/updates/2020/07/referrer-p...
So if this breaks something people probably already noticed. And Mozilla is merely aligning with the browser with the largest market share on this. (Also everyone who wants something different for their sites, it's configurable: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Re... )