Referer Reality(robinsloan.com)
robinsloan.com
Referer Reality
https://www.robinsloan.com/lab/referer/
22 comments
> Because that’d probably be a perfectly reasonable referrer
Well, it might be better than no referrer, but if the app is a mail app or feed reader or something, it doesn't really mean much more than "Safari" or "Chrome".
Well, it might be better than no referrer, but if the app is a mail app or feed reader or something, it doesn't really mean much more than "Safari" or "Chrome".
I love Robin's newsletters (and the books etc) and love "opinionated operator decisions make the internet go round".
Maybe I'm missing something but it seems like if i go gmail -> via a Robin Sloane Newsletter -> webpage, it seems generally useful to know Robin sent me. Like a backlink that's been liberated?
Is something different happening?
Maybe I'm missing something but it seems like if i go gmail -> via a Robin Sloane Newsletter -> webpage, it seems generally useful to know Robin sent me. Like a backlink that's been liberated?
Is something different happening?
I strongly disagree with a lot of claims here.
>> if it isn’t there, it’s probably for a good reason. > which isn’t really true anymore. For most websites, the majority — not just the plurality, but the majority — of visitors arrive by following a link inside an email or an app
I don't think the statement is factually backed up. At least I hate native apps.
> Even so, my custom query string is, in my calculation, an expression of digital etiquette: rather than dump a load of anonymous traffic on your doorstep, I reveal who’s linking, so a website or online shop operator can trace it back and get in touch, if wanted or needed
Anonymity considered harmful these days?
> a wave of new subscriptions and weren’t sure if they were legitimate; a brief email correspondence assured them that yes,
It's not legitimate unless it's signed (and if anybody gives a shit to verify it).
> Note that a handful of sites do choke on unexpected query strings, including YouTube (!),
This is a good habit IMHO.
Anyway, I'm thankful to the original post because it was a good reminder to re-review my browser settings.
Honestly I don't understand why the EU focused on the stupid cookie law instead of referers which are clearly privacy-violating.
If you use Firefox I recommend you make sure `network.http.referer.XOriginPolicy` set to 1.
>> if it isn’t there, it’s probably for a good reason. > which isn’t really true anymore. For most websites, the majority — not just the plurality, but the majority — of visitors arrive by following a link inside an email or an app
I don't think the statement is factually backed up. At least I hate native apps.
> Even so, my custom query string is, in my calculation, an expression of digital etiquette: rather than dump a load of anonymous traffic on your doorstep, I reveal who’s linking, so a website or online shop operator can trace it back and get in touch, if wanted or needed
Anonymity considered harmful these days?
> a wave of new subscriptions and weren’t sure if they were legitimate; a brief email correspondence assured them that yes,
It's not legitimate unless it's signed (and if anybody gives a shit to verify it).
> Note that a handful of sites do choke on unexpected query strings, including YouTube (!),
This is a good habit IMHO.
Anyway, I'm thankful to the original post because it was a good reminder to re-review my browser settings.
Honestly I don't understand why the EU focused on the stupid cookie law instead of referers which are clearly privacy-violating.
If you use Firefox I recommend you make sure `network.http.referer.XOriginPolicy` set to 1.
I also dont get why adding 'utm_source=Robin_Sloan_sent_me' somehow gels with his calculation it is 'an expression of digital etiquette: rather than dump a load of anonymous traffic on your doorstep, I reveal who’s linking'.
Theres no actual link to the referring website just some bunch of characters which look like someones name and some of which are in the actual referring URL.
And yeah I'm also thankful also to see that firefox setting.
And yeah I'm also thankful also to see that firefox setting.
I am totally sympathetic to the "this isn't a legible link for machines". After all, this utm_source implies that some human is looking at the logs manually (or at least, trying to decipher aggregated utm sources manually). However, we are well into a transition to a world where this isn't true, and 'utm_source=Robin_Sloan_sent_me' is probably just as good as 'utm_source=robinsloan.com' for LLMs of 2026+.
> Honestly I don't understand why the EU focused on the stupid cookie law instead of referers which are clearly privacy-violating.
Neither the ePrivacy directive (commonly called the "cookie law") nor the later GDPR focus on cookies. They are "technology neutral", applying to e.g. URL parameters and HTTP headers too, but just widely misunderstood and badly enforced.
Neither the ePrivacy directive (commonly called the "cookie law") nor the later GDPR focus on cookies. They are "technology neutral", applying to e.g. URL parameters and HTTP headers too, but just widely misunderstood and badly enforced.
It doesn't help that the most visible and best SEO'd sites purporting to explain the GDPR are made by advertising/tracking companies, or firms representing/selling services to them.
I'd guess that the average person doesn't know that the GDPR applies even when you're taking details from people by hand with a pen on paper.
I'd guess that the average person doesn't know that the GDPR applies even when you're taking details from people by hand with a pen on paper.
Yes. My diagnosis that this is due to bad enforcement and the refusal to make or stick to clear enforcement criteria. This lead to the deployment being that companies are as maliciously compliant as possible, with the criteria evolving as a loophole whack-a-mole with 10 year iteration time due to enforcers and courts dragging their feet.
Or set network.http.referer.spoofSource to true.
For the privacy of the user, nowadays the referer header are either shortened or voided by the browser. You reintroducing the string by a side-channel could be seen by users as a backstab, even if you see it as "etiquette".
At the very least, people seem to have settled on one or two args to use for this. e.g. "utm_source".
I tried entering ?utm_source=foo on Chris's site when it was posted the other day, and was surprised it didn't trigger the page. Pretty sure one or more of my firefox extensions remove that specific one from URLs before making the request.
Facebook combining tracking and IDs into an opaque UUID in URLs so they can't be removed is a level of user hostility beyond this.
I tried entering ?utm_source=foo on Chris's site when it was posted the other day, and was surprised it didn't trigger the page. Pretty sure one or more of my firefox extensions remove that specific one from URLs before making the request.
Facebook combining tracking and IDs into an opaque UUID in URLs so they can't be removed is a level of user hostility beyond this.
Firefox’s Enhanced Tracking Protection removes a few such parameters.
Now I have the stupid contrary idea, require a specific query string parameter to be present…
Now I have the stupid contrary idea, require a specific query string parameter to be present…
I must correct myself: utm_* is not part of Firefox’s query stripping, which only removes about 23 parameters: https://firefox.settings.services.mozilla.com/v1/buckets/mai...
For me, it comes from uBlock Origin’s AdGuard/uBO – URL Tracking Protection list, which removes almost three hundred parameters unconditionally and a lot more conditionally: https://github.com/uBlockOrigin/uAssets/blob/master/filters/...
(A few of the parameters in that list are a little alarming in a you-could-easily-name-a-legitimate-parameter-this sort of way, like user_email_address, maf and taid.)
For me, it comes from uBlock Origin’s AdGuard/uBO – URL Tracking Protection list, which removes almost three hundred parameters unconditionally and a lot more conditionally: https://github.com/uBlockOrigin/uAssets/blob/master/filters/...
(A few of the parameters in that list are a little alarming in a you-could-easily-name-a-legitimate-parameter-this sort of way, like user_email_address, maf and taid.)
If there's a query parameter that you have a legitimate use for, like `q` for searching, obviously you should configure your web server to let it through.
Even in that case, you might want to block unexpected values as early as possible in your stack. For example, if you have a legitimate use for a certain set of `utm_source` values, but someone sends you bobby tables, you probably shouldn't log it blindly.
Ditto for the Referer header -- there's a lot of spam, and some of those strings might even be dangerous. You can't trust any of them anymore.
Even in that case, you might want to block unexpected values as early as possible in your stack. For example, if you have a legitimate use for a certain set of `utm_source` values, but someone sends you bobby tables, you probably shouldn't log it blindly.
Ditto for the Referer header -- there's a lot of spam, and some of those strings might even be dangerous. You can't trust any of them anymore.
i'm confused, why would you go out of your way to add this tracking info to external links voluntarily? it doesn't benefit you, it just helps other websites segment their traffic?
When other website owners realize how much traffic you send them that is good for you. Maybe they want to sponsor that newsletter if it’s driving a lot of business.
Opinionated: I dont get the issue about people (incl. bots) visiting your website, regardless of where they originate from and what url they use.
Publishing something online means I want it to be seen.
Relying or enforcing Referer or query string basically contradicts the intent. Wanting to know where your users come from could be legit but then you basically distort the "public web" aspect to a "public trap" where you try to sneak on visitors.
Not my vibe
Publishing something online means I want it to be seen.
Relying or enforcing Referer or query string basically contradicts the intent. Wanting to know where your users come from could be legit but then you basically distort the "public web" aspect to a "public trap" where you try to sneak on visitors.
Not my vibe
> which isn’t really true anymore. For most websites, the majority — not just the plurality, but the majority — of visitors arrive by following a link inside an email or an app
I highly doubt this is true, unless your definition of "app" includes "web browser".
I highly doubt this is true, unless your definition of "app" includes "web browser".
If your traffic is mostly links from social media, it may well be accurate - up to orders of magnitude more folks are using Facebook/Twitter/BlueSky/LinkedIn from the mobile apps than from desktop browsers
I get adding query strings on links to your own domain, but from the original post, it sounds like the author was seeing third parties add query strings to links to their domain and I don't see what the point in doing that is, unless the entity adding those quey parameters is able to somehow intercept them between the browser and the website (which is possible if they control the browser or a cdn).
[deleted]
When I said, “if it isn’t there, it’s probably for a good reason”, I was fully aware of native apps, and decided to gloss over them, because I doubt (on no evidence) they’re so significant for me, and things like email and feed readers may well be web apps anyway. And because I’ve tended to fine nuance in the past, and am trying being concise instead.
While thinking about it—is it possible for a browser to know which app sent it the link, e.g. androidapp://com.example.app? Because that’d probably be a perfectly reasonable referrer, if a browser wanted to send it.
> Note that a handful of sites do choke on unexpected query strings, including YouTube (!)
I want to learn more about this.