HackerTrans
TopNewTrendsCommentsPastAskShowJobs

rsbadger

no profile record

Submissions

Show HN: Storage.to CLI, upload a file, get a link

3 points·by rsbadger·قبل 6 أشهر·0 comments

There and Back Again – An Ecommerce Tale

shoprocket.io
1 points·by rsbadger·قبل 4 سنوات·0 comments

How to setup a secure file server with Nginx

medium.com
4 points·by rsbadger·قبل 4 سنوات·0 comments

“Magic links” can end up in Bing search results, rendering them useless

medium.com
537 points·by rsbadger·قبل 4 سنوات·233 comments

Show HN: Shoprocket.io – Turn any website into a shop in seconds

shoprocket.io
12 points·by rsbadger·قبل 4 سنوات·1 comments

Trustpilot Extorts Businesses

medium.com
1 points·by rsbadger·قبل 4 سنوات·0 comments

comments

rsbadger
·قبل 4 سنوات·discuss
Unfortunately not - the links were indexed and shown in Bing search results
rsbadger
·قبل 4 سنوات·discuss
I do have a robots.txt to block this directory. But Bing only listens to that for what to crawl, not what to index.
rsbadger
·قبل 4 سنوات·discuss
I’m fairly certain they are. My links ended up indexed in Bing search results. The only place they were ever rendered was in private emails to users. Bing should not be indexing that.
rsbadger
·قبل 4 سنوات·discuss
Exactly that.
rsbadger
·قبل 4 سنوات·discuss
This is very useful information. You’d really hope that private emails would be excluded by default…
rsbadger
·قبل 4 سنوات·discuss
The only purpose of this link was to verify that the email address is valid. Once it’s verified, you can login.
rsbadger
·قبل 4 سنوات·discuss
Many people (most?) prefer to signup to services by email address. To do so, those email addresses must be verified. How would you verify it without sending them an email link?
rsbadger
·قبل 4 سنوات·discuss
You’re right that it was a bit of an oversight on my behalf, as the links were only generated after a verified human user action (signup) I had assumed the 1 time links to their email would be safe. But regardless of the link action, it shouldn’t be passing that data to Bingbot to crawl and (possibly) index in search engine results. Private email data should not be shared with search engine crawlers IMO.
rsbadger
·قبل 4 سنوات·discuss
I would be pretty mortified if browsers were using user browser sessions to scan content and pass it to bingbot…? What about if you’re browsing something local? Or your bank account?
rsbadger
·قبل 4 سنوات·discuss
Almost all of them. A good example is Dropbox link you send to someone. I could generate this link to a private file in my Dropbox, email it you, and Bing (may) index it.

https://www.dropbox.com/s/vucien2ns8jktga/denim%20bodywarmer...

I doubt many people realise this when they email "private" links...
rsbadger
·قبل 4 سنوات·discuss
Exactly - seems the safest way is to explicitly block known bots by user agent from even reaching pages you don't want indexed.
rsbadger
·قبل 4 سنوات·discuss
You may be seeing a stale version, try this: https://shoprocket.io/robots.txt?bypass=1

(I made a lot of changes today when testing all, including "visit as Bingbot" from their webmaster tools with and without the URL blocked by robots.txt)
rsbadger
·قبل 4 سنوات·discuss
Quite possibly…
rsbadger
·قبل 4 سنوات·discuss
They were indexed in Bing results, I’ve shared the URL of that in this thread
rsbadger
·قبل 4 سنوات·discuss
I don't think that's very surprising for most people, the real takeaway is that not only will Bing read your emails, but they may also index any links you send and serve them in search results.
rsbadger
·قبل 4 سنوات·discuss
Exactly. I also noticed Bing had accessed some non microsoft tokens too. Even gmail accounts were affected. I assume some people have connected their gmail account to the outlook client?
rsbadger
·قبل 4 سنوات·discuss
Exactly. From what I've heard today it sounds like most apps have an extra step between the email link and the login, usually a JS step, to check for bots.

Not adding that was my downfall I think.
rsbadger
·قبل 4 سنوات·discuss
AFAIK, most of those "private" links are actually just "unlisted" but they're still public. I'm sure Bing is indexing those too...
rsbadger
·قبل 4 سنوات·discuss
Absolutely
rsbadger
·قبل 4 سنوات·discuss
Yeh I was a bit reluctant to post that as it doesn't look great for my app! But here's the results: https://www.bing.com/search?q=https%3A%2F%2Fshoprocket.io%2F...