Tell HN: Opening this page will flag you as abusing GitHub
29 comments
2 years ago, I realised we were accidentally DOS'ing GitHub and they didn't care. I was impressed.
We had software for checking a piece of data. To do this, (we realised later) it fetched a schema from GitHub every time. The context is this software was used infrequently so that was fine - and also there was good reasons why it should reload the schema every time.
We then took this bit of software and dumbly used it in a tool for checking masses of data at once. We did this quite happily for 2 months or so, until one day I was trying to work out why it didn't run that fast. (It could only check 2 or 3 pieces of data a second).
At this point I realised/discovered all the above, went "oh shit" and quickly slapped in a request caching library. We stopped DOS'ing GitHub and the amount of checks we could do per second went right up.
But I'm pretty sure that at no point did GitHub rate limit us or block us during this - for which I was very impressed.
We had software for checking a piece of data. To do this, (we realised later) it fetched a schema from GitHub every time. The context is this software was used infrequently so that was fine - and also there was good reasons why it should reload the schema every time.
We then took this bit of software and dumbly used it in a tool for checking masses of data at once. We did this quite happily for 2 months or so, until one day I was trying to work out why it didn't run that fast. (It could only check 2 or 3 pieces of data a second).
At this point I realised/discovered all the above, went "oh shit" and quickly slapped in a request caching library. We stopped DOS'ing GitHub and the amount of checks we could do per second went right up.
But I'm pretty sure that at no point did GitHub rate limit us or block us during this - for which I was very impressed.
For those that are just curious what the flag looks like:
https://imgur.com/a/BOCUzWP
It looks like, based off a quick test it only blocks access to the website, not the git interaction.
https://imgur.com/a/BOCUzWP
It looks like, based off a quick test it only blocks access to the website, not the git interaction.
I'm curious if it's a legitimate abuse flag or if it's more like a rate limit with some unfortunate phrasing for the humanized message. Like am I just getting blocked at the edge for a bit or would there be a flag on my account somewhere now?
Yeah it's probably a very short-lived rate limit. I got that error the other night, searching a repo for how it returns its version information in the code, and I got that error as shown in the screenshot. I then clicked Back in my browser, clicked another link, and it was fine. /shrug
I guess a malevolent actor could put this link in their website (e.g. as the source of a hidden frame). Greatest impact if it's a developers website.
It makes sense; loading that README performs almost 1700 requests for GitHub resources. That's probably way off the regular GitHub usage and definitely enough to trigger a regular user's rate limit.
In Chrome you can view the source here: "view-source:https[DELETE]://github.com/olikraus/u8g2/wiki/fntlistall" (added the [DELETE] after the protocol to prevent people from accidentally loading the actual website rather than source)
In Chrome you can view the source here: "view-source:https[DELETE]://github.com/olikraus/u8g2/wiki/fntlistall" (added the [DELETE] after the protocol to prevent people from accidentally loading the actual website rather than source)
I want to click this but I don't want to rate limit a computer I use for work.. Can someone imgur this for those worried about clicking it? :D
It's just a page with a lot of small images (all directly loaded from the repo) and a short description for each. After loading half of the images GitHub throttled me and blocked requests with an error message about abuse.
After 2 minutes I was able to access GitHub again but I imagine they will block you for longer if you try to do it again and again.
After 2 minutes I was able to access GitHub again but I imagine they will block you for longer if you try to do it again and again.
Appreciate it!
[deleted]
Could GitHub mitigate the load to their servers by serving files via a caching layer? Obviously the first request would hit everything, but there’s no reason that subsequent requests should cause problems.
I remember, when a page had many-many-many images, we used to call it "56k killer" back in the old days, as it would take a long time to load that page on a 33k or a 56k modem.
Opening in an incognito window only got a fifth of the images to load, but it didn't seem to penalize other pages.
[HN gave me a "trouble serving your request", though...]
[HN gave me a "trouble serving your request", though...]
HN’s rate limiting can be pretty aggressive. I’ve hit it seemingly “just” clicking an upvote immediately after submitting a response (a common pattern as I often only remember to reward a good comment in the voting system after I’ve given it verbal praise).
Not causing issues for me. Maybe it got resolved? Or maybe I just don't have the right github-fu
Just tried from a residential connection on a Macbook. Only about a quarter of the images loaded, with the rest reporting a 429 error. Browsing to other pages afterwards shows me the "Access has been restricted" warning.
Accessing the page and refreshing gets me:
Access has been restricted You have triggered an abuse detection mechanism.
Please wait a few minutes before you try again; in some cases this may take up to an hour.
Access has been restricted You have triggered an abuse detection mechanism.
Please wait a few minutes before you try again; in some cases this may take up to an hour.
It did not cause issue directly for me either initially, but it does trigger the abuse mechanism when I reloaded the page
A simple lazy loading of images would save mega tons of bandwidth to them. This could be implemented in 10 lines of js ... Plus few lines of backend code to rewrite the raw html to make the images lazy loaded.
If you work at github, do this in few hours and be the hero :)
It’s built in nowadays.
<img loading=”lazy” />
<img loading=”lazy” />
Yes but not used here unfortunately or you won't get rate limited because you did 1700 requests in 30 sec just by visiting a github public page
Save some bandwidth and make the user wait for the images, break scrolling, etc.
The page just opened for me. No errors or strange messages.
429 - Too many requests (in an incognito tab)
[deleted]
https://github.com/olikraus/u8g2/wiki/fntlistall
Click at your own risk! ;>