CloudFlare is ruining the internet for me (2016)(slashgeek.net)
slashgeek.net
CloudFlare is ruining the internet for me (2016)
https://www.slashgeek.net/2016/05/17/cloudflare-is-ruining-the-internet-for-me/
181 comments
Yeah, that was my first thought when reading this.
It’s not that Cloudflare is ruining the internet for you. It’s the loads of illegitimate traffic and attacks that just so happen to originate in your country.
It’s not that Cloudflare is ruining the internet for you. It’s the loads of illegitimate traffic and attacks that just so happen to originate in your country.
This parallels the conversation going on in the ML world right now.
"They just so happen to look like gorillas!"
This is not okay.
I know it's just a pile of linear algebra and there's no conspiracy to screw a particular country.
But once it's brought up that your system is causing harm by increasing inequality in the world, you can't just say "But that's what the algo is spitting out!".
At least try to fix it.
"They just so happen to look like gorillas!"
This is not okay.
I know it's just a pile of linear algebra and there's no conspiracy to screw a particular country.
But once it's brought up that your system is causing harm by increasing inequality in the world, you can't just say "But that's what the algo is spitting out!".
At least try to fix it.
> 1+1=2, but I want it to be 3, why are you not attempting to fix it?
How do you fix that which is not broken?
I understand the problem you are trying to point out, but the problem is not in the technology, it’s in the application.
How do you fix that which is not broken?
I understand the problem you are trying to point out, but the problem is not in the technology, it’s in the application.
[deleted]
What learned state says it's okay to prompt for captchas every other click if you've never been a bad internet citizen, exactly?
Help me here.
You aren't being specific. You only restated that you leave a lot of visitors at the mercy of statistical algorithms.
Help me here.
You aren't being specific. You only restated that you leave a lot of visitors at the mercy of statistical algorithms.
> To address other parts of the blog though, there is nothing in our system that defaults to "do X for IP/Country y"... what there is, is learned state about traffic, clients, and the many layers of customer expressed configuration.
If it is not IP/country, then what would cause this user to be prompted for captcha repeatedly?
If it is not IP/country, then what would cause this user to be prompted for captcha repeatedly?
Shared ip address most likely. With which someone else is triggering the captcha. Or even a range of ips which are or have been allocated to someone else.
I imagine the solution should come about as follows.
Customers complain about captchas and seek alternative isps.
Someone else provides better connections without tainted ip addresses (without sketchy customers)
OR, the current isp decides to crack down on customers who are being labeled as abusive. Problem solved.
Someone is likely providing cheap internet to sketchy people, who taint the experience for other users. It is on them to crack down and stop selling to sketchy users. Ban their payment methods/identities, so cloudflare doesn't have to ban ips.
I imagine the solution should come about as follows.
Customers complain about captchas and seek alternative isps.
Someone else provides better connections without tainted ip addresses (without sketchy customers)
OR, the current isp decides to crack down on customers who are being labeled as abusive. Problem solved.
Someone is likely providing cheap internet to sketchy people, who taint the experience for other users. It is on them to crack down and stop selling to sketchy users. Ban their payment methods/identities, so cloudflare doesn't have to ban ips.
Companies like CloudFlare are contributing to the erosion of user privacy in the name of security. I experience the same kind of issues that the author of the article highlights, just because I take some privacy measures. It's my view that so-called security on the Internet, which implies confidentiality, integrity, and availability, is getting increasingly worse for end users. If I want to give up all of my privacy, the Internet is more convenient to use than ever. But for the privacy-conscious user, the Internet is becoming less usable by the day.
The article is complaining that CloudFlare (as a result of them refusing to track you) requires people to solve captchas over and over again. How is this hurting your privacy exactly?
Correct me if I’m wrong but isn’t one of the strongest signals recaptcha uses to prove “I am not a robot” the fact that you’re currently logged into a google account, have the associated cookies, etc? This “phoning home” from all these sites is a clear privacy challenge.
The good news is Safari and Firefox are making this harder and harder. The bad news is that you’ll be solving a lot more captchas.
The good news is Safari and Firefox are making this harder and harder. The bad news is that you’ll be solving a lot more captchas.
I wonder if privacy conscious browsers making their users have to solve more captchas will lead to more people disabling the privacy features... or there will be a time when captchas may become micro transaction based, so for a fee you might be able to skip them. A browser with support for an anonymous cryptocurrency to make the payments should be private and easy to use enough, but it might become an interesting world where you need to pay to remain anonymous.
Try creating a free email account on ProtonMail. I think they have an interesting approach very similar to what you're describing.
reCAPTCHA takes time and effort. It includes intentional delays and a random number of iterations (1-3). It also randomly fails and requires reloading the page. Instead of 2 second page loads, you spend 15-20 seconds. This makes browsing over a VPN frustratingly slow.
I'm convinced that reCAPTCHA is now just one of Google's tools to keep us all trackable and stop us from protecting our privacy. And CloudFlare is bringing it to half of the Internet. Bad bad bad. And shame.
I'm convinced that reCAPTCHA is now just one of Google's tools to keep us all trackable and stop us from protecting our privacy. And CloudFlare is bringing it to half of the Internet. Bad bad bad. And shame.
> This makes browsing over a VPN frustratingly slow.
This is my pet peeve, too. I browse over a VPN all the time and these things pop up all the time and I hate them. I wouldn't have a problem with this mechanism if it worked, but it doesn't! It will randomly fail in various silly ways: new challenges are faded in so slowly there's no way you can wait for it, other times it will just keep on asking you to solve new challenges even though you're solving them correctly (I counted them once and it asked me to solve one 11 times before I gave up). And sometimes it will simply fail to load properly, or it will break websites in various ways.
Lately, I've taken to solving the challenges badly in subtle ways, just because I can. If I'm gonna train Google's neural networks for free, I might as well train them badly...
This is my pet peeve, too. I browse over a VPN all the time and these things pop up all the time and I hate them. I wouldn't have a problem with this mechanism if it worked, but it doesn't! It will randomly fail in various silly ways: new challenges are faded in so slowly there's no way you can wait for it, other times it will just keep on asking you to solve new challenges even though you're solving them correctly (I counted them once and it asked me to solve one 11 times before I gave up). And sometimes it will simply fail to load properly, or it will break websites in various ways.
Lately, I've taken to solving the challenges badly in subtle ways, just because I can. If I'm gonna train Google's neural networks for free, I might as well train them badly...
>'Lately, I've taken to solving the challenges badly in subtle ways, just because I can."
Can you elaborate on this technique? Do you just purposefully solve them incorrectly a number of times before solving them correctly or something else?
Can you elaborate on this technique? Do you just purposefully solve them incorrectly a number of times before solving them correctly or something else?
If it asks me to identify traffic lights, for example, I also include a couple of squares that aren't traffic lights, like a tree on the side of the road. I'm more selective than a random algorithm, so that correctly gets me labeled as a meatbag, but I'm definitely not providing good answers, either.
At this point, to be honest, I'm really not sure about the effectiveness of this mechanism anymore. It seems to me like a poorly-trained neural net with convincing human-like interaction (e.g. clicks spaced unequally) could produce exactly slightly-incorrect results that let me through. There's no way a smart 17 year-old wizzkid hasn't figured it out already and getting rich selling it to spammers in onion space.
At this point, to be honest, I'm really not sure about the effectiveness of this mechanism anymore. It seems to me like a poorly-trained neural net with convincing human-like interaction (e.g. clicks spaced unequally) could produce exactly slightly-incorrect results that let me through. There's no way a smart 17 year-old wizzkid hasn't figured it out already and getting rich selling it to spammers in onion space.
This is a downright lie. CloudFlare isn't refusing to track people, CloudFlare is outsourcing the tracking.
My computer needs to contact Google to make those captchas work
They don’t make you solve captchas if they have enough information about you to think you’re unlikely to be abusive.
If you're on an ip that is part of a pool they consider unsafe, it's every time. When I use one of my vpns to browse, i end up skipping all websites using cloudfare.
This is a lot and it makes you realise how dangerous this situation is. Way too much traffic goes through cloudflare.
This is a lot and it makes you realise how dangerous this situation is. Way too much traffic goes through cloudflare.
Isn’t that like complaining that your insurance costs more if you live in a high theft neighborhood? I mean, it’s not a good situation but if you’re sharing network space with a lot of dodgy activity is the real problem that the site owners wanted protection from those threats or that your network provider doesn’t have an effective abuse management process?
if they consider the subnet you’re on unsafe have you taken the issue up with your VPN provider? if they don’t have a negative stance towards abuse and you continue to use them then you’re actually enabling this behavior. i also wouldn’t be surprised if they considered all vpns unsafe blocks. TOR is usually blocked entirely for abuse reasons.
I take privacy measures too (no VPN though) and I get Recaptcha on some sites because of that, but not particularly on Cloudflare-hosted sites as far as I can see.
Is the situation improved by using the Privacy Pass add-on [1] from Cloudflare?
Supposedly:
>Privacy Pass is a Chrome/Firefox browser extension to make browsing Cloudflare-protected websites a better experience for users. In particular, if a user IP address is designated to have a poor reputation then the user may have to solve a Cloudflare CAPTCHA page before they can gain access to such websites. Privacy Pass uses elliptic curve cryptography to generate 'anonymous' tokens after a single CAPTCHA page is solved. These tokens can be used in future engagements with Cloudflare websites to prevent having to solve more CAPTCHAs. The extension generates 30 tokens for each CAPTCHA solution and thus can be used to reduce CAPTCHA pages for each user by a similar factor.
[1] https://support.cloudflare.com/hc/en-us/articles/11500199265...
Supposedly:
>Privacy Pass is a Chrome/Firefox browser extension to make browsing Cloudflare-protected websites a better experience for users. In particular, if a user IP address is designated to have a poor reputation then the user may have to solve a Cloudflare CAPTCHA page before they can gain access to such websites. Privacy Pass uses elliptic curve cryptography to generate 'anonymous' tokens after a single CAPTCHA page is solved. These tokens can be used in future engagements with Cloudflare websites to prevent having to solve more CAPTCHAs. The extension generates 30 tokens for each CAPTCHA solution and thus can be used to reduce CAPTCHA pages for each user by a similar factor.
[1] https://support.cloudflare.com/hc/en-us/articles/11500199265...
So wait, "we shipped a bug, so we made a browser extension that lets you circumvent the bug". That's cloudflare's answer? I'm not impressed.
EDIT: people seem to be confused as to what bug cloudflare shipped. The bug is not having people solve captchas because their IP has a bad reputation. It's having them solve it over and over again.
You can put it however you want it, but if my app's UX is fine without cloudflare and it's shit with cloudflare, for a small but significant percentage of my users, then CF has a bug.
EDIT: people seem to be confused as to what bug cloudflare shipped. The bug is not having people solve captchas because their IP has a bad reputation. It's having them solve it over and over again.
You can put it however you want it, but if my app's UX is fine without cloudflare and it's shit with cloudflare, for a small but significant percentage of my users, then CF has a bug.
Thwarting denial of service attacks isn’t a bug.
You seem to be confused about what your rights are around website availability. Hint: you have no rights. Absent specific coercion by government, the owner of the website had all the rights. If she wants to require you to solve a Where’s Waldo first, that’s her prerogative. Your choice is to accept the terms or go elsewhere.
You seem to be confused about what your rights are around website availability. Hint: you have no rights. Absent specific coercion by government, the owner of the website had all the rights. If she wants to require you to solve a Where’s Waldo first, that’s her prerogative. Your choice is to accept the terms or go elsewhere.
It's discrimination by country/region.
It's like saying: oh, you are from Africa or Asia. The chance is higher you are a criminal, so do this test first.
Which is completely legal and encouraged. Here's an example: if you've ever shipped an ad-monetized free app, you've probably disabled regions like Russia, Iran, North Korea, etc.
You know why? Because the ad-revenue is worthless (and often malicious) and the users will be more trouble than they are worth. Same thing is happening with net traffic from other low value regions. One star reviews because users from $banned_region are complaining about lag due to their crappy wifi and/or some other issue you have no control over (defective ram in their 6 year old 2nd hand phone comes to mind)? Sign me up!
You know why? Because the ad-revenue is worthless (and often malicious) and the users will be more trouble than they are worth. Same thing is happening with net traffic from other low value regions. One star reviews because users from $banned_region are complaining about lag due to their crappy wifi and/or some other issue you have no control over (defective ram in their 6 year old 2nd hand phone comes to mind)? Sign me up!
Another example: on Ebay, one bit of bog standard anti-fraud advice is prohibiting international bids. This is because the overwhelming majority of bidders living in certain countries are fraudsters. The tiny slice of legitimate traffic attempting to make international purchases is not worth the massive increase in exposure to fraud risk.
I believe selling your products in some of those counties can get you in legal hot water as well.
And the problem is worse because, apparently, even solving the captchas repeatedly from a given IP address doesn't make it whitelisted, either. So, it fits the very definition of discrimination against a whole wider group, where the individual actions of any individual actors don't matter.
I’ve lived in Vietnam for the past 5 years and experienced these issues first hand. I’m also part of the team responsible for maintaining a relatively aggressive set of Cloudflare WAF rules at my current employer.
In these developing countries, great swathes of users are accessing the internet behind carrier-grade NAT.
This makes it increasingly likely that any individual user is sharing a public-facing IP with one or more bad actors.
In my experience, I’ve never had to solve more than one CAPTCHA per domain, and frankly clicking a checkbox isn’t that hard.
As far as discrimination goes, this is a much friendlier solution than just immediately rejecting connection requests from certain CIDRs, which is what would otherwise be happening.
In these developing countries, great swathes of users are accessing the internet behind carrier-grade NAT.
This makes it increasingly likely that any individual user is sharing a public-facing IP with one or more bad actors.
In my experience, I’ve never had to solve more than one CAPTCHA per domain, and frankly clicking a checkbox isn’t that hard.
As far as discrimination goes, this is a much friendlier solution than just immediately rejecting connection requests from certain CIDRs, which is what would otherwise be happening.
> In my experience, I’ve never had to solve more than one CAPTCHA per domain, and frankly clicking a checkbox isn’t that hard.
If it were that easy, there would be little complaint; the complaints seem to be that people get stuck on capchas indefinitely.
If it were that easy, there would be little complaint; the complaints seem to be that people get stuck on capchas indefinitely.
>"In these developing countries, great swathes of users are accessing the internet behind carrier-grade NAT."
Do you have any citations that CGN is any more prevalent in developing counties than in say Western Europe or the US? The last report from RIPE that I read indicates CGN usage in substantial in both the RIPE and APNIC regions.[1] How would IPv4 resource exhaustion be an economic issue?
>"In my experience, I’ve never had to solve more than one CAPTCHA per domain, and frankly clicking a checkbox isn’t that hard"
I imagine if you are personally "responsible for maintaining a relatively aggressive set of Cloudflare WAF rules" as you stated, you've probably become quite proficient at solving CAPTCHAs. I think people that don't mind jumping through hoops are a minority. Also just even if something isn't hard does not mean its any less annoying and degrading of the user experience. Those things are not mutually exclusive.
[1] https://ripe73.ripe.net/presentations/21-ripe73_cgn_richter....
Do you have any citations that CGN is any more prevalent in developing counties than in say Western Europe or the US? The last report from RIPE that I read indicates CGN usage in substantial in both the RIPE and APNIC regions.[1] How would IPv4 resource exhaustion be an economic issue?
>"In my experience, I’ve never had to solve more than one CAPTCHA per domain, and frankly clicking a checkbox isn’t that hard"
I imagine if you are personally "responsible for maintaining a relatively aggressive set of Cloudflare WAF rules" as you stated, you've probably become quite proficient at solving CAPTCHAs. I think people that don't mind jumping through hoops are a minority. Also just even if something isn't hard does not mean its any less annoying and degrading of the user experience. Those things are not mutually exclusive.
[1] https://ripe73.ripe.net/presentations/21-ripe73_cgn_richter....
>1.The IP address you are on has shown problematic activity online recently in one of our data sources. If you would like to look your IP up, then please look your IP up at Project Honeypot. If the IP address shows data for malicious activity, you can see why there. You can also attempt to whitelist your IP directly on that page by connecting from that IP. If no bad activity is seen from the IP address after a two-week period, then the challenge behavior will stop against that IP address.
https://support.cloudflare.com/hc/en-us/articles/203366080-W...
https://support.cloudflare.com/hc/en-us/articles/203366080-W...
Probably because those IP’s cycle, or get shared between a number of people. If you know that the IP has switched between illegitimate and legitimate 10 times before, you can’t just assume that it’s now valid after one captcha.
Discrimination by country/region often makes tremendous sense.
I tell Cloudflare to block all traffic from China because my services derive zero contribution and zero potential value from the Chinese market. The maximum potential positive contribution from China is near zero. The overwhelmingly likely contribution from China is attacks from within the country.
So, to summarize, in my particular case China provides nearly zero positive value and China is simultaneously one of the biggest attack origin countries. It would be the wrong decision to not aggressively discriminate against their traffic: I lose, in real terms, absolutely nothing from blocking all Chinese traffic.
I tell Cloudflare to block all traffic from China because my services derive zero contribution and zero potential value from the Chinese market. The maximum potential positive contribution from China is near zero. The overwhelmingly likely contribution from China is attacks from within the country.
So, to summarize, in my particular case China provides nearly zero positive value and China is simultaneously one of the biggest attack origin countries. It would be the wrong decision to not aggressively discriminate against their traffic: I lose, in real terms, absolutely nothing from blocking all Chinese traffic.
I’ve been in the same boat with my startup, 99% of SSH logins and lame, phpmyadmin-style attacks came from China. However I would ask is what you’re doing really good for humanity? I don’t personally think it’s ethical to block entire countries or regions from a service. China may not provide value to you, but you may provide immense value to people in China.
Maybe it would help you to travel the world more, but once I did I had a different view of things. The internet is truly a global entity, and the more we can do to keep the Internet unified the closer we can bring the planet together. To me that’s a much more important goal than short term profits or mitigating trivial attacks with poorly thought out geo-restrictions.
Maybe it would help you to travel the world more, but once I did I had a different view of things. The internet is truly a global entity, and the more we can do to keep the Internet unified the closer we can bring the planet together. To me that’s a much more important goal than short term profits or mitigating trivial attacks with poorly thought out geo-restrictions.
Except that it is quite likely the best quality Chinese attacks come from compromised machines in the United States.
there is still value in screening out all the low-quality attacks, though
Are you saying that OP is trying to do denial of service attacks? Stopping random users from accessing web sites isn't thwarting a DOS attack. If they classify random users as part of some bot net, it sounds like their algorithms are buggy.
I completely agree the website owner has the right to run whatever they want, so thanks for saying that as its something that too often gets overlooked.
At this point I basically refuse to use things with recaptcha or the stupid little cloud flare dots. I just close the tab and move on. I am just so tired of little dots, storefronts, cars, etc...
At this point I basically refuse to use things with recaptcha or the stupid little cloud flare dots. I just close the tab and move on. I am just so tired of little dots, storefronts, cars, etc...
You calling this a "bug" makes it sound like you expect Cloudflare to violate your privacy by tracking which websites you've visited in order to determine that you're not a threat. It sounds like the point of Privacy Pass is the only information it gives Cloudflare is that you've solved a Captcha recently but otherwise provides no information about where you did it (beyond the fact that it's a Cloudflare property).
Given the fact that they've resorted to providing an extension to do this, this suggests that they've deliberately engineered their services to not have access to that data internally, and that's a good thing.
Given the fact that they've resorted to providing an extension to do this, this suggests that they've deliberately engineered their services to not have access to that data internally, and that's a good thing.
If captchas are a bug, try running your own popular web service, and good luck keeping away the spam.
An extension is easy to install and is a reasonable way for the CDN to verify that you're not a spammer without requiring you to repeatedly prove it whenever your IP changes.
An extension is easy to install and is a reasonable way for the CDN to verify that you're not a spammer without requiring you to repeatedly prove it whenever your IP changes.
Your comment would make perfect if this whole captcha thing was required in order to post comments.
But they require it for static-like content that any decent site should be serving from cache.
But they require it for static-like content that any decent site should be serving from cache.
Should we be building the internet such that a single website can make it effectively unusable for any user at their arbitrary whim?
You mean, the internet that allows web property owners to elect to protect themselves from vandalism?
That sounds like a powerful use of personal choice to me -- allowed by an internet that (still) allows individuals to make choices in their own best interests.
That sounds like a powerful use of personal choice to me -- allowed by an internet that (still) allows individuals to make choices in their own best interests.
The problem is that it's often uninformed choice. Some people at LAX, for example, decided that my whole AS has no business accessing their website. (Yes, an international airport blocking international visitors — how cute.) And Cloudflare is the enabler.
Notice that you never see Akamai presenting these messages that you've been blocked.
Most of these pages where you get blocked are something that looks entirely static, should be cachable with the most basic nginx if dynamically generated, yet Cloudflare tells everyone that they need to protect such content from the users. (Some of their newer competitors that protect from more "bots" are even worse, BTW.)
Notice that you never see Akamai presenting these messages that you've been blocked.
Most of these pages where you get blocked are something that looks entirely static, should be cachable with the most basic nginx if dynamically generated, yet Cloudflare tells everyone that they need to protect such content from the users. (Some of their newer competitors that protect from more "bots" are even worse, BTW.)
I don't use cf, I'm running some mail services but i do block entire AS's after 5 brute force attacks from different IP addresses from same AS regardless of country of origin. This are always modem / routers left with default password, IP cameras with default password, various IoT devices with default password or all of the above with vulnerable firmware with CVE's dating way back. I think that if you are unable or can't be bothered to change the default password for your device you don't deserve internet access. There is much need for something like natural selection on the internet. It is getting to crowded out there.
An AS with an /8 is decidedly different from an AS with a /24. There could easily be millions of complete strangers behind a single ISP AS. Not saying you can’t choose whatever criteria for your service, but trying to pass off five-different-attacker-IPs-per-AS as fair is silly.
Edit: Even the CIDR block size isn’t a good indicator of the actual network size, due to NAT.
Edit: Even the CIDR block size isn’t a good indicator of the actual network size, due to NAT.
Their AS blocking functionality is based on the free "maxmind geoip2 ASN" database, LAX could have chosen to set up a nginx module or site middleware to perform the same block. CF's service offering is making this configuration easier and shifting thinks like having to update the DB onto CF.
And how's that any better? What's the likelihood that LAX would bother to block my AS if it wasn't a simply click courtesy of Cloudflare?
It's like that IBM saying: no-one's been fired for buying IBM. Doesn't make it a good choice, though.
It's like that IBM saying: no-one's been fired for buying IBM. Doesn't make it a good choice, though.
How exactly can one vandalize a static web page?
The worst I've seen is the SPIN website which always requires a captcha.
Cloudflare is just one more middleman extending their tentacles over the web.
The worst I've seen is the SPIN website which always requires a captcha.
Cloudflare is just one more middleman extending their tentacles over the web.
> web property owners
WTH is this newspeak, just say website owners.
WTH is this newspeak, just say website owners.
[deleted]
> a single website can make it effectively unusable for any user at their arbitrary whim
CF is a free service. Websites can choose to use it or not, and it certainly does not dictate the nature of the internet.
CF is a free service. Websites can choose to use it or not, and it certainly does not dictate the nature of the internet.
Nor the world or the internet is this black and white. And heartbleed begs to differ.
I'm not suggesting anything is black and white. No idea why you brought up heartbleed. You're free to choose to use free services, open source software, public resources etc. or not. That's not black and white, it's a bunch of choices you can make about resources without requiring money.
The author runs an ISP. Is this extension for the end user or can it be used at the ISP level for all affected IP addresses?
> .. to generate 'anonymous' tokens
Yeah, right. I think double quotes is more appropriate here.
Yeah, right. I think double quotes is more appropriate here.
The OP has posted a follow-up, but it’s also from 2016:
https://www.slashgeek.net/2016/06/07/cloudflare-making-inter...
Two things that have changed since then are 1) Cloudflare’s Privacy Pass browser extension and 2) their significant network expansion, both of which would be likely to affect the experience described.
One thing that has not changed, however, is how many (typically unsophisticated) web site operators actively search their logs for signs of suspicious / bot activity and then institute manual blocks in the hope of catching all of them. This is often done with very blunt instruments, such as whole-country blocks.
In contrast, people who are confident about their infrastructure can deal with the background noise of the Internet appropriately—by doing nothing.
Two things that have changed since then are 1) Cloudflare’s Privacy Pass browser extension and 2) their significant network expansion, both of which would be likely to affect the experience described.
One thing that has not changed, however, is how many (typically unsophisticated) web site operators actively search their logs for signs of suspicious / bot activity and then institute manual blocks in the hope of catching all of them. This is often done with very blunt instruments, such as whole-country blocks.
In contrast, people who are confident about their infrastructure can deal with the background noise of the Internet appropriately—by doing nothing.
I think a sizable portion of the small websites that use CF are on so much of a budget that they use a $10/month VPS for hosting then turn to CF for "ddos protection" in order to handle any spikes in traffic that would take down their under-provisioned server. Issue here is that CF's layer 7 flood protection (that's even on the free plans, you can use the rate limiting service for smaller-scale floods) doesn't take effect unless you are really getting hit hard, the cutoff is probably 500 requests per second or more.
When CF fails to protect against the small 30 request per second flood once, they're probably going to go through and add a bunch of aggressive blocks like blocking entire countries, as you've said.
When CF fails to protect against the small 30 request per second flood once, they're probably going to go through and add a bunch of aggressive blocks like blocking entire countries, as you've said.
Experienced the same during my Asia travels with Cloudflare 2016/2017.
Actually that was the biggest reason/negative ad not to consider Cloudflare when choosing a CDN. They simply don't respect Asia and other non first class countries with their standard settings.
It's like good Internet only for the rich people in the world!
I'm aware on the Chrome extension (really, why should I install this sh&+t on the first place?) and that you can change Cloudflare settings. But the usual IT admin won't change this settings and f&+ck up Asia.
I'm aware on the Chrome extension (really, why should I install this sh&+t on the first place?) and that you can change Cloudflare settings. But the usual IT admin won't change this settings and f&+ck up Asia.
Asia is the #1 attacker of all websites I work with.
Since 100% of the traffic (based on our analysis) coming from Asia is not legitimate business traffic, how would you advise those responsible for these sites security to handle this?
Edit: I have no interest in using Cloudflare...
Since 100% of the traffic (based on our analysis) coming from Asia is not legitimate business traffic, how would you advise those responsible for these sites security to handle this?
Edit: I have no interest in using Cloudflare...
Similar for me. From the 732745 login attempts last month, 52% were from China, followed by US with 13%. Here is a graph: https://i.imgur.com/YPAuTXO.png
The sheer volume of bot traffic surprised me at first, especially since my website has zero human visitors as far as I can tell, but the numbers are consistent month after month.
Nevertheless, my $1/month VPS can handle the traffic without a problem, so I see no need to ban or rate limit any IPs, especially since I hate captchas with a passion.
The sheer volume of bot traffic surprised me at first, especially since my website has zero human visitors as far as I can tell, but the numbers are consistent month after month.
Nevertheless, my $1/month VPS can handle the traffic without a problem, so I see no need to ban or rate limit any IPs, especially since I hate captchas with a passion.
> my $1/month VPS can handle the traffic without a problem, so I see no need to ban or rate limit any IPs, especially since I hate captchas with a passion.
I run a company that routinely scrapes government run, public domain websites. Sadly, many of these sites come with captchas. We can easily bypass these captchas by paying roughly $1.50/1000 captchas, but when scraping millions of pages a month, these costs become significant.
As far as I can tell, adding a captcha to a site does nothing to prevent bots, it just alters the economics of any business that relies on the data. I understand that bots can potentially slow down servers and cause disruptions for human users, but for the handful of government agencies that actually talk to us, we happily restrict scraping to certain hours of the day or limit overall traffic to a reasonable level. I would go further and happily give the money we're spending on solving captchas back to the government so they can upgrade their servers and make the system better for everyone.
For those that are conducting nefarious activities, captchas likely do nothing. For individuals, they are annoying. For legitimate scraping companies, they are a needless expense. Captchas are pretty obsolete.
I run a company that routinely scrapes government run, public domain websites. Sadly, many of these sites come with captchas. We can easily bypass these captchas by paying roughly $1.50/1000 captchas, but when scraping millions of pages a month, these costs become significant.
As far as I can tell, adding a captcha to a site does nothing to prevent bots, it just alters the economics of any business that relies on the data. I understand that bots can potentially slow down servers and cause disruptions for human users, but for the handful of government agencies that actually talk to us, we happily restrict scraping to certain hours of the day or limit overall traffic to a reasonable level. I would go further and happily give the money we're spending on solving captchas back to the government so they can upgrade their servers and make the system better for everyone.
For those that are conducting nefarious activities, captchas likely do nothing. For individuals, they are annoying. For legitimate scraping companies, they are a needless expense. Captchas are pretty obsolete.
> adding a captcha to a site does nothing to prevent bots, it just alters the economics of any business that relies on the data
Definitely agreed. Recently I have been working on a side-project that makes use of bypassing/placating reCaptcha and it has been trivial and not so costly.
If it is accounts you're creating, it simply puts a "reasonable" price on account creation. If it is about scraped content, once again, does the same. However these costs already existed in terms of compute resources and time anyway. Captchas hardly made it any harder.
Definitely agreed. Recently I have been working on a side-project that makes use of bypassing/placating reCaptcha and it has been trivial and not so costly.
If it is accounts you're creating, it simply puts a "reasonable" price on account creation. If it is about scraped content, once again, does the same. However these costs already existed in terms of compute resources and time anyway. Captchas hardly made it any harder.
Where do you get a $1/month VPS?
A few years ago, I found a promotional offer on lowendbox.com for chicagovps.net, but currently there are none that cheap.
However, keep in mind that low price often comes at a cost. For example, cloudatcost had a cheap VPS with a onetime payment for life (yeah, I know, too good to be true), but then retroactively invented a maintenance fee. Also several days of downtime were not uncommon.
However, keep in mind that low price often comes at a cost. For example, cloudatcost had a cheap VPS with a onetime payment for life (yeah, I know, too good to be true), but then retroactively invented a maintenance fee. Also several days of downtime were not uncommon.
Cool chart but... why are the countries not labelled?
Oops, I changed the y-scale from absolute values to percentage, but forgot to also update the y-position of the country labels, so they are way off-screen (pyplot does not support bar labels, so they have to be drawn by hand). Should be fixed now: https://i.imgur.com/YPAuTXO.png
Consider it a community-run pentest, and look for them to actually get past the gatekeepers. Set an alert for a successful login from an “Asian” IP and push them towards a honeypot. They’re doing work for you for free; take advantage of it.
Attacker means what? Serving too much traffic to bots (because that is what this captchas circumvent).
How about that you can measure traffic and reaction times depending on region and also detect peaks? Ever seen a Google captcha on Google search (yes they have them too), it's much much smarter and not saying stupidly to "wrong" countries: here is a captcha.
Not to mention that it's not just a matter of filling some random words anymore but more like a "neverending" cycle of identifying buses and traffic lights which can last 5+ minutes. I must have identified more buses for Google than I've actually seen irl :(
Especially when you hop on a VPN; I kid you not I have to spent literally 20 straight minutes following this bs. It’s infuriating for someone who self-respects their privacy.
I set up a Shopify store from China (where a VPN is mandatory), and eventually had to tell Shopify to delete the site because I literally couldn't complete the captchas any more to access it. Couldn't even back up anything before bailing.
Before the complete lockout it was taking around 20 minutes of busses, bikes, sidewalks, and stoplights each login. Nasty feeling when you come back and you realize you have to log back in for some reason.
Before the complete lockout it was taking around 20 minutes of busses, bikes, sidewalks, and stoplights each login. Nasty feeling when you come back and you realize you have to log back in for some reason.
Isn’t this more of a function of the totalitarian regime that requires you to use a VPN, than CloudFlare?
Maybe try a personal VPN on digital ocean? You'd be guaranteed a clean IP that only you have access to, as it's very likely CF only flags VPN IPs because some malicious traffic is coming from them. The only issues with this are sites that block the digital ocean ASN (like Crunchyroll).
Yeah this works. If you're going to setup AlgoVPN might as well set it up on multiple locations (which I speculate is probably going to cost you n times more, given the n number of locations you choose).
AlgoVPN works great regardless of provider. It's more secure and private than public VPNs (not to mention fast).
This comparatively to stuff like ExpressVPN where I had to go through the aforementioned hurdles.
AlgoVPN works great regardless of provider. It's more secure and private than public VPNs (not to mention fast).
This comparatively to stuff like ExpressVPN where I had to go through the aforementioned hurdles.
How many other VPNs are operating out of Digital Ocean?
I know torguard, at least for their wireguard endpoints, operates out of Digital Ocean.
Sorry, i was too vague. There are a bunch of VPNs running in DO. If you don't want to look like a VPN, it's not a great place to be.
Actually, not sure on DO's "relations" with VPNs. I've heard they're pretty strict on what you do since they're a somewhat small company that focuses on "droplets"/instances.
Personally I'd reccomend AWS or GCP given the free credits anyways.
Personally I'd reccomend AWS or GCP given the free credits anyways.
Please people do the internet a favor and disable the captcha.
Go to Firewall > Settings > Security Level (captcha)
And set it to "Essentially Off"
https://support.cloudflare.com/hc/en-us/articles/200170096-H...
https://support.cloudflare.com/hc/en-us/articles/200170096-H...
As someone living in Hong Kong I hate cloudflare. Way too many websites behind cloudflare block hong kong or cause me to have a captcha. This causes me to often route my traffic to my own vpn to counter that which is really annoying.
At least I'm lucky enough that my VPN is for some reason still not detected as a cloud hosting provider by either cloudflare or netflix.
At least I'm lucky enough that my VPN is for some reason still not detected as a cloud hosting provider by either cloudflare or netflix.
What do you suggest to the people running websites where 90%+ of the hack attempts come from Hong Kong?
Do you have numbers that say that 90%+ hacks come from Hong Kong? I mean I run rather popular websites with a lot of traffic in the US, EU and Asia and I don't see that.
I do see that with traffic from China (but then blocking ASN from Aliyun and other cloud provider is enough to stop most of it) but I don't really see that from HK. I would say that blocking all traffic from a region instead of just blocking ASNs from cloud hosts is like using a hammer to kill a fly. It might work but you get collateral damage.
And regardless when I do block ASNs, I block uncached resources. What's the point of blocking static pages?
I do see that with traffic from China (but then blocking ASN from Aliyun and other cloud provider is enough to stop most of it) but I don't really see that from HK. I would say that blocking all traffic from a region instead of just blocking ASNs from cloud hosts is like using a hammer to kill a fly. It might work but you get collateral damage.
And regardless when I do block ASNs, I block uncached resources. What's the point of blocking static pages?
I’ve been blocked by CloudFlare and have nothing to do in HK, in fact I live in the US and decided to use a fork of Chromium and apparently used an “unacceptable” user agent (which was easily solved by changing it, duh). Screw CloudFlare. They are doing cheap, low quality L7 http traffic filtering.
Please stop spreading this false narrative: https://www.slashgeek.net/2016/06/07/cloudflare-making-inter...
Most network attacks and Spams actually comes from the United States.
Most network attacks and Spams actually comes from the United States.
The Spamhaus page mentioned in the article you linked: https://www.spamhaus.org/statistics/countries/ now shows China as the top malicious country. TFA was written in 2016. In the last 3 years China has overtaken the USA.
So it's not a false narrative. Most attacks and spams do come from China, not the US.
So it's not a false narrative. Most attacks and spams do come from China, not the US.
I think they were suggesting the websites they run, 90%+ of the attacks come from Hong Kong, i.e. not that 90% of all website attacks come from HK. No way of knowing unless they publish the logs/stats
I think most likely they are amalgamating Hong Kong and China like some people do on the thread here. And yes, indeed some Data Centers in China are a big source of spam and automated attack, but it just doesn't make sense to block an entire country when you can just block hosting providers.
If they’re failed attempts, why care about them at all?
If you have good security practices, you don’t have to worry about the script kiddies. If you have bad security practices, blocking Asia won’t help you.
If you have good security practices, you don’t have to worry about the script kiddies. If you have bad security practices, blocking Asia won’t help you.
> reCAPTCHA prompts happens a lot, I know its only one click away
This actually isn't true when browsing over Tor. It always makes you do 5 or more rounds of "click the traffic lights", "click the bicycles", etc. I don't know why. It seems to go far beyond checking that you're not a machine, and I suspect they're just abusing Tor users to get free machine-learning training.
This actually isn't true when browsing over Tor. It always makes you do 5 or more rounds of "click the traffic lights", "click the bicycles", etc. I don't know why. It seems to go far beyond checking that you're not a machine, and I suspect they're just abusing Tor users to get free machine-learning training.
Per CloudFlare themselves: "Based on data across the CloudFlare network, 94% of requests that we see across the Tor network are per se malicious."
https://arstechnica.com/tech-policy/2016/03/new-data-suggest...
https://blog.cloudflare.com/the-trouble-with-tor/
https://arstechnica.com/tech-policy/2016/03/new-data-suggest...
https://blog.cloudflare.com/the-trouble-with-tor/
A sensible CAPTCHA would be fine, but reCAPTCHA is borderline malicious towards Tor users.
Heh.
I just found out last week my hosting provider had put my websites on cloudflare for me.
They are not my hosting provider anymore.
How did I find out?
Traffic to the sites (all legitimate) fell circa 75% overnight. (non english sites)
They are not my hosting provider anymore.
How did I find out?
Traffic to the sites (all legitimate) fell circa 75% overnight. (non english sites)
> Traffic to the sites (all legitimate)
How do you know it was all legitimate?
I don't understand how cloudflare can stay in business if they cause a 75% drop in legitimate traffic to any website.
How do you know it was all legitimate?
I don't understand how cloudflare can stay in business if they cause a 75% drop in legitimate traffic to any website.
> I don't understand how cloudflare can stay in business[...]
VC backing and recent IPO have them swimming in cash, but they currently aren't profitable.
VC backing and recent IPO have them swimming in cash, but they currently aren't profitable.
Out of all the recent S-1s they have one of the strongest aside from Zoom.
[deleted]
This.
I route all my traffic via mmy own vpn server at Hetzner for privacy and security reasons and this Cloudflare bullshit is infuriating at times.
Besides I guess 95% sites that use their free tier either don't actually need it or would be better off without it.
I route all my traffic via mmy own vpn server at Hetzner for privacy and security reasons and this Cloudflare bullshit is infuriating at times.
Besides I guess 95% sites that use their free tier either don't actually need it or would be better off without it.
From a technical standpoint, what else can they do?
Your IP is coming from the cloud just like all the actual bot traffic.
Your IP is coming from the cloud just like all the actual bot traffic.
If you're in Europe and it happens on the website of a European company: make it a GDPR case. If you need to solve a captcha to access the privacy policy, they are clearly in violation.
Can you explain why this is a violation, please?
Privacy policies need to be immediately accessible to users. Hiding your privacy info page behind captchas, using unclear names for links ("service status" as a link to privacy info for example), making users click through multiple pages to find it etc makes you non-compliant.
Basically: you cannot hide the information, you cannot make users jump through hoops (captchas, require signup/login, pay for accessing) to read them.
Basically: you cannot hide the information, you cannot make users jump through hoops (captchas, require signup/login, pay for accessing) to read them.
Thanks, that's very useful.
Presumably the fact that it's not the site owner mandating the captcha, but an intermediary service provider doesn't matter then?
Presumably the fact that it's not the site owner mandating the captcha, but an intermediary service provider doesn't matter then?
It really shouldn't, because the site owner is the one making the choice to use CF; CF is acting on their behalf (and the security-settings the site owner chooses at CF does influence whether and how often captchas are shown to users, i.e. "I'm under attack" mode). It would be different if the user's ISP did this.
This is another related issue, too, as CF is a data processor, so the controller (=site owner) needs to make users aware that their data is being shared with CloudFlare, as SSL terminates at CF, the content is analyzed and it's then (optionally re-encrypted) transmitted to the origin.
This is another related issue, too, as CF is a data processor, so the controller (=site owner) needs to make users aware that their data is being shared with CloudFlare, as SSL terminates at CF, the content is analyzed and it's then (optionally re-encrypted) transmitted to the origin.
They have an extension that addresses this: https://support.cloudflare.com/hc/en-us/articles/11500199265...
As someone who works for a CDN (not CloudFlare), this was certainly interesting to read. It is always good to remember that we owe something to our customers' end users in addition to our direct customers, the website owners.
We have to remember that those are real people and not just percentages of traffic that are affected when we make decisions like putting a captcha up against every visitor from certain countries. I like to think we were already doing that, but it is good to be reminded.
However, I am not quite sure I am getting what the author is suggesting when they say that sites should forgo a CDN. Maybe I am biased, but if you thought latency was bad when a datacenter near you went down for maintenance, try having to go all the way to the sites origin in New Jersey for every request. I am not aware of any way besides a CDN (or a CDN like setup) that would get you good performance for people in all countries.
So I get the frustration with the captchas, and I get the frustration with the lack of multiple datacenters near you, but I wonder if you will make things worse for yourself by advocating to not have a CDN.
We have to remember that those are real people and not just percentages of traffic that are affected when we make decisions like putting a captcha up against every visitor from certain countries. I like to think we were already doing that, but it is good to be reminded.
However, I am not quite sure I am getting what the author is suggesting when they say that sites should forgo a CDN. Maybe I am biased, but if you thought latency was bad when a datacenter near you went down for maintenance, try having to go all the way to the sites origin in New Jersey for every request. I am not aware of any way besides a CDN (or a CDN like setup) that would get you good performance for people in all countries.
So I get the frustration with the captchas, and I get the frustration with the lack of multiple datacenters near you, but I wonder if you will make things worse for yourself by advocating to not have a CDN.
A working CDN certainly helps performance a lot, but it's not impossible to have acceptable performance worldwide from US only hosting. The three big things you need to do are:
A) serve your site quickly --- there's going to be a lot of not as avoidable latency between you and the user, but anything after your server gets the request is on you.
B) Keep your page weight small. Of course, this is a good idea anyway, but transfer rates on high latency connections are more often limited by tcp slow start than bandwidth.
C) Use TLS 1.3 or at least TLS 1.2 with ALPN (which triggers TLS false start in at least Chrome, and I think other browsers), as these reduce the number of round trips during connection setup compared to standard TLS 1.2 or earlier. It's worth measuring http/1.1 vs http/2 vs http/3 to see what works best for a particular site on high bandwidth networks.
Honorary mention: make sure path MTU blackhole discovery is working. There are still plenty of networks that have path MTU blackholes, and sending packets the size clients said they could get doesn't always work.
A) serve your site quickly --- there's going to be a lot of not as avoidable latency between you and the user, but anything after your server gets the request is on you.
B) Keep your page weight small. Of course, this is a good idea anyway, but transfer rates on high latency connections are more often limited by tcp slow start than bandwidth.
C) Use TLS 1.3 or at least TLS 1.2 with ALPN (which triggers TLS false start in at least Chrome, and I think other browsers), as these reduce the number of round trips during connection setup compared to standard TLS 1.2 or earlier. It's worth measuring http/1.1 vs http/2 vs http/3 to see what works best for a particular site on high bandwidth networks.
Honorary mention: make sure path MTU blackhole discovery is working. There are still plenty of networks that have path MTU blackholes, and sending packets the size clients said they could get doesn't always work.
If you have a proper website without a gazzilion of useless JavaScript frameworks and custom fonts, there's hardly going to be an unacceptable experience latency-wise even if you have to go all over the world to fetch those resources.
I guess I didn't realize, until another commenter replied, that their traffic was actually blackholed... I thought it just had really bad latency, which they were complaining about.
In the author’s case, the CDN was worse than fetching from the origin; for some reason, Cloudflare just gave up on rerouting their traffic (not sure how this is even possible given their anycast DNS) and they never got the requested resources at all! Having even a 30s RTT would have actually been an improvement vs. this outcome.
[2016].
Cloudflare's been working on improvements to the CAPTCHA system for Tor users (https://www.zdnet.com/article/cloudflare-ends-captcha-challe...), maybe some of those have benefited foreign countries as well. And CloudFlare does do country-level blacklists (which show up as the CAPTCHA behavior described) so maybe StackExchange had/has an overaggressive firewall.
Cloudflare's been working on improvements to the CAPTCHA system for Tor users (https://www.zdnet.com/article/cloudflare-ends-captcha-challe...), maybe some of those have benefited foreign countries as well. And CloudFlare does do country-level blacklists (which show up as the CAPTCHA behavior described) so maybe StackExchange had/has an overaggressive firewall.
I don't know what's so unique about my connection, but I, too, receive these capchas on a daily basis, on the very same sites.
I'd rather they track me, by IP address and/or cookies, and stop this non-sense. I accept the cookies and have a static and dedicated IP, but there's never an end to Cloudflare's captchas.
I'd rather they track me, by IP address and/or cookies, and stop this non-sense. I accept the cookies and have a static and dedicated IP, but there's never an end to Cloudflare's captchas.
Do you routinely use a VPN or a shared connection like University housing?
How do you define a VPN? Noone other than myself has ever used my IP addresses in X years.
Honestly I wonder if even an unusual setup like a static IP would register on one of their heuristics. It could flag as a web server, which would theoretically be more likely to ping a site rapidly than a personal computer.
> Honestly I wonder if even an unusual setup like a static IP
I have internet at home through Comcast, a huge and universally hated ISP in the USA. I also run a home server (though it only takes incoming connections, and does not visit websites itself). I purchased a dynamic DNS service that would update my domain's DNS whenever my home internet's public IP changed.
In over 5 years and through numerous modem reboots, my IP address has not changed once. A year ago I transferred my domain to another provider; I did not bother setting up dynamic DNS again and my website still works fine.
I have not purchased a static IP from Comcast. When I initially set up the server I had read that my home's IP address can change anytime the modem reboots, or possibly anytime at all, to any IP in Comcast's pool - which is why I subscribed to dynamic DNS.
So a static IP may not be as unusual of a setup as you say it is.
I have internet at home through Comcast, a huge and universally hated ISP in the USA. I also run a home server (though it only takes incoming connections, and does not visit websites itself). I purchased a dynamic DNS service that would update my domain's DNS whenever my home internet's public IP changed.
In over 5 years and through numerous modem reboots, my IP address has not changed once. A year ago I transferred my domain to another provider; I did not bother setting up dynamic DNS again and my website still works fine.
I have not purchased a static IP from Comcast. When I initially set up the server I had read that my home's IP address can change anytime the modem reboots, or possibly anytime at all, to any IP in Comcast's pool - which is why I subscribed to dynamic DNS.
So a static IP may not be as unusual of a setup as you say it is.
Why would a static IP be considered something unusual? I’ve almost always had one with all of the home broadband services that I’ve used over the years.
(Edit: I mean that the static IP was almost always included by default in the standard setup, I didn’t have to request or purchase one)
(Edit: I mean that the static IP was almost always included by default in the standard setup, I didn’t have to request or purchase one)
What is the alternative solution here?
How do you effectively protect access to websites while simultaneously making sure you never get false positives?
I really don't think going back to not having Cloudflare-like services is a step forward. Is there another way?
I really don't think going back to not having Cloudflare-like services is a step forward. Is there another way?
It's afaik only Cloudflare's way. You don't see that behaviour from Google or Facebook. They really now how to handle the Internet in the "second" and "third world".
Cloudflare is like: we don't care about you, you are too poor or a bot. Here is a extension if you are tech-savvy enough to find and install it.
From my perspective Google is far worse. I don't ever see Cloudflare's Captchas but I get Google's all the damn time, because I try to avoid Google properties. They intentionally punish people that aren't regularly using their products.
Is it possible to have a "Cloudflare-like" service without using Javascript? I have Javascript turned off by default (no script) and run into the Cloudflare wall all of the freaking time, so much so, I refuse to use it on my websites.
You can set the security level to “essentially off” and the vast vast majority of your users shouldn’t have to face the JS browser check. That’s what I do on all of my sites.
The default is medium, and the sites where you run into checks all the time are probably on high.
Ref: https://support.cloudflare.com/hc/en-us/articles/200170056-W...
The default is medium, and the sites where you run into checks all the time are probably on high.
Ref: https://support.cloudflare.com/hc/en-us/articles/200170056-W...
Happens to me all the time, and I just move on.
So far, nothing of value was lost.
So far, nothing of value was lost.
There is some presumption that Cloudflare's behavior related to "second class" regions is based on statistics - percentage of traffic that is of ill-intent. Maybe that's not the case, but I suspect so.
CF customers probably know that any delays cost viewers/visitors, but losing a few good visitors is worth preventing a ton of bad visitors. And CF generally seems very thoughtful about their actions. If they make something unpleasant, they're likely to have a good reason for doing so.
CF customers probably know that any delays cost viewers/visitors, but losing a few good visitors is worth preventing a ton of bad visitors. And CF generally seems very thoughtful about their actions. If they make something unpleasant, they're likely to have a good reason for doing so.
Most properties protected by Cloudflare appear to be webpages that could easily be cached without any ill effects, for hours to days or weeks.
Why exactly do they need captcha protection?
Why exactly do they need captcha protection?
That’s like asking a gun shop why americans need so many guns
Have you done an exhaustive survey of those sites to confirm that they all are completely static? If not, ask yourself how many site operators are going to prefer a global toggle to having to make sure they protect every exposed endpoint, bearing in mind that this has very little impact on a high majority of their customers unless they’re operating a service targeted at VPN users.
These especially suck in China. It’s especially painful when you are finally free and clear of the great firewall only to be stopped by a mistuned fraud filter.
Assuming the website itself isn't blocked by the GFW, many site owners regularly either set a Cloudflare rule to block or challenge (recaptcha) all china traffic (or even all non-US traffic for US-only businesses) due to the amount of spam and malicious requests that come from IPs originating in those countries. My own sites has shown some attacks from within the US but not nearly as much as from China before I set up a firewall rule to block China.
I live in the EU, but use a self-hosted VPN (for privacy reasons) on a German IP (digitalocean). And I too get the annoying reCaltcha prompts or even bans “your IP range has been blocked” all the time.
PS. To be fair it’s not just coming from Cloudflare-protected sites. Webmasters and SaaS-app devs add this to their WAF layers everywhere. :(
PS. To be fair it’s not just coming from Cloudflare-protected sites. Webmasters and SaaS-app devs add this to their WAF layers everywhere. :(
It's probably less about the German IP and more about DigitalOcean.
Most people live in homes, not datacenters, which means that web sites expect human traffic to come from residential IPs, not datacenter IPs. What comes from the datacenter IPs is an endless stream of costly abuse, so they get CAPTCHA'd (if they want to support use cases like yours) or blocked (if they don't care about those)
Most people live in homes, not datacenters, which means that web sites expect human traffic to come from residential IPs, not datacenter IPs. What comes from the datacenter IPs is an endless stream of costly abuse, so they get CAPTCHA'd (if they want to support use cases like yours) or blocked (if they don't care about those)
Huh, I never had problems with cloudflare I used something like tor.
But I had TONS of New Zealand websites captcha me without a good reason. They all used some shitty local providers.
But I had TONS of New Zealand websites captcha me without a good reason. They all used some shitty local providers.
Where were you connecting from? From within NZ, unsurprisingly, I've never hit a CAPTCHA barrier on a NZ site.
I totally get it. For about 2 years I was subject to captchas for visiting websites of restaurants. No one else felt what I did because I don't have (never have and will never have) a Facebook account. What that meant for phone users trying to see the menu of 25% of any restaurant - a Facebook captcha. I think the only reason it went away is because I complained about it in hn. I think I got lucky.
As a CTO of a startup trying to build a product on the web, though, CF is awesome. It solves a whole bunch of problems for free, and I don't have to worry about stuff that I would otherwise have to worry about (and I don't have time to worry about those things on top of all the other things I have to worry about).
In a few years, when I have a team of engineers and can spare the resources/expertise, we'll come off CF and do it properly. Until then, CF is a great service.
Sorry that your experience from SEA is not great, but tbh we're not selling in SEA, so any traffic from there is just a resource drain on our servers. Anything to discourage traffic from areas we're not serving is a positive for us.
Anything to discourage bot traffic is a huge positive. CF won't stop the bots, but costing them some minor amount of money per visit is still positive.
In a few years, when I have a team of engineers and can spare the resources/expertise, we'll come off CF and do it properly. Until then, CF is a great service.
Sorry that your experience from SEA is not great, but tbh we're not selling in SEA, so any traffic from there is just a resource drain on our servers. Anything to discourage traffic from areas we're not serving is a positive for us.
Anything to discourage bot traffic is a huge positive. CF won't stop the bots, but costing them some minor amount of money per visit is still positive.
I live in a developing country and I did experience an odd surge in recaptcha triggering for sites behind cloudflare around the end of 2016, but it really hasn't happened since.
It's barely anything at all, but my personal website is behind cloudflare and I've never had any trouble.
It's barely anything at all, but my personal website is behind cloudflare and I've never had any trouble.
Networking in Asia seems to be really complicated due to the various political interests in play. I remember our Cloudflare rep explaining that China had two or three telcos that refused to exchange traffic directly so you had to send everything through Japan and back to the telco you wanted in China.
I’ve always wondered how all the expats in Bali accomplish anything. Everybody says they can work remotely via the internet and run their businesses, but when I was there the internet would go down for the entire island frequently and could be down for minutes or hours.
I’ve always wondered how all the expats in Bali accomplish anything. Everybody says they can work remotely via the internet and run their businesses, but when I was there the internet would go down for the entire island frequently and could be down for minutes or hours.
SSL is terminated on CF and website owners are allowing CF to generate SSL Cert and Key for them in free plan. It is a wildcard certificate.
CF has the ability to read/alter the information we are sending to (or receiving from) the actual website. CF also has the ability (I do not mean they do) to impersonate the original website without the owner’s knowledge, and with visitor’s trust.
CF has the ability to read/alter the information we are sending to (or receiving from) the actual website. CF also has the ability (I do not mean they do) to impersonate the original website without the owner’s knowledge, and with visitor’s trust.
In regards to the accidental censorship of content for users behind VPNs and proxies, on my site I have set the security level to essentially off and disabled any and all integrity checking of visitors. I haven't tested the effect on those users but it should allow anyone to retrieve the pages unmolested.
The upside is that the ‘second-class’ markets will be served by local services. And when western markets are completely saturated, western site owners will be scratching their heads wondering how they get any part of the ‘second-class’ markets.
Everything is collected.
From the Boot.
Bomb Utah.
Hmmm...well unfortunately a lot of scammers and thieves on the internet come out of third world countries. Like, it's crazy, but the economics of scam artistry make this worth while as a career in certain countries - for example Romania (https://abcnews.go.com/International/journey-hackerville-rom...).
So, websites are protecting themselves as best they can while pissing off the fewest of their customers by putting up more security measures blocked by country IP.
The real issue is the authoritarian governments that are making it so countries (like China) are completely fire-walled off from the internet.
Have you heard that Iraq is having massive street protests at the moment? If not, it might be because the government cut off internet access so a lot of the stories and media about the protests are not making it onto the social media sites. That's scary.
So, websites are protecting themselves as best they can while pissing off the fewest of their customers by putting up more security measures blocked by country IP.
The real issue is the authoritarian governments that are making it so countries (like China) are completely fire-walled off from the internet.
Have you heard that Iraq is having massive street protests at the moment? If not, it might be because the government cut off internet access so a lot of the stories and media about the protests are not making it onto the social media sites. That's scary.
How is it Google.com doesn’t require captchas to open the home page? Nor does apple.com or any number of massive-traffic sites. The idea that captcha is the best, most effective, or most secure way to protect against DoS is just lazy.
To open the home page? True... But to view search results? I get captcha demands from google all the time depending on the search query (sometimes a query I make looks like something a bot might supply, like searching for a few IP addresses in a row) or if I'm using a VPN or Tor.
I get captchas on google.com all the time, especially if I make several complicated queries with operators in a short time span. And as this other guy says you are guaranteed to get a captcha from Tor
OP lives in Southeast Asia and suggests website owners choose a different CDN and fails to propose an alternative. ¯\_(ツ)_/¯
There are two complaints,
(1) cloudflare requires a captcha for visitors from some regions (like SouthEast Asia)
(2) cloudflare does not have enough nodes in SouthEast Asia, and OP feels being rerouted to another node defeats the purpose of a CDN.
Yet, Cloudflare does (1) because they often see attacks from those regions. I'm not sure blaming Cloudflare for this is the right strategy. Regarding (2), CDNs do not just benefit users, they benefit the website too. Getting rid of the CDN is not a solution. Is there a better free CDN for that region? Is multi-CDN easy to setup?
There are two complaints,
(1) cloudflare requires a captcha for visitors from some regions (like SouthEast Asia)
(2) cloudflare does not have enough nodes in SouthEast Asia, and OP feels being rerouted to another node defeats the purpose of a CDN.
Yet, Cloudflare does (1) because they often see attacks from those regions. I'm not sure blaming Cloudflare for this is the right strategy. Regarding (2), CDNs do not just benefit users, they benefit the website too. Getting rid of the CDN is not a solution. Is there a better free CDN for that region? Is multi-CDN easy to setup?
> Getting rid of the CDN is not a solution.
Why not? The whole everyone-needs-Cloudflare is a made up problem, which depends on many false narratives.
And why do we as website visitors have never heard of Akamai, yet it's hard to find anyone who's never seen these captchas from Cloudflare and Incapsula?
> Is there a better free CDN for that region?
You can only find free mice in mousetraps.
Why not? The whole everyone-needs-Cloudflare is a made up problem, which depends on many false narratives.
And why do we as website visitors have never heard of Akamai, yet it's hard to find anyone who's never seen these captchas from Cloudflare and Incapsula?
> Is there a better free CDN for that region?
You can only find free mice in mousetraps.
> Why not? The whole everyone-needs-Cloudflare is a made up problem, which depends on many false narratives.
I've never seen anyone say "everyone needs cloudflare" except maybe CF itself.
> why do we as website visitors have never heard of Akamai
Akamai is not free, they have a trial period that is free. It's not in the same space.
> You can only find free mice in mousetraps.
I don't understand your point. You feel CF is a trap?
I've never seen anyone say "everyone needs cloudflare" except maybe CF itself.
> why do we as website visitors have never heard of Akamai
Akamai is not free, they have a trial period that is free. It's not in the same space.
> You can only find free mice in mousetraps.
I don't understand your point. You feel CF is a trap?
> I don't understand your point. You feel CF is a trap?
Of course it is. If you aren't paying for service, you're not the customer, you're the product.
The whole mandatory TLS campaign is part of the lock-in, too.
Of course it is. If you aren't paying for service, you're not the customer, you're the product.
The whole mandatory TLS campaign is part of the lock-in, too.
Not to mention, CloudFlare has become synonymous with hosting and protecting the worst websites on the internet, with currently only two exceptions made, seemingly at the CEO's whim.
CloudFlare protects websites dedicated to doxing, stalking and harassing their victims. And when their victims complain, CloudFlare forwards all their personal information directly to said website owners while doing nothing about it. Websites that host content like the Christchurch massacre video and manifesto. Websites that have bullied people to the point of suicide.
I'm all for the importance of freedom of speech and being able to say offensive things on the internet, but CloudFlare is protecting sites that flagrantly violate the law.
CloudFlare protects websites dedicated to doxing, stalking and harassing their victims. And when their victims complain, CloudFlare forwards all their personal information directly to said website owners while doing nothing about it. Websites that host content like the Christchurch massacre video and manifesto. Websites that have bullied people to the point of suicide.
I'm all for the importance of freedom of speech and being able to say offensive things on the internet, but CloudFlare is protecting sites that flagrantly violate the law.
The CEO's blog posts make it clear that CF really wants to bee the city water company that sells access to "dumb pipes": pipes that can route, filter and transform the stuff that goes through them while not knowing about exactly what it is that is going through them (at least when some maliciousness filters haven't triggered); the city water doesn't want to start cutting people's water off because of what they use the water for. Their stance is that they don't have the power to remove that content, all they can do is make the website vulnerable to ddos attacks and whatnot by kicking them off the service.
While 8chan being kicked off CF did bring them offline (their new anti-DDOS provider was told to kick 8chan off by the upstream bandwidth provider, and the domain hasn't worked since to my knowledge) the daily stormer is still working. CF kicking TDS off their network didn't stop their website from working.
While 8chan being kicked off CF did bring them offline (their new anti-DDOS provider was told to kick 8chan off by the upstream bandwidth provider, and the domain hasn't worked since to my knowledge) the daily stormer is still working. CF kicking TDS off their network didn't stop their website from working.
I understand that, and I don't think websites should be deplatformed just for saying unpopular things.
But when a site clearly crosses into blatant criminality, it's disappointing when everyone who has the power to rein them in (Google with PageRank, CloudFlare with protection, the US government with criminal proceedings) decides to pass the buck on to someone else. It's always someone else's problem, meanwhile people's lives are being ruined and lost.
These sites would not be as highly profitable without CloudFlare's network, so I think it's fair that if someone wants to use CloudFlare, they're aware of what they're supporting when they give CloudFlare their money.
But when a site clearly crosses into blatant criminality, it's disappointing when everyone who has the power to rein them in (Google with PageRank, CloudFlare with protection, the US government with criminal proceedings) decides to pass the buck on to someone else. It's always someone else's problem, meanwhile people's lives are being ruined and lost.
These sites would not be as highly profitable without CloudFlare's network, so I think it's fair that if someone wants to use CloudFlare, they're aware of what they're supporting when they give CloudFlare their money.
If it's "blatant criminality", sue them in court. Deplatforming is literally passing the buck to some other platform.
[deleted]
There are many good reasons to hate Cloudflare.
IMHO, them failing to be the prosecutor, judge, jury and executioner for deplatforming inconvenient content aren't quite one of them.
IMHO, them failing to be the prosecutor, judge, jury and executioner for deplatforming inconvenient content aren't quite one of them.
This wasn't true in 2016 and isn't true in 2019 either.
Whilst the system was named and blogged about in Sep 2017 https://blog.cloudflare.com/meet-gatebot-a-bot-that-allows-u... the internal repo has the first working code drop in Feb 2015.
The (D)DoS protection systems and team, which I am the Engineering Manager for, treat all attacks equally according to the nature of the attack rather than the target of the attack.
To address other parts of the blog though, there is nothing in our system that defaults to "do X for IP/Country y"... what there is, is learned state about traffic, clients, and the many layers of customer expressed configuration.