Requests dropped when using Cloudflare’s free tier for a commercial project(pawelurbanek.com)
pawelurbanek.com
Requests dropped when using Cloudflare’s free tier for a commercial project
https://pawelurbanek.com/cloudflare-free-plan
127 comments
I am talking internally with the customer support team about what happened to this customer's traffic. It is definitively not the case that doing "four requests per minute" gets you rate limited.
My application[1] with similar rpm as OP received 503 status last month for about an hour first time in 1.5 years, I assumed it to be an one-off issue at CF end. It happened while I was using it and so I'm now concerned if it's a regular affair for my users, most of whom are not from my geographical location(data-center).
I'm a free plan user because my application is not monetized and obviously would switch to paid plan once it starts generating money. CF free plan has been indispensable for many such free projects.
UptimeRobot(Free) doesn't show anything fishy, except for CF error 521 occasionally. Again UptimeRobot requires paid version to check for HTTP status message, guess I'll have to do it or setup a self hosted(recommendations?) solution to monitor HTTP status of CF Free application.
If CF does perform rate limiting for such low rpm, I would prefer to be intimated as even third rate shared hosting providers do it and I hope CF wouldn't put their reputation on the line for something like this.
[1]https://needgap.com
I'm a free plan user because my application is not monetized and obviously would switch to paid plan once it starts generating money. CF free plan has been indispensable for many such free projects.
UptimeRobot(Free) doesn't show anything fishy, except for CF error 521 occasionally. Again UptimeRobot requires paid version to check for HTTP status message, guess I'll have to do it or setup a self hosted(recommendations?) solution to monitor HTTP status of CF Free application.
If CF does perform rate limiting for such low rpm, I would prefer to be intimated as even third rate shared hosting providers do it and I hope CF wouldn't put their reputation on the line for something like this.
[1]https://needgap.com
Talk to support.
Why don't you send an email to the customer if there is an error while serving content by cloudflare? It can be a daily digest of the errors of the day.
If the problem is on cloudflare's side then the cloudflare user doesn't even know about it if it's an intermittent error.
If the problem is on cloudflare's side then the cloudflare user doesn't even know about it if it's an intermittent error.
While we are on the subject of rate limiting and such, does Cloudflare allow using its service for serving video files for a media heavy social media app where we store the videos on Backblaze B2?
I’m working on such an app, and am hoping to use Cloudflare + Backblaze B2, because of the bandwidth alliance. https://www.cloudflare.com/en-gb/bandwidth-alliance/
However, the Terms of Service say:
> 2.8 Limitation on Serving Non-HTML Content
> The Services are offered primarily as a platform to cache and serve web pages and websites. Unless explicitly included as part of a Paid Service purchased by you, you agree to use the Services solely for the purpose of (i) serving web pages as viewed through a web browser or other functionally equivalent applications, including rendering Hypertext Markup Language (HTML) or other functional equivalents, and (ii) serving web APIs subject to the restrictions set forth in this Section 2.8. Use of the Services for serving video or a disproportionate percentage of pictures, audio files, or other non-HTML content is prohibited, unless purchased separately as part of a Paid Service or expressly allowed under our Supplemental Terms for a specific Service. If we determine you have breached this Section 2.8, we may immediately suspend or restrict your use of the Services, or limit End User access to certain of your resources through the Services.
https://www.cloudflare.com/en-gb/terms/
And I am wondering what kind of subscription would be appropriate for my use-case if any, that would allow me to host the files on Backblaze B2, and serve these files through Cloudflare.
(Cloudflare would also be hosting our API, so it’s not just video files that we want to serve via Cloudflare.)
I’m working on such an app, and am hoping to use Cloudflare + Backblaze B2, because of the bandwidth alliance. https://www.cloudflare.com/en-gb/bandwidth-alliance/
However, the Terms of Service say:
> 2.8 Limitation on Serving Non-HTML Content
> The Services are offered primarily as a platform to cache and serve web pages and websites. Unless explicitly included as part of a Paid Service purchased by you, you agree to use the Services solely for the purpose of (i) serving web pages as viewed through a web browser or other functionally equivalent applications, including rendering Hypertext Markup Language (HTML) or other functional equivalents, and (ii) serving web APIs subject to the restrictions set forth in this Section 2.8. Use of the Services for serving video or a disproportionate percentage of pictures, audio files, or other non-HTML content is prohibited, unless purchased separately as part of a Paid Service or expressly allowed under our Supplemental Terms for a specific Service. If we determine you have breached this Section 2.8, we may immediately suspend or restrict your use of the Services, or limit End User access to certain of your resources through the Services.
https://www.cloudflare.com/en-gb/terms/
And I am wondering what kind of subscription would be appropriate for my use-case if any, that would allow me to host the files on Backblaze B2, and serve these files through Cloudflare.
(Cloudflare would also be hosting our API, so it’s not just video files that we want to serve via Cloudflare.)
They do NOT allow video or large files (+50MB) on the Free plan or even on the paid plans!
You'll have to get an Enterprise account for that (it's no longer starting at $5K/mo, you can get custom accounts setup to match your needs).
You'll have to get an Enterprise account for that (it's no longer starting at $5K/mo, you can get custom accounts setup to match your needs).
> it's no longer starting at $5K/mo, you can get custom accounts setup to match your needs
Meaning that it might be possible to get what we need at a suitable price? $5K/mo is way over our budget until we have actual users and revenue.
If so then that is great. Thanks for the tip :)
Meaning that it might be possible to get what we need at a suitable price? $5K/mo is way over our budget until we have actual users and revenue.
If so then that is great. Thanks for the tip :)
They do have a separate offering for video streaming which is pay as you go.
https://www.cloudflare.com/products/cloudflare-stream/
https://www.cloudflare.com/products/cloudflare-stream/
It's not compatible with Cloudflare Workers afaik.
For what is is worth, I've been using their Free Tier to serve over 10k requests per hour with ~60% html / ~40% media (mainly pictures, very few videos) and I've never had a problem.
However, for a CDN for video content YMMV. Perhaps you should try contacting them directly to have a written estimate.
However, for a CDN for video content YMMV. Perhaps you should try contacting them directly to have a written estimate.
I basically have something similar to your use-case. While I'm on the free tier for all my domains, the supplemental agreement, at least in the US version, removes the specific limitation to serve primarily html/text content if you enable Cloudflare Workers, which also has a free tier, applies on an account and not domain level, and has no requirement for you to serve your content through their KV system. I ended up paying for my CF Workers subscription for a different project on my account but the system worked fine prior to that. I served 111GB of a combination of m4a and png files in data from my B2 bucket via CF in the past month.
Video is really not allowed in the TOS for Cloudflare Workers either, see this discussion about the topic:
https://community.cloudflare.com/t/cloudflare-workers-live-v...
https://community.cloudflare.com/t/cloudflare-workers-live-v...
Serving videos (cached or not) is only allowed by using Cloudflare Stream or as a part of Enterprise contract.
Is it possible that this person's free-plan traffic was routed through a POP that was under attack? Then, upon upgrading to Pro the routing rules for the account were modified to avoid under-attack data centers.
I think it might also be worth considering whether returning 503 is the right response when rate limited. 5xx indicates server failure, a "429 Rate Limited" is a more appropriate response.
4xx is a client error and 429 indicates that particular client is making too many requests. 5xx is a server error and 503 says the service itself is unavailable irrespective of the client’s actions. In this particular case it’d be unavailable for contractual rather than technical reasons, but it’s still the right status
If it is indeed an issue of rate limiting.
Regardless of the reason being the limit or not - are these limits published anywhere? It doesn’t appear to be the case.
A great first step would be to take responsibility for what the customer has proven to happen. "It is definitively not the case that doing" is directly at odds with what the author provides, you might want to reframe your message to "it shouldn't be like this but unfortunately it is like this right now, sorry" rather than dismissing the author with that they are not correct.
You shouldn't assume that he is correct either; i'm a free plan user and have never experienced this
If you're on free plan you have no way to know if this has happened to your website.
But, I thought in your case you did know something was wrong: client complained and you saw 503 errors?
> no way
you can (and should) put monitors on the outside of your infrastructure as well as inside. I've multiples hitting the login page and robots file where I work, and never got an unaccounted 503
you can (and should) put monitors on the outside of your infrastructure as well as inside. I've multiples hitting the login page and robots file where I work, and never got an unaccounted 503
Like I wrote, for some integrations it is not possible to gather all the logs. Also how will you know that a client accessing your website in a browser gets 503 instead of your web page?
You don't, but your monitors will show the 503 happening, how often, on which endpoint operation and in which regions; that will give you a pretty good picture of whether is actually your CDN layer or something else triggering the 503
What if it's a specific browser with a specific TLS stack, or something weird?
what if it's green lizard man chewing cables?
one has to start somewhere reasonable and realistic first.
one has to start somewhere reasonable and realistic first.
Proven is a strong word and rate limiting is not the only possible explanation. Something is happening and we don't know what.
[deleted]
There is a connection limit???
I’d look into idle connection handling, I’ve had problems in the past with cloudflare and it’s long open sockets to origin being quietly closed at origin, breaking requests to cloudflare.
Please keep us posted. From a naive perspective, what happened has a flavor of "protection fees." A detailed explanation would help quickly dispel this perspective.
It's going to be hard for me to keep you posted because it would mean discussing a customer's account and I can't do that. That's confidential.
Hopefully OP will come back and update us
OP's update here : https://news.ycombinator.com/item?id=26072153
TLDR; He's retracted his claims
TLDR; He's retracted his claims
As someone who has run a wide range of sites on Cloudflare for many years, I am certain this is bupkis.
1. Cloudflare doesn’t send 503 when they’re deliberately rate limiting; they send it when there’s an unexpected problem. It’s fairly rare, and it usually results in something being posted on cloudflarestatus.com if it persists.
2. Sometimes error rates are low enough or the conditions are obscure enough that you may have to contact support to point out that it’s happening. This is even rarer.
3. When you switch between plans, the IP addresses assigned to you may change, and users may hit different edges. If the issue is regional (as is usually the case), this may resolve it.
4. I have been sending tens of requests per second to a free plan over the course of several years and have never measured an increase in 503 responses on Cloudflare’s end compared to a paid plan.
5. Cloudflare’s response mentions specific 503 errors that are fairly rare; they’re notable because they look like a standard Nginx 503 page, rather than a nice Cloudflare error page. The only difference is that they will say “cloudflare” at the bottom where the Nginx version would normally be. You probably frequent sites that use Cloudflare free plans; how often do you see this error message?
6. What the hell is this n=1 correlation? It reads like a conspiracy theory. It is a conspiracy theory.
There are plenty of reasons to criticize Cloudflare, but this isn’t one of them.
1. Cloudflare doesn’t send 503 when they’re deliberately rate limiting; they send it when there’s an unexpected problem. It’s fairly rare, and it usually results in something being posted on cloudflarestatus.com if it persists.
2. Sometimes error rates are low enough or the conditions are obscure enough that you may have to contact support to point out that it’s happening. This is even rarer.
3. When you switch between plans, the IP addresses assigned to you may change, and users may hit different edges. If the issue is regional (as is usually the case), this may resolve it.
4. I have been sending tens of requests per second to a free plan over the course of several years and have never measured an increase in 503 responses on Cloudflare’s end compared to a paid plan.
5. Cloudflare’s response mentions specific 503 errors that are fairly rare; they’re notable because they look like a standard Nginx 503 page, rather than a nice Cloudflare error page. The only difference is that they will say “cloudflare” at the bottom where the Nginx version would normally be. You probably frequent sites that use Cloudflare free plans; how often do you see this error message?
6. What the hell is this n=1 correlation? It reads like a conspiracy theory. It is a conspiracy theory.
There are plenty of reasons to criticize Cloudflare, but this isn’t one of them.
While this blog post has very little in the way of proof, we actually got screwed over by CF a few weeks back. Basically, one morning all requests started being slow: small responses would take 7 seconds (within a few milliseconds, if you accounted for the RTT to the edge) with 7 seconds TTFB, larger ones (not sure exactly what the threshold was, but it was hundreds of KiB, not MiB) would take 1 minute (again, almost exactly 1 minute), but these would just stream very slowly. All while having cf-cache-status: HIT.
It turned out we got throttled because we were serving some (a minority, something like 10% of the traffic) video files from the domain and they wanted us to upgrade to enterprise (from business, I think). We just stopped serving video through them.
The annoying part was that we weren't notified and, obviously, had some downtime (no way you can call a site loading in 5 minutes up). They said they have released an update that does notify customers when this happens, though.
It turned out we got throttled because we were serving some (a minority, something like 10% of the traffic) video files from the domain and they wanted us to upgrade to enterprise (from business, I think). We just stopped serving video through them.
The annoying part was that we weren't notified and, obviously, had some downtime (no way you can call a site loading in 5 minutes up). They said they have released an update that does notify customers when this happens, though.
Hmm, we've been experiencing vastly reduced throughput as well (something like 100 kB/s on images >100 kB in size, cache hit). We're on the free plan, though. Makes me wonder if there's actually some throttling going on behind the scenes.
Might this have something to do with people using Cloudflare to share child pornography a couple of years ago? They've been talking about it again recently.
I can only comment on our exact situation, where the response times were as I mentioned them, exactly. I've not experienced that, but I guess it's worth shooting a question to support (though, expect several days on the free plan)
Today there was a topic on HN about customers who complain about limiting free tiers. That’s a great example of it, especially from someone who provides paid SaaS product.
Free tier is awesome for small websites, keeping domains and many other things. But if you run business who earns, then just pay for that god damn thing, and stop ranting. :-)
Free tier is awesome for small websites, keeping domains and many other things. But if you run business who earns, then just pay for that god damn thing, and stop ranting. :-)
Someone came on HN to report that a free-tier product is failing in a way that’s undetectable and doesn’t make sense as described by the products’ marketing materials, and the resolution is to pay (a small amount) for the upgraded version. One of the Cloudflare engineers* immediately responds to the article saying he’s looking into things because this shouldn’t be happening. Everything seemed to be going so well in this story. I only continued reading the comments to find the first one where someone turns this into a story of blame for the poster being cheap, and your post was it.
ETA: * The CTO of Cloudflare
ETA: * The CTO of Cloudflare
It’s not the case with everyone, OP could wait until supports get back to him and investigates everything, and then push a pice on that. Bug -> Contact Support -> Wait for answer & solutions -> Happy or not ending.
What if he accuses them without justification? Which is probably a case here.
What if he accuses them without justification? Which is probably a case here.
> What if he accuses them without justification? Which
He's already doing that. Even if it turns out that he happens to be right, it's coincidence.
He's already doing that. Even if it turns out that he happens to be right, it's coincidence.
This is a reasonable objection to TFA, but it is an entirely different objection than your original post I responded to.
They did get back to him. What evidence do you have that they were still investigating anything? They answered his questions and the answers were no good.
He didn't say cheap though. You had to rephrase what he said so you had something to attack.
I think Cloudflare is fine in most of the cases. Putting title "Never use Cloudflare free plan" with so weak arguments in the blog post is too much.
There's been a spate of "never" or "always" posts getting on the front page in recent days.
The graphs containing 503 errors to two different sites which were both resolved by paying seem to indicate his case is strong.
If that's our standard of evidence then I've resolved bugs by getting a coffee and doing nothing else. :)
Bug resolved. Reason: not enough coffee.
Seriously these systems are complicated and it's easy to confuse correlation for causality.
Bug resolved. Reason: not enough coffee.
Seriously these systems are complicated and it's easy to confuse correlation for causality.
Could this be explained by something like “there was some bug causing this person’s config to be in a weird state (on the cloudflare side), and changing the payment tier merely gave the system a kick that got it out of the bad state? After all, the person payed $20pm not because they thought it would give a better reliability but because they wanted better support for a weird issue. I’m not really very appalled by the idea that free users don’t get great support.
OP here. This article generated some attention and traffic. The claims contained in it are incorrect. After another contact with CF support, it turned out that the Bot Fight Mode behaves differently for Free and PRO plans. That's what caused the instant improvement after upgrading. Another way to resolve the issues I was experiencing would have been to disable the bot fight mode altogether or add a custom page rule disabling it. My website never experienced any traffic throttling. I've decided to remove the article, to stop spreading the misinformation about CF services.
Chatting with the folks internally I'll make sure that our documentation and support are clearer here. This should have been sorted out the moment you contacted support (or even before with clearer documentation). Sorry you had so much trouble.
I'm starting to lose faith in cloudflare lately - we're on the business plan and there's been some really strange UI bugs in the dashboard where it looks like state just isn't reflected properly (Access and Load Balancing being ones we see quite a bit)
Summary of things that we've had issues with lately:
- Caching of a specific route just stopped working. I log a support ticket (nightmare to find in the dashboard) and miraculously it starts working again after the ticket gets responsded to.
- Some of our staff got locked out of part of our app because we exceeded the 5 free users of the "Access" plan. Upgraded to the 50 user plan in the new Teams part and it still didn't work. Contacted support, fixed again but no explanation. (multiple day turnaround on tickets)
We're invested in their tech a lot and I love workers - they really take some big tasks off us. If Azure come up with something compelling we'll probably switch though (assuming I can make sense of Azure's billing and product naming strategy)
Summary of things that we've had issues with lately:
- Caching of a specific route just stopped working. I log a support ticket (nightmare to find in the dashboard) and miraculously it starts working again after the ticket gets responsded to.
- Some of our staff got locked out of part of our app because we exceeded the 5 free users of the "Access" plan. Upgraded to the 50 user plan in the new Teams part and it still didn't work. Contacted support, fixed again but no explanation. (multiple day turnaround on tickets)
We're invested in their tech a lot and I love workers - they really take some big tasks off us. If Azure come up with something compelling we'll probably switch though (assuming I can make sense of Azure's billing and product naming strategy)
Cloudflare might have other issues from time to time, but I've never experienced the similar problem.
I'm running around 40 domains on Cloudflare Free plan. About 20 of these domains have some traffic.
I'm running around 40 domains on Cloudflare Free plan. About 20 of these domains have some traffic.
>but I've never experienced the similar problem.
How do you know for sure that you didn't? One of the main takeaways from the original article is that these errors are invisible from server side (your own stack) because it's the traffic is being throttled before it gets to your own servers. Unless you have very good and consistent client side availability metrics you probably wouldn't notice the 2%-3% drop.
How do you know for sure that you didn't? One of the main takeaways from the original article is that these errors are invisible from server side (your own stack) because it's the traffic is being throttled before it gets to your own servers. Unless you have very good and consistent client side availability metrics you probably wouldn't notice the 2%-3% drop.
There is a pretty big collection of Hungarian historical photos online, a bit more than a terabyte, well above 100K of them and the photos are on B2 with Cloudflare fronting it. There's many terabytes of traffic each month and it's on the free tier. Since the hosting costs are covered by yours truly, I can assure you the project couldn't possibly afford cloud storage traffic prices. We would need to store it on a VPS and all the problems that come with it -- disk size, availability, reliability etc. I am so grateful for B2-CF to do this.
Special kudos for allowing the -- very cheap, only $5 -- worker addon without forcing a paid plan.
Special kudos for allowing the -- very cheap, only $5 -- worker addon without forcing a paid plan.
I just upgraded back to Pro, i will post the results in 1-2 days time. I used to use pro but there's really no benefit or difference.
Anyway i did a quick check, my 503 error is currently at 3.23k the past 7 days , that's out of 1 million request the past 7 days.
I was about to upgrade to ARGO and pay for it since i was optimising the bandwidth the past month until I saw this article which really gave me a shock.... because I heavily invested in Cloudflare stock market...... Which so far has good returns and i still believe in it but this article is critical ... If what this person say is true, I might exit Cloudflare earlier, from the stock market i mean.
Anyway i did a quick check, my 503 error is currently at 3.23k the past 7 days , that's out of 1 million request the past 7 days.
I was about to upgrade to ARGO and pay for it since i was optimising the bandwidth the past month until I saw this article which really gave me a shock.... because I heavily invested in Cloudflare stock market...... Which so far has good returns and i still believe in it but this article is critical ... If what this person say is true, I might exit Cloudflare earlier, from the stock market i mean.
[deleted]
Sounds weird. I'm not saying to trust Cloudflare blindly, but I used a social media site (very "reddit-like") my friend made and kept for over a year on free plan with no issues. Around 20-80 people online were actively creating content, private messages, browsing, upvoting, with live notifications with websockets and so on, it was a quite busy little site for a year or so). There never was issues with couldflare and it basically allowed the site to run on two smallest VPS machines mentioned friend could find.
Also why is author giving trust in a free service for his paid service? CF is not that expensive.
Also why is author giving trust in a free service for his paid service? CF is not that expensive.
This article seems fairly scant on details to be honest. I understand how they came to the conclusion, but there's still quite a lot of conjecture here.
That's the point. Because of Cloudflare's opacity, they can't get any more information. Conjecture is all they have.
I feel like the main problem is the inability to properly debug your problem without paying while the plan essentially necessitates a degree of self-diagnostic when things go wrong. I also serve an xml feed via CF and also use it as a caching service in front of my Backblaze B2 bucket, and on the free tier by virtue of the files I'm serving far fewer unique users (less than half of the author's last 30 days) but a far greater amount of data (111GB). I'm not certain if the author should jump from what's clearly a vague and uncommitted conjecture from the Cloudflare tech about where the 503 error comes from straight to "they are definitely throttling me", since I've had the opposite experience and have seen 503s in all sorts of situations unrelated to throttling, but with a tiny sample size and no adequate diagnostic tooling it's also impossible for me or anyone to say that he's not being throttled, except CF themselves. I realize that it's their business model but it seems counter-intuitive that the service they sell seems partly "better ability for you to fix your problems yourself", sort of the opposite of say, managed/unmanaged servers. It'd make more sense, from a customer's point of view, that if I'm paying for a service, there should be service in both senses of the word - the product and the servicing of issues if possible, but not making diagnostics more transparent makes little sense because it prevents people who want to use the service from continuing to use the service, presumably part of CF's goal.
My cost benefit analysis is that the Cloudflare free plan is a bargain. At any time I can instantly turn on the protection I might need because I have already integrated Cloudflare into my infrastructure.
I am careful about what technologies I add to my stack because I am a sole developer, but Digital Ocean and Cloudflare have been big wins for me.
I am careful about what technologies I add to my stack because I am a sole developer, but Digital Ocean and Cloudflare have been big wins for me.
Unpopular opinion - there's a lot of entitled people here and jumping to conspiracy.
There is no such thing as a free lunch [0] for people to depend a commercial service on. If your business model depends on another company providing you a free service perhaps you should reconsider.
This seems to be a person being cheap and jumping to conclusions, claiming broad assumptions and conjecture as fact.
Yes, cloudflare's free tier is deliberately dropping your requests to foil your freeloading commercial company - raise pitchforks!
Assuming this is all true (which I don't), I don't feel sympathy for the author for not purchasing a paid plan.
[0] https://en.wikipedia.org/wiki/There_ain%27t_no_such_thing_as...
There is no such thing as a free lunch [0] for people to depend a commercial service on. If your business model depends on another company providing you a free service perhaps you should reconsider.
This seems to be a person being cheap and jumping to conclusions, claiming broad assumptions and conjecture as fact.
Yes, cloudflare's free tier is deliberately dropping your requests to foil your freeloading commercial company - raise pitchforks!
Assuming this is all true (which I don't), I don't feel sympathy for the author for not purchasing a paid plan.
[0] https://en.wikipedia.org/wiki/There_ain%27t_no_such_thing_as...
I have nothing but good things to say about Cloudflare's free services. However my usage is limited to small static sites, which are free of surprises.
I was doing a couple of POSTs a minute on a paid plan, and I saw a random 2-3% of request to be failing they never hit my backend, speaking to support wasn't helpful at all, that's when we moved away from Cloudflare, I wouldn't base my business on it.
For what it's worth, Cloudflare actually provides error code stats for the free plan in Account Analytics. The 5xx errors bucket is at 0.18% for me, but I don't have that many requests.
One gotcha for CloudFlare is that I find they will serve CAPTCHA pages for asset requests (like JavaScript or CSS) which will break your site (obviously the user won't see the captcha when the browser was expecting JS or an image).
To avoid this you need to turn the firewall and security feats to "Essentially Off" at least for asset requests (you can do this partial blocking via a page rule).
That being said this doesn't seem to be an issue with Free vs Paid, just a general problem with their blocking.
To avoid this you need to turn the firewall and security feats to "Essentially Off" at least for asset requests (you can do this partial blocking via a page rule).
That being said this doesn't seem to be an issue with Free vs Paid, just a general problem with their blocking.
Are you sure they didn't fix this? I remember getting those all the time, but haven't encountered one for several months now.
I would have to check the timeline but I was seeing this frequently with their IPFS gateway (as you can't control the security settings). It was a couple of months ago now so maybe it has been fixed recently.
I'm curious, do you know how it was fixed? Is it not serving a captcha for the second request on a connection or something? Or does it somehow figure out that asset requests are "safe"?
I'm curious, do you know how it was fixed? Is it not serving a captcha for the second request on a connection or something? Or does it somehow figure out that asset requests are "safe"?
My experience was with using Tor Browser to access websites using cloudflare.
I'd expect it to be based on the Accept request header (and possibly the Content-Type response header to prevent bypassing it). Or perhaps even the Content-Type of past requests to that url.
But I don't really understand what the purpose of these captcha checks is in the first place. Handling a captcha challenge is more expensive for the server than most GET requests. Perhaps it's done in anticipation of later POST requests (which can't be blocked transparently without breaking the functionality of the website).
I'd expect it to be based on the Accept request header (and possibly the Content-Type response header to prevent bypassing it). Or perhaps even the Content-Type of past requests to that url.
But I don't really understand what the purpose of these captcha checks is in the first place. Handling a captcha challenge is more expensive for the server than most GET requests. Perhaps it's done in anticipation of later POST requests (which can't be blocked transparently without breaking the functionality of the website).
I have two domains with millions of requests a month and hundreads of GB of bandwidth, both of them work quite well even in free mode.
However today and few days ago too it seems requests going through Cloudflare are just timing out. So we finally move to paid plan
We're monitoring both endpoints (CF and the origin)
However today and few days ago too it seems requests going through Cloudflare are just timing out. So we finally move to paid plan
We're monitoring both endpoints (CF and the origin)
Do keep us updated, I also upgraded but i really dont know what benefits are there to pro plan, tried it before felt no difference.
For now I'd say
the advanced cache statistics (will help us reduce traffic sent to the S3 origin, in one case, and reduce bandwidth cost)
Traffic is served from the closest location to the user (instead of the bigger central locations, where they can serve traffic cheaper)
Plus paid suppport, for sure
the advanced cache statistics (will help us reduce traffic sent to the S3 origin, in one case, and reduce bandwidth cost)
Traffic is served from the closest location to the user (instead of the bigger central locations, where they can serve traffic cheaper)
Plus paid suppport, for sure
Have you talked to support about this problem?
Not yet, but easier to contact support once you get the subscription.
So that's another reason to subscribe since it's 20$ month. (AWS and other SaaS cost us an order of magnitude more)
Any case, what I meant to say, is that free plan works fairly well even at higher volumes.
If you expect more reliability, should definitely subscribe.
So that's another reason to subscribe since it's 20$ month. (AWS and other SaaS cost us an order of magnitude more)
Any case, what I meant to say, is that free plan works fairly well even at higher volumes.
If you expect more reliability, should definitely subscribe.
This seems a bit outrageous if true. They advertise as a DDOS protection service and then do this?
That is SUPER scary, as a high volume website
If you're such a high volume site then why not pay for CDN/DDOS Protection? I'm amazed so many people feel this is something they deserve for free, and then complain when there's some kind of limitation on the free thing they're receiving.
If someone gave me a free car and I suddenly discovered that it had stopped working at 8:30am on Monday morning because the free car plan didn't support peak hour, just when I needed it most, I'd be annoyed as hell. This would be the case no matter how much I got paid at my job. Getting a nasty surprise because you didn't realise a product was inferior is upsetting for reasons that should be pretty obvious.
Admittedly the cloudflare free plan does say it's for things "that aren’t business-critical", but what does that actually mean in terms of resource quota and expected uptime?
Admittedly the cloudflare free plan does say it's for things "that aren’t business-critical", but what does that actually mean in terms of resource quota and expected uptime?
I don't care as much for DDOS protection as for free easy SSL which gets semi annoying to set up for me personally. I assumed cloudflare was an option from their advertisment and never worried about it. Now I suppose I will.
So I looked on their site and they charge $20 a month for 4 hour support response for urgent issues. [0]
Isn't that quite reasonable? Especially if you're scared if your site goes down.
[0] https://www.cloudflare.com/en-gb/plans/
Isn't that quite reasonable? Especially if you're scared if your site goes down.
[0] https://www.cloudflare.com/en-gb/plans/
How do you expect to be protected from DDoS if not via limiting the number of connections?
I know from experience that they might take some actions if you take too much of a DDoS on their free plan but I never heard of that on some usual traffic, especially not when it's as low as the blog suggests (sub 10 rpm).
The attacks my blog received were in the thousand requests per second area when it got suspended.
I've been using Cloudflare for a site that has around 400k requests per day on a Free plan (although I did buy a dedicated certificate and a domain from them) and Cloudflare has been very reliable to me except few hiccups in the past. I've had an instance of Cloudflare failing only on a single region, but after talking to support it turns out to be some route caching issue on their side, which subsequently fixed. I would not assume anything until there's some clarification on Cloudflare side. It might just be a bug.
They have also censored and removed sites in the past without notice.
nervous tick Reading this title triggered traumas :D
Same applies for AWS 'Free tier' database usage. Nothing free about it practically.
Same applies for AWS 'Free tier' database usage. Nothing free about it practically.
Has OP tried cloudfront (seeing that he uses AWS)?
this was my first question, but it seems like this was a price driven decision.
Sounds scary.
I would not want to use a CDN with some hidden limits. Especially not if I don't even get informed when rate limited.
I can imagine them rate limiting me in a way that I will never notice. Like only limiting requests from some other country or continent.
What is a good CDN for a site that has about a million visitors per month? And how much would one have to expect to pay for it?
I would not want to use a CDN with some hidden limits. Especially not if I don't even get informed when rate limited.
I can imagine them rate limiting me in a way that I will never notice. Like only limiting requests from some other country or continent.
What is a good CDN for a site that has about a million visitors per month? And how much would one have to expect to pay for it?
i am running a site with 30k+ MAU and 1mm+ req/month (according to CF stats) and did not experience any throttling issue
I thought about responding to some comments here but I feel like this point deserves a comment on its own: not to defend Cloudflare, but charging at least $19 / month, having 100 customers (as he said on the post) and not using a proper fully featured WAF to protect an enterprise product is... amateurish?
Also, if the infrastructure is in AWS, CloudFront would cost cents if he really does have "4 requests per minute" with full integration and logs.
I have the feeling that with the advent of cloud solutions, very big companies depend on these really small and very useful tools that disregard almost entirely a number of best practices and standards. The Solarwinds incident is going to happen a lot more, that's for sure.
Regarding Cloudflare: I'm fine with the free product not being great, but hiding the logs is not understandable. I bet a lot more people would upgrade if they knew.
Also, if the infrastructure is in AWS, CloudFront would cost cents if he really does have "4 requests per minute" with full integration and logs.
I have the feeling that with the advent of cloud solutions, very big companies depend on these really small and very useful tools that disregard almost entirely a number of best practices and standards. The Solarwinds incident is going to happen a lot more, that's for sure.
Regarding Cloudflare: I'm fine with the free product not being great, but hiding the logs is not understandable. I bet a lot more people would upgrade if they knew.
What does the WAF get you that Cloudflare's free plan doesn't? I'm not doubting you, I genuinely want to know.
Cloudflare doesn't offer a WAF in the free plan, only a DDoS attack mitigation service. This means it doesn't go into the application level (i.e. no SQL Injection protection or XSS). They do, however, offer a filtering capability.
You can see more in the Cloudflare page [1], they explain well what it is. None of the features they talk about come in the free version.
[1] https://www.cloudflare.com/learning/ddos/glossary/web-applic...
[1] https://www.cloudflare.com/learning/ddos/glossary/web-applic...
An SLA for starters
I've always seen free plans as a way to get started. A great way for a startup to get things up before they get money rolling in. Then as you grow, you outgrow the free plan but by then you have the money to upgrade.
Is the issue with free services in general? Does any other vendor (AWS, Fastly or Akamai) offer a decent free service?
It does seem to be more Marketing if you want to lock in customers before they get too big to switch.
It does seem to be more Marketing if you want to lock in customers before they get too big to switch.
Free Services like this are offered for 3 reasons
1. Marketing...
2. Data collection (after all if you are not paying you are the product) There is monetary value in collection, observation, and tracking of a vast amount of internet traffic.
3. Education. If you get people just starting out in their careers using your product or service for they hobby, personal project, etc then when they have a business need for some like your product or service they will just naturally choose you. Adobe and MS has been doing this for decades with low or no cost education licensing
1. Marketing...
2. Data collection (after all if you are not paying you are the product) There is monetary value in collection, observation, and tracking of a vast amount of internet traffic.
3. Education. If you get people just starting out in their careers using your product or service for they hobby, personal project, etc then when they have a business need for some like your product or service they will just naturally choose you. Adobe and MS has been doing this for decades with low or no cost education licensing
"I used a free service and it wasn't great"
Then use another service or pay for it.
This is highly unlikely to be as simple as the author conjects, we're talking about a service that processes an enormous amount of traffic and if what the author suggests is true, would someone else not have noticed by now? It's certainly possible that different infrastructure is used by free/paid plans, and perhaps this specific site was hosted on an unhealthy instance. But we don't have any external data points here to analyse - only those reported by Cloudflare, along with anecdotal reports.
Edit: I see the title has now been changed, at least we're reducing clickbait.
Then use another service or pay for it.
This is highly unlikely to be as simple as the author conjects, we're talking about a service that processes an enormous amount of traffic and if what the author suggests is true, would someone else not have noticed by now? It's certainly possible that different infrastructure is used by free/paid plans, and perhaps this specific site was hosted on an unhealthy instance. But we don't have any external data points here to analyse - only those reported by Cloudflare, along with anecdotal reports.
Edit: I see the title has now been changed, at least we're reducing clickbait.
For a low volume slack bot, why even have Cloudflare front it?
[deleted]
You can blame Cloudflare for not meeting their promises on free tier. But you can't blame them for "hiding" the errors. You are responsible to monitor your service, and it's good practice to have that happen externally from your environment. If you have a public API, monitor it publicly. Things like Pingdom and synthetic monitoring exist for this reason.
99566E92F7A4(1)
[deleted]
what a load of horseshit, this is my free plan 30 day stats, 2 order magnitude more request, a order magnitude less cache hits, no request limit in sight
https://i.imgur.com/UOlW836.png
edit: changed bull crap to the more appropriate horseshit in response to this unsubstantiated-bordering-conspiracy blog post, because if the former is already enough to triggers the community downvoting brigades, no point in showing any restraint
https://i.imgur.com/UOlW836.png
edit: changed bull crap to the more appropriate horseshit in response to this unsubstantiated-bordering-conspiracy blog post, because if the former is already enough to triggers the community downvoting brigades, no point in showing any restraint
I guess there is a gamble/risk
my money is on "there's some piece of this guy stack that's causing cf servers to timeout on reads"
Probably similar to AWS/GCP's grey listing systems, which increase quotas slowly over time.
What sort of traffic patterns do you have? Are they dispersed between a large number of Cloudflare datacenters? Or few?
Very few, we're a regional level startup, the CF map doesn't show it well because it's per state and not per pop on the free account, but basically all hits in logs are marked MXP or FCO