We handle 80TB and 5M page views a month for under $400(blog.polyhaven.com)
blog.polyhaven.com
We handle 80TB and 5M page views a month for under $400
https://blog.polyhaven.com/how-we-handle-80tb-and-5m-page-views-a-month-for-under-400/
424 comments
> We consume 2GB/S, maxed out, which ends up being 5184TB a month. We have millions on users a day, all on a streaming video platform.
> It cost less than $2K/Month.
The solution in this article is serving on the order of 100TB/month for $400/month including a high speed global CDN, their API and database servers being hosted reliably, and redundancy and backup being handled by someone else.
Your solution is hosting on the order of 1000s of TBs of month (ignoring the database and other aspects of this website), but the price is an order of magnitude higher. You’ve also given up all of the automatic redundancy and hands-off management, and you don’t have the benefit of a high speed global CDN.
But more importantly, you have significantly higher engineering and on-call overhead, which you’re valuing at $0.
If anything, that only makes Polyhaven’s solution sound more impressive.
> Of course, for this price, you don't have redundancy and horizontal scaling.
Which is a huge caveat. The global CDN makes a big difference in how the site loads in global locations. Maybe not a big concern if you’re serving of static files with a lot of buffering, but they have a dynamic website and a global audience and they said fast load times are important.
> You also don't have to maintain and debug a system with redundancy and horizontal scaling.
But you have to do literally everything else manually and maintain it yourself, which is far from free.
All of these alternative proposals that value engineering time at $0/hour and assume your engineers are happy to be on-call 24/7 to maintain these servers are missing the point. You pay for turnkey solutions so you don’t have to deal with as much. Engineers don’t actually love to respond to on call events. If you can offload many of your problems to a cloud provider and free up your engineers for a nominal cost, do it.
> It cost less than $2K/Month.
The solution in this article is serving on the order of 100TB/month for $400/month including a high speed global CDN, their API and database servers being hosted reliably, and redundancy and backup being handled by someone else.
Your solution is hosting on the order of 1000s of TBs of month (ignoring the database and other aspects of this website), but the price is an order of magnitude higher. You’ve also given up all of the automatic redundancy and hands-off management, and you don’t have the benefit of a high speed global CDN.
But more importantly, you have significantly higher engineering and on-call overhead, which you’re valuing at $0.
If anything, that only makes Polyhaven’s solution sound more impressive.
> Of course, for this price, you don't have redundancy and horizontal scaling.
Which is a huge caveat. The global CDN makes a big difference in how the site loads in global locations. Maybe not a big concern if you’re serving of static files with a lot of buffering, but they have a dynamic website and a global audience and they said fast load times are important.
> You also don't have to maintain and debug a system with redundancy and horizontal scaling.
But you have to do literally everything else manually and maintain it yourself, which is far from free.
All of these alternative proposals that value engineering time at $0/hour and assume your engineers are happy to be on-call 24/7 to maintain these servers are missing the point. You pay for turnkey solutions so you don’t have to deal with as much. Engineers don’t actually love to respond to on call events. If you can offload many of your problems to a cloud provider and free up your engineers for a nominal cost, do it.
A python that knows python and linux is enough to support the up time needs and evolutions of this particular product. Actually a part time one.
The entire team is composed of half of one dev.
I'm 100% sure it's way cheaper than anybody that has AWS on resume.
Of course, some things have to give, like the global CND, and some data guarantees.
Everything is a compromise. It's all depend of what is important for your project.
EDIT: also, my comment was not meant to oppose the article, but rather confirm the view that you should calibrate your setup to your project. Doing so will lead to great savings in hosting, and project complexity. A lot of projects don't need the cloud.
The entire team is composed of half of one dev.
I'm 100% sure it's way cheaper than anybody that has AWS on resume.
Of course, some things have to give, like the global CND, and some data guarantees.
Everything is a compromise. It's all depend of what is important for your project.
EDIT: also, my comment was not meant to oppose the article, but rather confirm the view that you should calibrate your setup to your project. Doing so will lead to great savings in hosting, and project complexity. A lot of projects don't need the cloud.
> The entire team is composed of half of one dev.
Who is on-call 24/7/365, never takes vacation ever, and is always available to fix the website?
It’s weird how much HN hates jobs with on-call requirements, but every time cloud services come up the solutions always involve forcing someone to be permanently on call to save a few hundred dollars per month in hosting costs.
Who is on-call 24/7/365, never takes vacation ever, and is always available to fix the website?
It’s weird how much HN hates jobs with on-call requirements, but every time cloud services come up the solutions always involve forcing someone to be permanently on call to save a few hundred dollars per month in hosting costs.
Nobody is on call 24H.
If the system has a problem during the night, the users will wait until the morning.
The world doesn't stop because a streaming service is down for a few hours. It's not a medical service, or something thousands of businesses rely on.
It just looses a bit of money and users are grumpy for a day because they had to wait until they could access new content.
It's ok.
If the system has a problem during the night, the users will wait until the morning.
The world doesn't stop because a streaming service is down for a few hours. It's not a medical service, or something thousands of businesses rely on.
It just looses a bit of money and users are grumpy for a day because they had to wait until they could access new content.
It's ok.
> Nobody is on call 24H.
This one dev is literally on call 365 days a year and can never be away from a computer on vacation. If he leaves, the project has no one.
How is that not a problem? Surely you can see that this isn’t reasonable for anyone who wants to run a business, or any employee who doesn’t want the website to be their life.
> If the system has a problem during the night, the users will wait until the morning.
> It just looses a bit of money and users are grumpy for a day
If you’re running a website where extended outages are no big deal and you don’t care about lost revenue, then it’s not really a valid comparison to the typical business website.
Your situation is unique, not a model by which other companies should follow.
This one dev is literally on call 365 days a year and can never be away from a computer on vacation. If he leaves, the project has no one.
How is that not a problem? Surely you can see that this isn’t reasonable for anyone who wants to run a business, or any employee who doesn’t want the website to be their life.
> If the system has a problem during the night, the users will wait until the morning.
> It just looses a bit of money and users are grumpy for a day
If you’re running a website where extended outages are no big deal and you don’t care about lost revenue, then it’s not really a valid comparison to the typical business website.
Your situation is unique, not a model by which other companies should follow.
> If you’re running a website where extended outages are no big deal and you don’t care about lost revenue, then it’s not really a valid comparison to the typical business website.
I think you're severely underestimating how many businesses make a significant amount of money from their website, but doesn't actually have full-time developer available. An extended outage would cause significant revenue loss, but it's typically not a problem because outages are surprisingly rare when you (1) have a very stable traffic pattern and (2) you don't spend a lot of time adding features and refactoring. Pretty much every cloud outage we've seen was caused by a human configuration error, not fault machines.
I think you're severely underestimating how many businesses make a significant amount of money from their website, but doesn't actually have full-time developer available. An extended outage would cause significant revenue loss, but it's typically not a problem because outages are surprisingly rare when you (1) have a very stable traffic pattern and (2) you don't spend a lot of time adding features and refactoring. Pretty much every cloud outage we've seen was caused by a human configuration error, not fault machines.
> I think you're severely underestimating how many businesses make a significant amount of money from their website, but doesn't actually have full-time developer available.
No, I’m well aware. But there’s a simple solution to this problem: Don’t try to run and maintain your own servers. Pay a little extra to use cloud hosting and let it be someone else’s problem.
I take issue with these calls to setup and maintain your own custom solutions and servers, while also suggesting that the cost of engineering and maintaining such a custom setup should be ignored.
Running your own servers and not having developers is a recipe for an endless stream of contracting invoices that are going to cost far, far more than just using a hosted cloud solution.
No, I’m well aware. But there’s a simple solution to this problem: Don’t try to run and maintain your own servers. Pay a little extra to use cloud hosting and let it be someone else’s problem.
I take issue with these calls to setup and maintain your own custom solutions and servers, while also suggesting that the cost of engineering and maintaining such a custom setup should be ignored.
Running your own servers and not having developers is a recipe for an endless stream of contracting invoices that are going to cost far, far more than just using a hosted cloud solution.
I'd be on board with "pay a little extra to rent dedicated servers", but "move to one of the big-three cloud providers" doesn't sound like a sound financial decision for the case presented.
> pay a little extra
The whole idea is that it's not a little extra, but 2 orders of magnitude.
The whole idea is that it's not a little extra, but 2 orders of magnitude.
> No, I’m well aware.
No, apparently not.
> But there’s a simple solution to this problem: Don’t try to run and maintain your own servers. Pay a little extra to use cloud hosting and let it be someone else’s problem.
But that's only if it IS a "problem" in the first place. You have defined it as such, although Bitecode themselves said that for them, it simply isn't. (To paraphrase: "If the site is down, then it's down; so what? We'll fix it when we're in the office again.")
Just plain ignoring whether something is "a problem" or not is hardly being "well aware".
No, apparently not.
> But there’s a simple solution to this problem: Don’t try to run and maintain your own servers. Pay a little extra to use cloud hosting and let it be someone else’s problem.
But that's only if it IS a "problem" in the first place. You have defined it as such, although Bitecode themselves said that for them, it simply isn't. (To paraphrase: "If the site is down, then it's down; so what? We'll fix it when we're in the office again.")
Just plain ignoring whether something is "a problem" or not is hardly being "well aware".
[deleted]
> An extended outage would cause significant revenue loss, but it’s typically not a problem
This just seems like a bad decision from a business perspective. You are willing to endure a significant outage that will cost a lot of money but not pay to prevent it? Machines can and will fail.
This just seems like a bad decision from a business perspective. You are willing to endure a significant outage that will cost a lot of money but not pay to prevent it? Machines can and will fail.
You keep moving the goal post.
It seems it's criminal to run a service on the cheap, there must be a terrible human being behing it.
Well no, the dev is not attached 365 days on its computer.
A freelancer is hired part time for the duration of the vaccations. It cost a full dev salary for one month, taking in consideration training time, that's all.
> If you’re running a website where extended outages are no big deal and you don’t care about lost revenue, then it’s not really a valid comparison to the typical business website.
Most services can actually go down once a month, and be a viable business. You are not google or facebook.
In fact, most human service goes down for days: bakeries, lawyers, teachers, plumbers.
The fact your think internet services should be up all time is only in your head. Humans adapt perfectly.
It's not that a big deal. Most of our softwares are not as important as we want to think.
If you really want a 99.99999% up time, you going to increase your service quality by 10%, and your service cost by 1000000%.
The funny thing is, the downtime of ourservice has not being more than github's downtime in the last few years. So honestly the freelacer is mostly hired to have drinks on the house. Because monolythes are very reliable in the first place.
> Your situation is unique, not a model by which other companies should follow.
Every situation is unique, I never, ever stated it was " a model by which other companies should follow". You did.
There is no such thing as "a model by which other companies should follow". You must adapt to the situation and goals. Engineering is about compromises.
My post is simply stating the reality that you can get very far with good old tech.
And a lot of projects don't need the cloud, or high availability. Yet they pay premium for it.
It seems it's criminal to run a service on the cheap, there must be a terrible human being behing it.
Well no, the dev is not attached 365 days on its computer.
A freelancer is hired part time for the duration of the vaccations. It cost a full dev salary for one month, taking in consideration training time, that's all.
> If you’re running a website where extended outages are no big deal and you don’t care about lost revenue, then it’s not really a valid comparison to the typical business website.
Most services can actually go down once a month, and be a viable business. You are not google or facebook.
In fact, most human service goes down for days: bakeries, lawyers, teachers, plumbers.
The fact your think internet services should be up all time is only in your head. Humans adapt perfectly.
It's not that a big deal. Most of our softwares are not as important as we want to think.
If you really want a 99.99999% up time, you going to increase your service quality by 10%, and your service cost by 1000000%.
The funny thing is, the downtime of ourservice has not being more than github's downtime in the last few years. So honestly the freelacer is mostly hired to have drinks on the house. Because monolythes are very reliable in the first place.
> Your situation is unique, not a model by which other companies should follow.
Every situation is unique, I never, ever stated it was " a model by which other companies should follow". You did.
There is no such thing as "a model by which other companies should follow". You must adapt to the situation and goals. Engineering is about compromises.
My post is simply stating the reality that you can get very far with good old tech.
And a lot of projects don't need the cloud, or high availability. Yet they pay premium for it.
> A freelancer is hired part time for the duration of the vaccations. It cost a full dev salary for one month, taking in consideration training time, that's all.
You pay a full dev salary for one month every time someone wants to take a break?
It’s baffling that anyone can read an article about someone spending $400/month on cloud services and then start proposing things like this as an alternative.
Engineering labor is expensive. Cloud is surprisingly cheap once you factor in engineering costs.
You pay a full dev salary for one month every time someone wants to take a break?
It’s baffling that anyone can read an article about someone spending $400/month on cloud services and then start proposing things like this as an alternative.
Engineering labor is expensive. Cloud is surprisingly cheap once you factor in engineering costs.
One month part time once a year as an additional cost is way, way cheaper than anything else.
> Cloud is surprisingly cheap once you factor in engineering costs.
No, it's not, since you need somebody qualified to operate it. And such qualificatied employees is very expensive. And you will need them on call anyway, since it will break, just in different way than a bunch of private servers.
I'd argue the cloud would be more expensive, even if hosting were not, because you need a more expensive team to run it.
> Cloud is surprisingly cheap once you factor in engineering costs.
No, it's not, since you need somebody qualified to operate it. And such qualificatied employees is very expensive. And you will need them on call anyway, since it will break, just in different way than a bunch of private servers.
I'd argue the cloud would be more expensive, even if hosting were not, because you need a more expensive team to run it.
He never said they don't care about lost revenue. I am willing to guess they considered the lost revenue, weighed it against the cost of having someone on-call 24/7, and considered that the lost revenue was cheaper.
If you look at how frequently sites like Reddit used to have downtime, it doesn't seem to matter too much for consumer products. Having half a day downtime once a year might be completely acceptable.
If you look at how frequently sites like Reddit used to have downtime, it doesn't seem to matter too much for consumer products. Having half a day downtime once a year might be completely acceptable.
> This one dev is literally on call 365 days a year and can never be away from a computer on vacation. If he leaves, the project has no one.
How does the cloud solve this?
How does the cloud solve this?
You can host on AWS and still get hours-long downtime, as has happened recently.
Whether the cost of trying to add another 9 to your uptime is worth the marginal benefit is for each company to decide. Each 9 gets exponentially more expensive. A lot of companies who think otherwise actually can afford (and will, sooner or later, be forced) to be down once in a while.
Whether the cost of trying to add another 9 to your uptime is worth the marginal benefit is for each company to decide. Each 9 gets exponentially more expensive. A lot of companies who think otherwise actually can afford (and will, sooner or later, be forced) to be down once in a while.
"forcing someone to be permanently on call"
The company I work for has a lot of stuff in the cloud. We seem to have quite a few position's worth of people permanently on call.
The percentage of "data center"-type on-call situations has perhaps gone down somewhat since moving into the cloud, but it has not gone to zero, and it was never the majority of problems anyhow.
It seems like you're sneaking the idea in that if they would just pay a lot more money, the person on call wouldn't have to be on call. I'd like to know what cloud you're using, because it doesn't seem to be any of the ones I know of. If your service is critical to your business or project, you've got someone on some sort of call (maybe not overnight call, which is the really rough bit), period, or you've got a business that can disappear at any second.
The company I work for has a lot of stuff in the cloud. We seem to have quite a few position's worth of people permanently on call.
The percentage of "data center"-type on-call situations has perhaps gone down somewhat since moving into the cloud, but it has not gone to zero, and it was never the majority of problems anyhow.
It seems like you're sneaking the idea in that if they would just pay a lot more money, the person on call wouldn't have to be on call. I'd like to know what cloud you're using, because it doesn't seem to be any of the ones I know of. If your service is critical to your business or project, you've got someone on some sort of call (maybe not overnight call, which is the really rough bit), period, or you've got a business that can disappear at any second.
I think it's bizarre you're giving cloud marketing material to a person, whose in a position to know their own situation, as-if you were describing their situation.
Like.. what are you expecting to happen? You'll just gaslight them into thinking they dont know how their business works?
Like.. what are you expecting to happen? You'll just gaslight them into thinking they dont know how their business works?
I’m discussing the linked article, which is what this comment section is for and it’s what the parent commenter was comparing to.
Whether or not a site is hosted in the cloud, it will break from time to time. S** invariably happens, no matter what. So, even if you host in the cloud, you're going to have problems, but they will be different problems. A developer to backup and support the site will be required, one way or the other. Case in point: polyhaven.com (the subject of this article) is not reachable as I write this.
One has to presume that they have priced this and found the non-cloud version over-all cheaper.
Labour isnt expensive if you're operating towards a minimum needed to function, and your systems are sufficiently operationally stable.
Labour isnt expensive if you're operating towards a minimum needed to function, and your systems are sufficiently operationally stable.
> Labour isnt expensive…
Engineering labor is definitely expensive.
The entire $400/month bill for this linked website will only get you 2-3 hours of consulting time. They’re getting an enormous value by just offloading the work to someone else and not having to worry about it.
Engineering labor is definitely expensive.
The entire $400/month bill for this linked website will only get you 2-3 hours of consulting time. They’re getting an enormous value by just offloading the work to someone else and not having to worry about it.
That is based on the id a cloud base service would not need a team to make it work.
I'd argue it actually need a more expensive team.
I'd argue it actually need a more expensive team.
You have to do an apple-to-apples comparison though. If you're comparing a single colo'd machine vs a 3000-EC2 instance fleet with a load balancer, api nodes, database nodes (and requisite db admin team), and Kafka and DynamoDBs somewhere in there, then the cloud is going to be more expensive to manage.
Barring in-depth research (which I'd love to read if someone has any links), it's not clear on a 1:1 basis what's cheaper. Paying for someone's time to research hardware and talk to vendors, run POs for them, figure out where/how to install them (Equinix is expensive), and RMA hard drives as that comes up; versus not paying for that and instead paying a cloud vendor for that privilege. Throw on top a changing hiring landscape (how much 'sysadmins' cost vs 'devops') and it really depends on the size of this hypothetical fleet that we're trying to manage, and how complicated the backend of the site is. If there's no real backend to speak of, Cloudflare's CDN for static assets is going to be way cheaper, and available now, vs anything you could possibly build from scratch, that would maybe be ready in a couple months.
Barring in-depth research (which I'd love to read if someone has any links), it's not clear on a 1:1 basis what's cheaper. Paying for someone's time to research hardware and talk to vendors, run POs for them, figure out where/how to install them (Equinix is expensive), and RMA hard drives as that comes up; versus not paying for that and instead paying a cloud vendor for that privilege. Throw on top a changing hiring landscape (how much 'sysadmins' cost vs 'devops') and it really depends on the size of this hypothetical fleet that we're trying to manage, and how complicated the backend of the site is. If there's no real backend to speak of, Cloudflare's CDN for static assets is going to be way cheaper, and available now, vs anything you could possibly build from scratch, that would maybe be ready in a couple months.
if only there were an `if`
Does it work for video as well?
Thanks! Exactly that! I always tell that customers and their response is:
-But google/facebook/amazon...
-But uptime needs to be 99.999
-But everyone uses cloud
Most businesses are not a trading-market, have less then 100 peoples (aka you are probably not another amazon), and no bonus using a cloud/kubernetes etc.
But it's the same old story, in the 00's i used the ~same arguments against buying OracleDB ;)
-But google/facebook/amazon...
-But uptime needs to be 99.999
-But everyone uses cloud
Most businesses are not a trading-market, have less then 100 peoples (aka you are probably not another amazon), and no bonus using a cloud/kubernetes etc.
But it's the same old story, in the 00's i used the ~same arguments against buying OracleDB ;)
And you can tell them all of those are possible, but they need to have a massive budget, not just for the monthly bills but also to hire experts to set things up with those constraints.
I worked at a company once that, from higher up, said that they had to have five nines of uptime. We had some really good cloud engineers there (one guy set up a server / internet container for the military in Afghanistan; in hindsight he said they should've just sent a container of porn dvd's), and they really went to town. For that five nines uptime, you're already pretty much required to set up your infrastructure to use multiple availability zones, everything redundant multiple times, etc.
Of course, the actual software we wrote was just a bunch of CRUD services written in nodejs (later scala because IDK), on top of a pile of shit java that abstracted away decades of legacy mainframes.
I worked at a company once that, from higher up, said that they had to have five nines of uptime. We had some really good cloud engineers there (one guy set up a server / internet container for the military in Afghanistan; in hindsight he said they should've just sent a container of porn dvd's), and they really went to town. For that five nines uptime, you're already pretty much required to set up your infrastructure to use multiple availability zones, everything redundant multiple times, etc.
Of course, the actual software we wrote was just a bunch of CRUD services written in nodejs (later scala because IDK), on top of a pile of shit java that abstracted away decades of legacy mainframes.
Either your compute demand is highly elastic, or your revenue and profit scale with usage, otherwise the Cloud is probably not for you. At least not in the longterm or for running websites.
> uptime needs to be 99.999
Isn't AWS down like every two months for a few hours? That's far off the 99.999% mark. No one can guarantee 100% uptime and sometimes it's even better to have that under your control (eg. have a dedicated server and a backup one from different providers).
My point is that, if you want the highest possible uptime, you shouldn't rely on a single (cloud) provider.
Isn't AWS down like every two months for a few hours? That's far off the 99.999% mark. No one can guarantee 100% uptime and sometimes it's even better to have that under your control (eg. have a dedicated server and a backup one from different providers).
My point is that, if you want the highest possible uptime, you shouldn't rely on a single (cloud) provider.
any single one of those arguments is cargo-cult actually, isn't it?
All means, no end.
All means, no end.
Well most of them are Non IT businesses, so they just know what peers/family etc told them what todo (or what they do), then read some (oneside-)sponsored media and now they ~know whats best, it's Azure with WindowsServerDatacenter-edition, SQLServer-enterprise and McAffee antivirus ;) but who can be mad about it? It's just not their field of expertise or even interest.
fair enough, however they juggle the buzzwords.
If they didn't but cared about their business (they know I hope) and make hard requirements to IT, that would help. IT should then come up with solutions to those real-world problems. We're not talking just about hobbyists, do we?
That's what Eric Evans talks about in DDD as I got it.
If they didn't but cared about their business (they know I hope) and make hard requirements to IT, that would help. IT should then come up with solutions to those real-world problems. We're not talking just about hobbyists, do we?
That's what Eric Evans talks about in DDD as I got it.
> fair enough, however they juggle the buzzwords.
Oh absolutely true, you don't want to look completely clueless in front of people who take your money to setup an infrastructure ;)
>and make hard requirements to IT, that would help
That would make stuff so much more easy. Often it even lacks even a inventory of used applications/hardware....network.
Just one example:
Made a plan for new hardware and network (new cabling etc), then i walked around the workshop and there was that dusty machine running...i asked what it is...a dos-machine with...wait...TOKENRING. That machine was an integral part of the whole workshop ;) However we made a virtual FreeDOS machine and buy'd a software converter for the machine protocol, the 25yo cnc-machine needed a new card (ethernet instead of token-ring, very lucky we found that thing) so there was that one little DOS-machine no one though about who could stop the whole "modernization".
Oh absolutely true, you don't want to look completely clueless in front of people who take your money to setup an infrastructure ;)
>and make hard requirements to IT, that would help
That would make stuff so much more easy. Often it even lacks even a inventory of used applications/hardware....network.
Just one example:
Made a plan for new hardware and network (new cabling etc), then i walked around the workshop and there was that dusty machine running...i asked what it is...a dos-machine with...wait...TOKENRING. That machine was an integral part of the whole workshop ;) However we made a virtual FreeDOS machine and buy'd a software converter for the machine protocol, the 25yo cnc-machine needed a new card (ethernet instead of token-ring, very lucky we found that thing) so there was that one little DOS-machine no one though about who could stop the whole "modernization".
> you don't want to look completely clueless
That's why I give them money, so they bare with my cluelessness on their field. Would do it myself otherwise.
Or the way round: "I hire smart people for a lot of money, why would I tell them what to do" (Steve Jobs)
> that one little DOS-machine
The donkey does the work and the horse gets the fame.
That's why I give them money, so they bare with my cluelessness on their field. Would do it myself otherwise.
Or the way round: "I hire smart people for a lot of money, why would I tell them what to do" (Steve Jobs)
> that one little DOS-machine
The donkey does the work and the horse gets the fame.
[deleted]
> But it's the same old story, in the 00's i used the ~same arguments against buying OracleDB ;)
But no matter how logically convincing your arguments were, most of the time upper manglement just went on buying Oracle, right...?
But no matter how logically convincing your arguments were, most of the time upper manglement just went on buying Oracle, right...?
Get two servers, stick them in different cities. $4k a month and you have resilience. $6k a month and you have resilience and double your throughout.
Some stuff makes sense to put on iaas, dns often does for example.
Some stuff makes sense to put on iaas, dns often does for example.
4k/mnt for 2 Servers? That's a little bit to much, 200euro for two servers at hetzner is perfectly fine ;)
I say server, presumably it’s the entire solution of however many servers, storage, network etc.
If it’s 2k/month for a non resilient solution, it’s in the order of 4k for a resilient solution (you need to relocate assets both ways but it’s in that order of magnitude)
If it’s 2k/month for a non resilient solution, it’s in the order of 4k for a resilient solution (you need to relocate assets both ways but it’s in that order of magnitude)
You can have redundancy and horizontal scaling with private servers and still end up paying less than what you would on AWS and the like.
I have some clients who use AWS and others who prefer colo and/or dedicated servers from traditional datacenters. The latter group can afford to over-provision everything by 3-4x, even across different DC's if necessary. DC's aren't yesterday's dinosaurs anymore. The large ones have a bunch of hardware on standby that you can order at 3 a.m. and start running deployment scripts in minutes.
I have some clients who use AWS and others who prefer colo and/or dedicated servers from traditional datacenters. The latter group can afford to over-provision everything by 3-4x, even across different DC's if necessary. DC's aren't yesterday's dinosaurs anymore. The large ones have a bunch of hardware on standby that you can order at 3 a.m. and start running deployment scripts in minutes.
Not including extra human cost in the analysis is just disingenuous. I think to manage private servers of that size you would need at least two extra experts totalling at least $20k/month.
> I think to manage private servers of that size you would need at least two extra experts totalling at least $20k/month.
What? You set up the deployment once, and then you only need to touch it when things go horribly wrong, which is every couple of months, or to make minor quick tweaks and run some updates. Let's be generous, and say you need 10 h/month, which is about 1/16 of a person-month. And if things go horribly wrong, everybody drops what they are doing to fix things, anyway, no matter if you're on AWS, dedicated/colo or run your own data center.
When you significantly change your architecture/deployment, then you need to put in more time again, but if you build your code with need to scale and such things in mind from the get-go, then that won't come up much or at all.
What? You set up the deployment once, and then you only need to touch it when things go horribly wrong, which is every couple of months, or to make minor quick tweaks and run some updates. Let's be generous, and say you need 10 h/month, which is about 1/16 of a person-month. And if things go horribly wrong, everybody drops what they are doing to fix things, anyway, no matter if you're on AWS, dedicated/colo or run your own data center.
When you significantly change your architecture/deployment, then you need to put in more time again, but if you build your code with need to scale and such things in mind from the get-go, then that won't come up much or at all.
> What? You set up the deployment once, and then you only need to touch it when things go horribly wrong, which is every couple of months
Right, which is exactly why people pay extra for cloud managed services.
If things are going “horribly wrong” every couple of months then you must necessarily be on call 24/7 and never take vacation or time away. In practice, you need at least two people to manage on-call coverage so you’re not completely uncovered if someone gets sick, decides to take vacation, wants to travel away from a computer and so on.
Right, which is exactly why people pay extra for cloud managed services.
If things are going “horribly wrong” every couple of months then you must necessarily be on call 24/7 and never take vacation or time away. In practice, you need at least two people to manage on-call coverage so you’re not completely uncovered if someone gets sick, decides to take vacation, wants to travel away from a computer and so on.
Things go horribly wrong with AWS hosted stuff as well. And a lot of companies have a single-point-of-failure AWS person. While you're not wrong in general, nothing you just wrote is specific to running on dedicated servers vs AWS.
We're talking volunteer-run projects, though. Who cares if it's not available 24/7? Best-effort is good enough. Those managed cloud services also fail often, you just have no information and no recourse about it.
Things go horribly wrong every year or so. The site goes down. It's fine. This isn't Facebook.
What?! Maybe if you hire SV-skilled engineers on location in Sillicon Valley, but you can easily serve 2GB/second (on infra worth $2K/month) with one sys-admin dealing with it, and for way less than a whopping $10k/month.
> with one sys-admin dealing with it,
And if that one sys-admin wants to go on vacation? Or travel away from a computer? Or takes another job?
You can never have “just one” admin handling a server and being on-call 24/7.
Would you really want a job where you could never, ever be away from a computer because you’re the only on-call person? This doesn’t work.
And if that one sys-admin wants to go on vacation? Or travel away from a computer? Or takes another job?
You can never have “just one” admin handling a server and being on-call 24/7.
Would you really want a job where you could never, ever be away from a computer because you’re the only on-call person? This doesn’t work.
> Would you really want a job where you could never, ever be away from a computer because you’re the only on-call person? This doesn’t work.
Works fine for me. $20K a month for two people doing f*k all is insane.
Works fine for me. $20K a month for two people doing f*k all is insane.
Why would it be necessary to have an engineer on call 24/7 ? If you do your risk calculations and an outage of 12 hours is acceptable in the expected frequency you just let the engineer have a nice evening and night an deal with the outage in the morning.
If outages are only to be expected once a year an you can tolerate 48 hours of outage you don't need any on call engineer. Most outages are cause by changes. You can test those and plan putting these in production and have elevated monitoring after this to catch problems early.
Only problem remaining is hardware outages. And those are very rare as long as you do decent lifecycle maintenance.
As others said before. Not everyone is Google or Amazon and needs 99,999% uptime.
But that AWS deployment doesn't manifest out of thin air either. Kubernetes and AWS knowledgable engineers aren't any less expensive.
Are you hiring? What infrastructure engineer are you paying $10,000/month for just to manage 4 servers? LOL
They didn’t mention a fleet size but 2GB/s is a single commodity server.
You don’t even need a single employee to manage a single server…
You don’t even need a single employee to manage a single server…
Which "commodity server" could serve 2GB/s(16gbps) to public internet?
Pretty much anything. How about a Dell R340 with a dual-10G NIC and some SSDs? That's not commodity, that's cheap, a commodity server would be a dual-Xeon but that's overkill for serving 16gbps.
I think that is the problem with modern day's Web Dev. ( Sorry )
With Cloud and SaaS. They are so abstracted from Hardware that their knowledge on basic hardware and server, everything from CPU, Storage I/O and Network are close to zero.
With Cloud and SaaS. They are so abstracted from Hardware that their knowledge on basic hardware and server, everything from CPU, Storage I/O and Network are close to zero.
At Netflix we’re doing close to 400Gbps on 1U commodity hardware, and pretty inexpensive.
And that is 400Gbps encrypted!
Side Note, aren't they 2U?
Side Note, aren't they 2U?
Yep! All TLS.
The production boxes are 2U, but it can be done in a 1U box.
The production boxes are 2U, but it can be done in a 1U box.
What do you imagine these experts would do? Rebuild the servers from raw materials each month, and hand-code all software in ASM weekly?
In the real world, once most hosting platforms are up and running, the maintenance overhead is pretty low.
In the real world, once most hosting platforms are up and running, the maintenance overhead is pretty low.
No, the system is maintained by one single part time dev. That's the entire dev team.
> extra human cost
Where? Costs vary hugely across the world
Where? Costs vary hugely across the world
If you are worried about getting your blog post to the top of Reddit or Hacker News (I've never been there myself); you can have a very modest web server or even a pay per request serverless sort of thing and pay $20 real quick to Cloudflare if you happen to get popular. It's the Bart Simpson method of highly scalability[0], for static content you can have global datacenter coverage in a couple minutes or so if you use them for DNS to start with. It even works if the origin server goes down.
[0]https://www.youtube.com/watch?v=aKU3hMvD31w
[0]https://www.youtube.com/watch?v=aKU3hMvD31w
> you can have a very modest web server or even a pay per request serverless sort of thing and pay $20 real quick to Cloudflare if you happen to get popular.
I get the impression that a lot of the critics in this thread don't really understand Cloudflare, how cheap it is, or even the concept of CDNs in general.
$20/month for Cloudflare Pro is a steal for what you get. Spinning up a dedicated server in a single datacenter somewhere isn't going to give the same results, especially if your users are geographically distributed like in this case.
I get the impression that a lot of the critics in this thread don't really understand Cloudflare, how cheap it is, or even the concept of CDNs in general.
$20/month for Cloudflare Pro is a steal for what you get. Spinning up a dedicated server in a single datacenter somewhere isn't going to give the same results, especially if your users are geographically distributed like in this case.
> I get the impression that a lot of the critics in this thread don't really understand Cloudflare, how cheap it is, or even the concept of CDNs in general.
You’re talking past the point here. It doesn’t matter how cheap if you’re fundamentally opposed to enabling cloud flare to reach its meat hooks further into the Internet.
This is no different from arguments about embedding google analytics or “just paying for windows” instead of using Linux.
You’re talking past the point here. It doesn’t matter how cheap if you’re fundamentally opposed to enabling cloud flare to reach its meat hooks further into the Internet.
This is no different from arguments about embedding google analytics or “just paying for windows” instead of using Linux.
CDNs are a commodity business with tons of competition. Cloudflare is winning by offering a better product.
You should take issue with all the other companies that have failed to deliver something as compelling.
You should take issue with all the other companies that have failed to deliver something as compelling.
I don't think the problem is a specific CDN. Is that everyone ends up using the same CDN, so when Cloudflare has problems, it affects everyone. Same with AWS, large swaths of the internet goes down if AWS does, which sounds great for AWS in marketing material, but less great for the general usability of the web.
> You’re talking past the point here
What point? Nobody said anything about "cloud flare to reach its meat hooks" in the article or the above thread except you?
What point? Nobody said anything about "cloud flare to reach its meat hooks" in the article or the above thread except you?
> What point? Nobody said anything about "cloud flare to reach its meat hooks" in the article or the above thread except you?
The OP mention "cloud flare to reach its meat hooks" in the thread attacking those who haven't jumped into Cloud flare's bandwagon by putting up a strawman on how that's only due to ignorance.
OP clarified that misrepresentation by pointing out the risk of allowing a single company to control the CDN market specifically and serving web content in general.
The OP mention "cloud flare to reach its meat hooks" in the thread attacking those who haven't jumped into Cloud flare's bandwagon by putting up a strawman on how that's only due to ignorance.
OP clarified that misrepresentation by pointing out the risk of allowing a single company to control the CDN market specifically and serving web content in general.
I figure helping Cloudflare get it's meathooks in the internet offsets all the really big companies that have their meathooks in, or at any rate doesn't worsen the real problem.
> (...) helping Cloudflare get it's meathooks in the internet offsets all the really big companies (...)
What? No. Cloudflare reported a revenue of half a billion dollars, and already controls about half the CDN market.
Let's put things in perspective: in comparison with Cloudflare's business, AWS is a minor player and an underdog with less than half of Cloudflare's market share.
Cloudflare is by no means a small company or an upstart or a David among Goliaths. Cloudflare is in fact and by far the Goliath of the CDN world.
What? No. Cloudflare reported a revenue of half a billion dollars, and already controls about half the CDN market.
Let's put things in perspective: in comparison with Cloudflare's business, AWS is a minor player and an underdog with less than half of Cloudflare's market share.
Cloudflare is by no means a small company or an upstart or a David among Goliaths. Cloudflare is in fact and by far the Goliath of the CDN world.
Just to understand you better: are you only talking about CDN activities from AWS here? Because I see websites talking about a quarterly revenue of tens of billions of dollars for AWS.
> Spinning up a dedicated server in a single datacenter somewhere isn't going to give the same results, especially if your users are geographically distributed like in this case.
Maybe not, but is the target audience that shills out $20/month really the type of people who have optimized their site to such an extent that shaving 50ms off the request latency by having your edge cache geolocated is really the type of thing that makes the difference? most of that group could probably do a lot of other optimizations that probably count for more.
Maybe not, but is the target audience that shills out $20/month really the type of people who have optimized their site to such an extent that shaving 50ms off the request latency by having your edge cache geolocated is really the type of thing that makes the difference? most of that group could probably do a lot of other optimizations that probably count for more.
> Maybe not, but is the target audience that shills out $20/month really the type of people who have optimized their site to such an extent that shaving 50ms off the request latency by having your edge cache geolocated is really the type of thing that makes the difference?
The common mistake is to pick a server geographically close to yourself, only access it from low-latency connections, and then assume that everyone in the world is seeing the same thing.
Or to only visit your own site with everything already in the browser cache. If you're not seeing cold start loads, you're not seeing what every new visitor to your website is seeing.
Consider the Photopea.com website. The author explained in a comment below that he spends $60/month to host the site without a CDN. Several of us loaded the site and it took 2.5 - 5.0 seconds to load. He could sign up for a cheap Cloudflare account, reduce the size of his server (due to caching), and the load times for everyone would drop by a significant amount.
If you're hosting simple, static content like a blog for an audience that doesn't care about load times, then of course nothing matters. But for modern, content-rich websites (photos especially) it can actually be a substantial improvement to add a CDN even if you have a single fast server. You may not see it, but visitors from distant locations definitely will see a difference.
The common mistake is to pick a server geographically close to yourself, only access it from low-latency connections, and then assume that everyone in the world is seeing the same thing.
Or to only visit your own site with everything already in the browser cache. If you're not seeing cold start loads, you're not seeing what every new visitor to your website is seeing.
Consider the Photopea.com website. The author explained in a comment below that he spends $60/month to host the site without a CDN. Several of us loaded the site and it took 2.5 - 5.0 seconds to load. He could sign up for a cheap Cloudflare account, reduce the size of his server (due to caching), and the load times for everyone would drop by a significant amount.
If you're hosting simple, static content like a blog for an audience that doesn't care about load times, then of course nothing matters. But for modern, content-rich websites (photos especially) it can actually be a substantial improvement to add a CDN even if you have a single fast server. You may not see it, but visitors from distant locations definitely will see a difference.
With some browser security policy that blocks part of the download, the homepage www.photopea.com clocks in at 3.80MB (so it should be much higher in practice). In this case, it's mostly JS, so designing your website properly (without JS, especially if the app itself is wasm not JS) would have much better savings than moving to CloudFlare CDN.
A CDN is more times than not the wrong answer to a real problem. Shave off your website and consider content-addressed protocols for big static asset download (like the textures from the article). If you run your website as a lightweight glorified Bittorrent index you'll notice your costs are suddenly a lot less, and you can still have a smaller "Download over the web" button as fallback.
A CDN is more times than not the wrong answer to a real problem. Shave off your website and consider content-addressed protocols for big static asset download (like the textures from the article). If you run your website as a lightweight glorified Bittorrent index you'll notice your costs are suddenly a lot less, and you can still have a smaller "Download over the web" button as fallback.
> Consider the Photopea.com website. The author explained in a comment below that he spends $60/month to host the site without a CDN. Several of us loaded the site and it took 2.5 - 5.0 seconds to load
This is a conclusion i am extremely doubtful of.
Ping time new york <-> tokoyo is about 180ms. So lets say as a worse case the ping time to the single server is 180ms (its probably not that bad), and lets say the latency to cloudflare edge server is 20ms.
So using cloudflare on a cache hit (best case), you save something like 160ms per roundtrip.
Which don't get me wrong is a huge savings and worth it (although this scenario is hugely exagerated).
However say you want to load the page in under 1 second instead of 5 seconds. In this scenario you would basically have to have 25 round trips to bring the site from 5 seconds to 1 second just on rtt savings of having a geo located edge server. If your site needs 25 round trips to load, something else is clearly wrong. (And this is an exagerated case, the real world the benefit would probably be much less)
To be clear i'm not saying that geo located edge caches are bad or useless. They are clearly very beneficial thing. Its just not the be all and end all of web performance, and most people in the demographic we are talking about probably have much more important things to optimize (otoh using cloudflare is cheap and doesnt require a lot of skill, so it is a very low hanging fruit)
This is a conclusion i am extremely doubtful of.
Ping time new york <-> tokoyo is about 180ms. So lets say as a worse case the ping time to the single server is 180ms (its probably not that bad), and lets say the latency to cloudflare edge server is 20ms.
So using cloudflare on a cache hit (best case), you save something like 160ms per roundtrip.
Which don't get me wrong is a huge savings and worth it (although this scenario is hugely exagerated).
However say you want to load the page in under 1 second instead of 5 seconds. In this scenario you would basically have to have 25 round trips to bring the site from 5 seconds to 1 second just on rtt savings of having a geo located edge server. If your site needs 25 round trips to load, something else is clearly wrong. (And this is an exagerated case, the real world the benefit would probably be much less)
To be clear i'm not saying that geo located edge caches are bad or useless. They are clearly very beneficial thing. Its just not the be all and end all of web performance, and most people in the demographic we are talking about probably have much more important things to optimize (otoh using cloudflare is cheap and doesnt require a lot of skill, so it is a very low hanging fruit)
> So using cloudflare on a cache hit (best case), you save something like 160ms per roundtrip.
Per packet. If you're doing a cold start, you'll pay that latency cost several times over: first the TCP handshake (3 roundtrips), and then the TLS handshake (2 more roundtrips). That's 800ms of extra latency before you even get to sending the first HTTPx request.
Cold start latency matters a lot.
Per packet. If you're doing a cold start, you'll pay that latency cost several times over: first the TCP handshake (3 roundtrips), and then the TLS handshake (2 more roundtrips). That's 800ms of extra latency before you even get to sending the first HTTPx request.
Cold start latency matters a lot.
> In this scenario you would basically have to have 25 round trips to bring the site from 5 seconds to 1 second
You’re forgetting that the TCP protocol itself is bidirectional. High latency connections will have lower throughout, especially at the beginning of transmission, because the data isn’t literally just streaming in one direction.
You’re forgetting that the TCP protocol itself is bidirectional. High latency connections will have lower throughout, especially at the beginning of transmission, because the data isn’t literally just streaming in one direction.
Anything over 100ms [1] is perceived as not-instant by a user. If you wait 2RTTs with 50ms per round trip, then you've already exceeded this threshold.
[1]: https://stackoverflow.com/questions/536300/what-is-the-short...
[1]: https://stackoverflow.com/questions/536300/what-is-the-short...
1.13 seconds to load https://www.google.co.uk/ (even after preloading it and agreeing to their popup)
3.44 seconds to do a search for "donkey"
3.44 seconds to do a search for "donkey"
Don't you get tht same for free with GitLab Paged and last I checked Cloudflare was free too for caching static HTML assets?
> last I checked Cloudflare was free too for caching static HTML assets?
If not free then very cheap, as I understood TFA: Wasn't that why they have two separate domains and serve static assets from one of them, to be able to use the cheapest Cloudflare tier for that domain = those assets?
If not free then very cheap, as I understood TFA: Wasn't that why they have two separate domains and serve static assets from one of them, to be able to use the cheapest Cloudflare tier for that domain = those assets?
[deleted]
superkuh(1)
I've had my personal $5/mo Digital Ocean VPS Wordpress site hit the top of HN before. I kept an eye on htop, but it handled it just fine. Exciting times.
As long as you have a decent backend it's no problem. If you're using some python/ruby/JS thing you're probably going to need some kind of reverse proxy to keep up with top of HN. If you're using a Haskell/Rust/C++/etc. compiled backend you're probably fine.
$20 a month for a Static site? that feels like a first world problem. I feel like everything is over engineered, unless you need subsecond delivery of your assets which are very huge, cdn doesn’t even makes sense. For a blog whats the point, your site is not going to receive heavy traffic every hour from global locations everyday. If you are concerned about returning users cache your assets on their machine. If your site is video heavy I can understand. I run a Django site for a zoo on a $10 instance, I have 2 others running on same instance. Never had issue with page speed even on 2G or under load my instance didn’t suffer. My storage on s3 is proxied via Nginx and cached on user device and in Nginx, I never even had a downtime due to traffic. I use fail2ban for basic protection. If it comes to DDOS im behind cloudflare free tier. $20 per month for a blog? Lol.
You missed the point, it's "pay $20 real quick to Cloudflare if you happen to get popular". There is a generous free tier, and for years I have paid nothing to Cloudflare, but I serve 2GB total every month to visitors, YMMV.
No I didn’t miss the point, $20 is weeks worth of meal in majority of countries. People from those countries run successful blogs without paying a penny to cloudlfare. I serve much more than 2GB per month without all that on a dynamic site. Paying cloudlfare for that is like putting a tarp over a dumpster fire to quickly hide it. I would rather think why my static site fails like shit when it doesn’t even need server side processing and correct it. I’m not saying cloudflare/CDN is useless, it shines when you are serving huge assets every hour to lot of people globally, or want to secure from bot traffic, or other useful features it provides. Heck you could even host a static site on s3 or netlify for free and answer all the traffic in the world. Remember Google returned subsecond results much before cloudflare or global CDNs were in place.
One variable you miss is the price of your own time.
To pay USD $20 and unload a problem to some 3rd party service takes under 30min, while running a scalable, high performance web-server on low budget is hard and time consuming - even impossible for devs with no sufficient devops/admin skills, which is sadly a majority.
Back in the day, as a student with no money and a time to spare I used to do it all myself too, guerrilla style. Nowadays tinkering with my private servers would mean taking time from my real job, and that just doesn't make sense financially, my time is way more valuable and scarce now.
That's why we have the economy of specialists in the first place. One can do everything in DIY fashion, but in our civilization it's usually cheaper to hire a plumber's or carpenter's services than to invest in learning the skills, buying the tools and then doing it, if it's not your primary source of income. It's no different with CDN services.
To pay USD $20 and unload a problem to some 3rd party service takes under 30min, while running a scalable, high performance web-server on low budget is hard and time consuming - even impossible for devs with no sufficient devops/admin skills, which is sadly a majority.
Back in the day, as a student with no money and a time to spare I used to do it all myself too, guerrilla style. Nowadays tinkering with my private servers would mean taking time from my real job, and that just doesn't make sense financially, my time is way more valuable and scarce now.
That's why we have the economy of specialists in the first place. One can do everything in DIY fashion, but in our civilization it's usually cheaper to hire a plumber's or carpenter's services than to invest in learning the skills, buying the tools and then doing it, if it's not your primary source of income. It's no different with CDN services.
I never spent any other time other than the initial deployment. Like I said if your tap is leaking find a competent plumber or if you have the skill fix the leak, don’t pay for water tanker every time you are out of water. Im not sure why you need high performance, scalable multisharded and other big cloud tech to run a static site. Im not against using cloudlfare, there is a usecase for it. I cant stress more about why cloudflare is such an overkill for static sites. A cheap VPS can go a long way before you have to start worrying about not being able to serve traffic for a static site, its not a blocking call. We would be wasting unnecessary resources, if we don’t fix the actual problem.
Then let your site fail, it's your choice.
For the rest of us who would rather not let our site fail, a quick one time $20 tarp over dumpster to handle the traffic in the meantime is good.
For the rest of us who would rather not let our site fail, a quick one time $20 tarp over dumpster to handle the traffic in the meantime is good.
My site never failed thats the thing without paying $20. And many developers can actually do it too. If you can’t serve decent traffic for a static site, I would rather fix the leaking tap once and for all than wasting money on a water tanker every time I am out of water.
And I am not saying you have to pay $20 every time. It's there to handle the traffic while you identify and fix the problem.
It's your choice if you don't want redundancy in place if any incidents happened.
It's your choice if you don't want redundancy in place if any incidents happened.
What does the $20 buy vs free plan? During a spike I was able to handle 30k pageviews per hour with Cloudflare free plan + $40 vps. Pretty much all pageviews were cached by Cloudflare and didn't hit my server. How does the $20 plan help?
I invite you to do the calculations of serving 80tb of data serving from S3.
Over 6k was my result but happy to be proved wrong.
What are you serving on a static site/ blog worth 80TB? I also said CDN makes sense for huge files. Please don’t nitpick.
4 kg of feathers is the same weight as 4 kg of lead.
Same thing applies to millions of 10kb files. Whether or not files are large is irrelevant to whether a CDN is a good idea.
If every file only ever gets requested once it may be pointless.
Same thing applies to millions of 10kb files. Whether or not files are large is irrelevant to whether a CDN is a good idea.
If every file only ever gets requested once it may be pointless.
Makes sense, thats what I mean too. There is a use case for CDNs which do no make sense to pay for if its a static site. Which can be done much cheaper and for free most of the time. Unless you are running a google scale static site.
Interpreting ‘weeks’ as 2 weeks, the GDP necessary to call that weeks worth of meals is $520 dollars. There’s only like 6 countries with a GDP that low.
If you are only talking about food, it may stretch a bit further, but you’re still far away from the majority of countries.
If you are only talking about food, it may stretch a bit further, but you’re still far away from the majority of countries.
You know what isn't served from a CDN? Hacker News. If HN can handle the traffic then so should be some static blog site.
As a designer, it's really easy for me to put something on Cloudflare... doing what HN does would take at least more than a few hours (and the knowledge) to set up properly.
Hacker news is text only though
Cloudflare's free tier is more than enough for basic CDN serving and DDOS protection. No need for Pro.
I had a blog post on the frontpage of hn for more tht 20 hours and my cheap Vps for 3€ a month could handle it perfectly since my website is just a statically generated website with hugo.
> pay $20 real quick to Cloudflare
Once cloudflare captures the web market we'll all pay back with interest. They are not a charity.
Once cloudflare captures the web market we'll all pay back with interest. They are not a charity.
Is there a good service to do this automatically (my site uses DigitalOcean and CloudFlare free), or even test how much load you can handle?
I tend not to realise when my site goes viral, as I'm based in Australia whereas my largest audience is in b the US (and I'm a bit of a Luddite!)
I tend not to realise when my site goes viral, as I'm based in Australia whereas my largest audience is in b the US (and I'm a bit of a Luddite!)
https://loader.io is a nice tool for that
Legend, thanks!
Wow, really glad I ran this tool.
Looks like with some dead basic optimisations (free versions of WP Fastest Cache and Autoptimise), my Wordpress site can handle around 1500 requests per second on a $5 DigitalOcean VPS before it starts to slow down.
On the old site, running on a shared host with less optimisations it would crap out at less than 10!
Seems like I don't need to worry about this after all.
Looks like with some dead basic optimisations (free versions of WP Fastest Cache and Autoptimise), my Wordpress site can handle around 1500 requests per second on a $5 DigitalOcean VPS before it starts to slow down.
On the old site, running on a shared host with less optimisations it would crap out at less than 10!
Seems like I don't need to worry about this after all.
[deleted]
> This website blog.polyhaven.com/how-we-handle-80tb-and-5m-page-views-a-month-for-under-400/ is currently offline. Cloudflare's Always Online™ shows a snapshot of this web page from the Internet Archive's Wayback Machine. To check for the live version, click Refresh.
hm
hm
I tried looking, and I find it very irritating that in that whole web page, there is not one single link in the menu/footer/sidebar to polyhaven.com home page so I can click and discover what it actually is. Not one.
This occurs on many company blogs as well operating under a subdomain like blog.whatever.com
To be clear, this is a very tangential and irrelevant nitpick and I understand it does not contribute to the content of the website itself.
This occurs on many company blogs as well operating under a subdomain like blog.whatever.com
To be clear, this is a very tangential and irrelevant nitpick and I understand it does not contribute to the content of the website itself.
Yes! - the same goes with support sites on different domains (support.whatever.com or whatever.zendesk.com etc) that appear in google results and don't have a link to the company.
I think this is a result of app stores enforcing links to external digital products thing. That's why all these support pages which need to be linked from app, don't have a single link to main site.
I had no problems navigating there. Every single link in the hamburger lead me to polyhaven.com
To the polyhaven.com home page? I don't think it did. They are all sub pages which adds another navigation step to see the root page, which is what I find irritating.
Not having a link forces people to navigate to the page on their own. For a lot of people that takes the form of a Google search. Lots of people googling specifically for your brand increases your search rankings.
Or so goes the theory anyway.
Or so goes the theory anyway.
I handle ~150TB and 26M page views for ~$500 by simply renting a few dedicated servers at hetzner. And if I didn't need quite a lot of processing power (more than average website), it would be much lower. I only need so many servers for the CPU power, not traffic.
This! Hetzner root servers, there is nothing else I know on the market that comes close in price/power ratio when you use the "Serverbörse". I run multiple low traffic websites, databases and a heavy traffic elk instance + my complete homelab in proxmox / docker for 48€ a month and there is even room for more. Highly recommended!
What is your strategy for dealing with server failure?
My app allows for "share nothing" architecture, basically using multiple DNS A records as load balancing. Currently it has 6 servers.
Even if 5 of them would go down at the same time, the site would still work as intended (thought probably couldn't handle peak load with less than 3 or 4). If one of or two are down, nothing happens.
Also completely reinstalling a server takes around an hour.
Even if 5 of them would go down at the same time, the site would still work as intended (thought probably couldn't handle peak load with less than 3 or 4). If one of or two are down, nothing happens.
Also completely reinstalling a server takes around an hour.
Yes, but comment I was replying to mentioned a 48 euro budget, it is a price of a single server.
I do not have an HA setup BUT all of my proxmox vms are snapshot backuped by proxmox backup server every night to my home NAS and to my office NAS. You can use one of many storage providers. SSHFS also works.
This is the cheapest and lowest administration solution I used till today.
For production usage I would recommend 3-4 similar speced 28€ machines and run a replicated proxmox cluster or ceph proxmox cluster.
As long as you treat servers as cattle, you can use Hetzner's own load balancer service and then you don't have a SPOF that you manage yourself. Their LBs are advertised as redundant / fault tolerant.
I don't know if it is possible to treat "Serverbörse" servers as cattle. They are all different. I know that k8s and Docker Swarm could in theory balance load between different machines, but never tried it in practice. But I had in practice some weird glitches with different CPU/motherboards/memory.
Also, comment I was replying to mentioned a 48 euro budget, it is a price of a single server.
Also, comment I was replying to mentioned a 48 euro budget, it is a price of a single server.
i think the whole point of vm/container is to enable "servers as cattle"
Yes, but "Serverbörse" machines are very non-uniform, which may be bad for load balancing. See yourself: https://www.hetzner.com/sb
Still you can get a nice server with i7 cpu 32gb memory with 20tb traffic for ~28 euro, vat incl. This is super competitive even with DO or others.
yes, it may be bad for indiscriminate/dumb load balancing. then again this only works in the firstplace if the work units represent a somewhat equivalent and small-ish workload.
once you place value on determinism (in regards to time spent on a task) you want a tightly specced distribution mechanism and/or a feedback loop to communicate busystate back to the LB.
once you place value on determinism (in regards to time spent on a task) you want a tightly specced distribution mechanism and/or a feedback loop to communicate busystate back to the LB.
I have 3 servers, 2/3 don't have hardware timestamping on the interface, 1 does. Makes a huge difference when it comes to NTP.
Which hetzner server have hardware timestamping on the interface?
I'd be really interested to know, since to the best of my knowledge, they don't have PTP solutions in their datacenter.
I'd be really interested to know, since to the best of my knowledge, they don't have PTP solutions in their datacenter.
I have two of the AX41-NVMe, and only one was lucky enough to get a "Intel Corporation I210 Gigabit Network Connection (rev 03)" which has the hardware timestamping.
In the finland datacenter, it appears there is a PTP running, though the offset is ~2.33 seconds off from NTP. Chrony says its a false-ticker, but I haven't really figured out how to get it configured correctly nor have I asked Hetzner for help. I've mostly just played around with it.
I did manage to also accidentally discover a local ISP's stratum 1's server is extremely close to me, as in a few microseconds away (I accidentally put hetzner's servers as a pool instead of a server and my NTP 'discovered' the stratum 1) ... I'm not using it, but I've thought about reaching out to them to ask if I can use it if I'm very nice.
In the finland datacenter, it appears there is a PTP running, though the offset is ~2.33 seconds off from NTP. Chrony says its a false-ticker, but I haven't really figured out how to get it configured correctly nor have I asked Hetzner for help. I've mostly just played around with it.
I did manage to also accidentally discover a local ISP's stratum 1's server is extremely close to me, as in a few microseconds away (I accidentally put hetzner's servers as a pool instead of a server and my NTP 'discovered' the stratum 1) ... I'm not using it, but I've thought about reaching out to them to ask if I can use it if I'm very nice.
Would you care to share the IPs? (both for the Hetzner PTP one and the local ISP stratum 1)
Is the 2.33s stable? If so, after deducing the stable offset, it could still be valuable.
Is the 2.33s stable? If so, after deducing the stable offset, it could still be valuable.
`/dev/ptp0` and `/dev/pps0` magically showed up and yeah, it appears to be stable. Chrony actually has it marked as a non-false ticker atm and is marked as a backup. Here's the stats:
Name/IP Address NP NR Span Frequency Freq Skew Offset Std Dev
==============================================================================
PPS 64 40 67m -6.827 33.059 -72ms 99ms
PTP 6 3 20 -9.585 0.005 -119ms 10ns
192.168.100.1 6 3 88h -0.159 0.000 -12ms 3815ns
192.168.100.2 6 3 29h -0.176 0.299 -11ms 2817us
ntp1.hetzner.de 14 10 154m -0.003 0.006 -47us 12us
ntp2.hetzner.de 11 6 103m -0.001 0.005 -21us 5711ns
ntp3.hetzner.de 9 6 137m -0.006 0.007 -72us 8949ns
That sounds very cool, did you happen to write about it? I currently run all this at home using a dedicated computer but electricity here isn’t cheap and I’d like to move everything to proxmox (although I’m struggling to figure out how to move all these more or less manually set up docker compose services and lxc containers)
Any experience with other providers of dedicated servers, e.g. Digitalocean?
Does DO even offer dedicated servers? I have used their vps in the past. It was ok, no issues, but more expensive than hetzners vps or dedicated for large traffic/cpu needs.
Yes DO does. Generally slightly more expensive than Hetzner, with a few more AWS-like features.
You cannot go wrong with Leaseweb or Nforce if you're looking for something in EU.
> This website blog.polyhaven.com/how-we-handle-80tb-and-5m-page-views-a-month-for-under-400/ is currently offline. Cloudflare's Always Online™ shows a snapshot of this web page from the Internet Archive's Wayback Machine. To check for the live version, click Refresh.
oh the irony
oh the irony
Indeed not the greatest advertisement for their infrastructure, but regardless you have to admit that <$400 is fairly impressive.
I bet it's powered by wordpress.
Aged well I see...
> To avoid this problem in the future, I decided to splurge a bit and go for a cloud solution where I wouldn’t have to worry about reliability, performance, scaling or integrity ever again: Google Firestore.
> Google Firebase is nice and convenient, but it is quite expensive. We could investigate some other managed database options in future.
I've only seen people get annoyed with Firestore over time, and migrating out of it. People do end up worrying. At first, They seem to choose Firestore because it's strongly marketed and seems suitable for a new project. And then data modeling, high prices or scalability becomes a problem.
> Google Firebase is nice and convenient, but it is quite expensive. We could investigate some other managed database options in future.
I've only seen people get annoyed with Firestore over time, and migrating out of it. People do end up worrying. At first, They seem to choose Firestore because it's strongly marketed and seems suitable for a new project. And then data modeling, high prices or scalability becomes a problem.
What is a good solution compared to firebase ? I mean for a new project, alone something like user handling, registration, email sending, recovering passwords seem to cost some effort. Also be able to react to events, is kind of trivial in firebase. So is it really a bad choise (for the beginning)? Later you can always try to optimize.
so the problem here is if you have all of those needs, then really soon in the life of the project you are going to need good tooling to handle them.
So choosing something which makes it "one-click" to set up but total madness to manage is a really short-term optimization, only worth it for a pure prototype which you will throw away no matter how successful.
If you know you need those things to reach success, then it is better to make the up-front investment to get good tools for those.
If you still want to go with a cloud provider, AWS Amplify has some interesting tooling. I've build products both against Amplify and Firestore (and Firebase). Yes, Firebase is a few days to a week faster to set up (integrated user management, as you say), but AWS gives more sophisticated control and is built around scripted deployments.
You pay for it, of course, and I'm not arguing AWS vs running your own server. I am saying if the choice is AWS or Firebase, that a few days researching the choice would give you knowledge you could use for launching the next 10 prototypes you have in mind.
So choosing something which makes it "one-click" to set up but total madness to manage is a really short-term optimization, only worth it for a pure prototype which you will throw away no matter how successful.
If you know you need those things to reach success, then it is better to make the up-front investment to get good tools for those.
If you still want to go with a cloud provider, AWS Amplify has some interesting tooling. I've build products both against Amplify and Firestore (and Firebase). Yes, Firebase is a few days to a week faster to set up (integrated user management, as you say), but AWS gives more sophisticated control and is built around scripted deployments.
You pay for it, of course, and I'm not arguing AWS vs running your own server. I am saying if the choice is AWS or Firebase, that a few days researching the choice would give you knowledge you could use for launching the next 10 prototypes you have in mind.
SupaBase is a good alternative in my opinion but they are not as feature rich yet - but you do get SQL again which is delightful
Many use Firebase for auth and user management only.
This is undoubtedly better than most chuck-it-on-AWS types would achieve, but it's still quite a bit more expensive than could achieved with marginally more effort. They are already on Digital Ocean - if they used their managed database offering, they could slash their Firestore bill and get rid of their Argo bill, bringing the cost sub $200 (but - massively reduced returns on investment of effort).
My thought is why don't the authors use github pages to host their static site with their links to a dedicated server with an unmetered link since their whole project is CC0. Downloading assets doesn't need to be low latency, thus I don't think the caching spatially near the user is important. Having a user login for patreon is something that would be missing and require some more thought I admit.
I think quite a lot of other people have mentioned in the thread that they are getting a lot of other "benefits" from using multiple services, but I don't see how these help solve the problem of data delivery besides taking advantage of the Cloudflare + Backblaze alliance which is $31 if their main website is a static one.
I think quite a lot of other people have mentioned in the thread that they are getting a lot of other "benefits" from using multiple services, but I don't see how these help solve the problem of data delivery besides taking advantage of the Cloudflare + Backblaze alliance which is $31 if their main website is a static one.
When I see per month or per day stats I get suspicious.
Let's say a month has 30 days -- 5 million views a months is 166,666k per day, 6944 per hour, 116 per minute, and ~1.92 per second.
1.92 qps.
Of course it's not expensive, it's a tiny amount of traffic!
Let's say a month has 30 days -- 5 million views a months is 166,666k per day, 6944 per hour, 116 per minute, and ~1.92 per second.
1.92 qps.
Of course it's not expensive, it's a tiny amount of traffic!
At 80TB (~83886080 MB) and 5000000 that's 16,77 MB per request in average. And at 1.92 qps that's 32,2 MB per second.
So they're saturating 25% of a gigabit uplink every second of the month.
So they're saturating 25% of a gigabit uplink every second of the month.
To look at a different angle, I would imagine the traffic isn't evenly distributed, and part of being happy with your offering will be if things don't grind to a halt during peaks.
In ideal world where the traffic is a horizontal line, yes. In real world you probably get idle times, and peaks of 20x that traffic.
Also note that they say "page views" which can translate in many more requests per each page opened.
Also note that they say "page views" which can translate in many more requests per each page opened.
It is indeed tiny. The backend for my app currently handles 500 qps for 300$ in GCP. I guess traffic is the biggest chunk for OP.
Yes. It should be per month with peak request per minutes or better per seconds. Since we are always optimising for the rare peak usage. Instead once you sum out to monthly or even yearly a lot of things are meaningless.
I know, but it is impressive that they are doing it for $400.
It's impressive it takes that much money to do almost no traffic.
A raspberry PI could handle the compute.
A raspberry PI could handle the compute.
Only if the traffic is evenly distributed during the day. Which it's almost certainly not.
You can get a $15 2.5 gigabit ethernet adapter for that Raspberry and now you're set for peak loads.
The traffic graphs in their blog look pretty stable. Maybe 1.5x, but not hugely uneven.
[deleted]
No it’s not. 2 qps is pitiful. If anything $400 is crazy expensive. I guess most of it goes to traffic/storage costs, otherwise they are being ripped off.
According to the blog post half the money goes to their CDN (Cloudflare) which is a good decision. And about $100 for the database hosting.
I don't think they are overpaying for what they are getting. $400 is not a lot for a proper site that serves a lot of bytes. I just don't think it's that impressive either -- it's just yet another website serving static assets through a CDN, with low QPS too, so I don't see why it's noteworthy.
I don't think they are overpaying for what they are getting. $400 is not a lot for a proper site that serves a lot of bytes. I just don't think it's that impressive either -- it's just yet another website serving static assets through a CDN, with low QPS too, so I don't see why it's noteworthy.
[deleted]
Fun read, love finding those combination perf/cost wins!
Have you considered using Firestore in Datastore mode[0]? It might make all of your reads free[1], though migrating could be a project.
[0] https://cloud.google.com/datastore/docs/firestore-or-datasto...
[1] https://stackoverflow.com/a/53313540
Have you considered using Firestore in Datastore mode[0]? It might make all of your reads free[1], though migrating could be a project.
[0] https://cloud.google.com/datastore/docs/firestore-or-datasto...
[1] https://stackoverflow.com/a/53313540
This website is one of the fastest web pages I’ve ever used. Although it’s simple, it immediately loads pages and pages of images. Like in a flash. I’m on mobile web and it’s such a snappy experience.
If Poly Haven is reading this: you nearly have an amazing FCP on the models browser but the external normalize.css file is killing you.[0] Self-hosting would drastically improve your 75p paint time.
I'd also encourage you load your fonts late via JS. Your main JS package competes right now with WOFF files from Google Fonts for priority and there's no need for that.
[0] https://www.webpagetest.org/result/220106_BiDc42_428a3caec56...
I'd also encourage you load your fonts late via JS. Your main JS package competes right now with WOFF files from Google Fonts for priority and there's no need for that.
[0] https://www.webpagetest.org/result/220106_BiDc42_428a3caec56...
> I'd also encourage you load your fonts late via JS
can't tell if joke, so:
there are already enough sites that display content for a split second and then some script runs (or fails?) and there is either nothing on screen or an error message.
this is ridiculous - please stop!
can't tell if joke, so:
there are already enough sites that display content for a split second and then some script runs (or fails?) and there is either nothing on screen or an error message.
this is ridiculous - please stop!
Loading experience for this would be that a fallback font loads first and is later replaced. If something errors, it’s not replaced.
What you’re describing has been thankfully avoidable for many years.
What you’re describing has been thankfully avoidable for many years.
this failure mode has been avoidable like forever and i have only seen it become more widespred in the last decade.
be sure i blame ad-tech and analytics
be sure i blame ad-tech and analytics
Why not just font-display:swap? Seems like the simpler solution.
optional would be more along the lines of what we want, but loading with JS let’s us load even later. They’re using swap today.
Agreed it's impressively fast, but a lot of the speed is just due to heavily compressing the thumbnails.
Agreed, incredibly fast
I was running www.Photopea.com with 10M page views a month for $40 a year.
Now, I upgraded to $60 a month. I never used any CDN.
Now, I upgraded to $60 a month. I never used any CDN.
Just wanted to say that I’ve been a Photoshop user since 4.0, but in the last year I’ve found myself loading photopea more than loading photoshop. Fantastic product. My main reason is purely the load speed - I can get a basic file up and running far faster. Great work on it - I’m a big fan!
> I was running www.Photopea.com with 10M page views a month for $40 a year.
> Now, I upgraded to $60 a month. I never used any CDN.
Why not go back to the $40/month plan and spend $20/month on Cloudflare Pro?
I was able to load photopea.com in about 5000ms, uncached. That's not terrible, but it was slow enough that I wondered for a few seconds if the site was broken. A CDN would cut that load time massively, and it wouldn't even be a net cost increase because you could downsize your server.
> Now, I upgraded to $60 a month. I never used any CDN.
Why not go back to the $40/month plan and spend $20/month on Cloudflare Pro?
I was able to load photopea.com in about 5000ms, uncached. That's not terrible, but it was slow enough that I wondered for a few seconds if the site was broken. A CDN would cut that load time massively, and it wouldn't even be a net cost increase because you could downsize your server.
Honest question: Why is everyone pushing CentralizedFlare pro so hard in this thread?
Cause it solves real issues for dirt cheap....
I mean really cold cache from the above user is at 5000ms that's pretty horrible. When I tried I got 2800ms. Take a look around it seems to be mostly from fetching static content which is a classic problem CDNs solve.
I mean really cold cache from the above user is at 5000ms that's pretty horrible. When I tried I got 2800ms. Take a look around it seems to be mostly from fetching static content which is a classic problem CDNs solve.
> Cause it solves real issues for dirt cheap...
One doesn't even need pro. In our use, Cloudflare's pages.dev and workers.dev serves single-digit TB traffic with triple-digit million hits at $30/mo. We pay $0/mo for origin servers since there aren't any.
One doesn't even need pro. In our use, Cloudflare's pages.dev and workers.dev serves single-digit TB traffic with triple-digit million hits at $30/mo. We pay $0/mo for origin servers since there aren't any.
Got it, yeah, web-tier caching is annoying to do yourself. Cheers and thank you kind sir or madam!
It doesn't have to be annoying, just easier.
People have been citing $30/month. Even if you value your time at min wage, its probably still cheaper than setting up your own varnish (and that's assuming you don't care about improving the last little bit of latency with geo located edge servers)
People have been citing $30/month. Even if you value your time at min wage, its probably still cheaper than setting up your own varnish (and that's assuming you don't care about improving the last little bit of latency with geo located edge servers)
Can we not call companies silly names. It seems childish.
To answer the actual question - its the obvious answer. They make a product that works well and is relatively cheap. Is it without drawbacks? obviously not, nothing is. However, for a significant segment of the market the value proposition makes sense.
To answer the actual question - its the obvious answer. They make a product that works well and is relatively cheap. Is it without drawbacks? obviously not, nothing is. However, for a significant segment of the market the value proposition makes sense.
Is there a decentralized solution you would recommend that’s better?
I appreciate you are replying to someone, but i feel like this doesn't quite make sense. You're running a website. Why would your web server be decentralized? HTTP is a client/server protocol after all.
Because caching is part of the HTTP spec. If content doesn't change on every request, you cache it with a timeout. If users are looking at your site from around the world, you want the cache to exist as close to them as possible. This is the job of the CDN. In Cloudflare's case, they sit between your web server and the user and can be that HTTP cache. If your site is popular, you might still have multiple versions of it across regions.
Because the linked article is about Cloudflare? It's like asking why a comment thread on an article talking about apples is discussing apples. Or are you asking why so many people are _pro_ Cloudflare?
It happens to be a big bullet point in TFA, so it appears in discussions of TFA, duh.
Because many competitors have taken an "Enterprise focused" approach. The choice of CDNs for more than media hosting and without $$$ commitment is quite small.
What would you propose as an alternative?
AWS CloudFront
Cloudfront's bandwidth pricing is outrageously high at $0.08/GB (up to $0.12/GB for some regions). Meanwhile, Cloudflare doesn't even charge for bandwidth.
> That's not terrible, but it was slow enough that I wondered for a few seconds if the site was broken.
So what? Unless your website is offering some superficial junk that can easily be found elsewhere, you’re not going to lose a user.
For something like photopea, there aren’t sub-second alternatives out there.
So what? Unless your website is offering some superficial junk that can easily be found elsewhere, you’re not going to lose a user.
For something like photopea, there aren’t sub-second alternatives out there.
Almost a perfect quote of GM in the 80s when Japanese cars took over it's market share
> For something like photopea, there aren’t sub-second alternatives out there.
if there is't one now, there will be. Besides, with a little CDN and tweaking you improve your users enjoyment. Why not?
> For something like photopea, there aren’t sub-second alternatives out there.
if there is't one now, there will be. Besides, with a little CDN and tweaking you improve your users enjoyment. Why not?
Show me one major media site that takes < 10 seconds to load
There's plenty of other image editing software.
Yeah, I opened web dev tools in firefox and looked at the request times. Seems strange not to use a CDN to improve the load time with a CDN, particularly with users in a different continent
Wow I’ve been using photopea forever and I’m a little starstruck to see you in the comments. Wanna say thank you for all the years your tool helped me
I would love to use a CDN. But I am afraid of things I do not fully understand, and I am afraid of making things too complex.
If I update a single file, how long it takes until nobody in the world can access the old version anymore? Does it take seconds / minutes / hours? Also, some files should not be cached at all (like PHP requests).
I am afraid it would take me days or weeks to learn everything and to cofigure the CDN properly, and I am risking being offline for a part of the world during that time. Also, if there is a problem at the CDN, my website would be broken, too. If someone could help me, you can write me at [email protected]
If I update a single file, how long it takes until nobody in the world can access the old version anymore? Does it take seconds / minutes / hours? Also, some files should not be cached at all (like PHP requests).
I am afraid it would take me days or weeks to learn everything and to cofigure the CDN properly, and I am risking being offline for a part of the world during that time. Also, if there is a problem at the CDN, my website would be broken, too. If someone could help me, you can write me at [email protected]
Man i tried photopea the other day and it runs faster than my old photoshop. You need to slow it down :)
Photopea is excellent. thank you sincerely.
Regards oftentimes forgotten Linux user
Regards oftentimes forgotten Linux user
[deleted]
Ever since MacOS broke Gimp I use Photopea regularly! Fantastic site!
Please tell more about this ‘MacOS broke Gimp’ thing, because it's the first time I hear of that and I'd like to dodge a firsthand experience.
I used it successfully on my Mac last week. It's still GIMP (bleh!) but it works.
You can consider Affinity Photo
All of the Affinity products are top notch. I wanted a simple vector tool and ended up liking Affinity Designer so much I picked up a course to learn it more in-depth. The best part is their pricing model doesn't suck, so I'm happy to pay for their wares.
Which course did you take?
It was an iPad specific course on Udemy [0] a couple years ago. Looks like it hasn't been updated in a while, but it was great for what I was after.
[0] https://www.udemy.com/course/vector-drawing-on-the-ipad-with...
[0] https://www.udemy.com/course/vector-drawing-on-the-ipad-with...
Same suggestion, purchased during the last Christmas sales period. PS’s price is out of my reach as a non-professional user.
In November 2020, Gimp 2.10.14 didn't work well or at all on Big Sur. The issues got workarounds and bugfixes, but there are still complaints about performance. https://gitlab.gnome.org/GNOME/gimp/-/issues/5917
Thanks for the straightforward answer.
i love photopea! helps me a lot when i share this with my client who have no photoshop or illustrator
Looks good.
FYI I tried to bookmark it with cmd-D, but the app has hijacked that combo!
FYI I tried to bookmark it with cmd-D, but the app has hijacked that combo!
Thanks! I wish we could detect all browser shortcuts, see https://twitter.com/ivankutskir/status/1422671831006392329
you, sir, are an absolute legend
[deleted]
On a similar note - does anyone have a recommendation for a budget cdn for video? I don’t necessarily need encoding capabilities (though it’s a plus), but rather high storage/bandwidth options.
BunnyCDN (mentioned in the blog post) has this functionality. It is mentioned in the blog post and they're using it for image processing, but it works for videos too.
I use it myself, but just as a normal static file CDN, but they have dedicated tabs in their UI for video stuff. Their bandwidth pricing is also very reasonable ($5/TB with their "bulk" option and $10/TB for standard)
I use it myself, but just as a normal static file CDN, but they have dedicated tabs in their UI for video stuff. Their bandwidth pricing is also very reasonable ($5/TB with their "bulk" option and $10/TB for standard)
First of all, kudos, I’m a junkie for such optimizations myself. It’s a pleasure that your systems are optimized well, you sleep well and you’re proud :)
Two thoughts:
1) CloudFlare offers incredible service. What would need the team of netadmins/sysadmins can be handled via their UI easily. I use Argo in one project and yes, it is a very efficient tool. CloudFlare is becoming like AWS, knowing its tools is a skill. Maybe one day well see an official CloudFlare Solution Architect certification?.. :))
2) For such traffic heavy websites I don’t think there’s a better combination than Static Websites hosted over CDN, which is cached aggressively and for the dynamic part serverless as a backend. If you configure it properly (cache optimization, rate limiting to not get a huge paycheck if DDoSed, etc…) this setup is like set-up-and-forget-it, without the need to invest loads of resources into it.
Two thoughts:
1) CloudFlare offers incredible service. What would need the team of netadmins/sysadmins can be handled via their UI easily. I use Argo in one project and yes, it is a very efficient tool. CloudFlare is becoming like AWS, knowing its tools is a skill. Maybe one day well see an official CloudFlare Solution Architect certification?.. :))
2) For such traffic heavy websites I don’t think there’s a better combination than Static Websites hosted over CDN, which is cached aggressively and for the dynamic part serverless as a backend. If you configure it properly (cache optimization, rate limiting to not get a huge paycheck if DDoSed, etc…) this setup is like set-up-and-forget-it, without the need to invest loads of resources into it.
I'm new to this stuff, but what the source article describes is essentially just that it is cheaper to run a static website with a CDN than without - right? I'm mostly curious because they described the cloudflare service as a cache layer rather than a CDN.
> They described the cloudflare service as a cache layer rather than a CDN.
Not sure how most CDNs work, but for Cloudflare, content is not actively pushed to edge nodes, it’s only pulled to an edge when needed and then cached there usually for an hour or two.
The caching is still distributed, and all of the tcp+https roundtrips are done to a local data center which makes things faster.
> what the source article describes is essentially just that it is cheaper to run a static website with a CDN than without - right?
Right. The cost savings comes from caching done by the CDN, and public static content is super easy to cache.
The distributed part makes things faster, but doesn’t necessarily save money.
Not sure how most CDNs work, but for Cloudflare, content is not actively pushed to edge nodes, it’s only pulled to an edge when needed and then cached there usually for an hour or two.
The caching is still distributed, and all of the tcp+https roundtrips are done to a local data center which makes things faster.
> what the source article describes is essentially just that it is cheaper to run a static website with a CDN than without - right?
Right. The cost savings comes from caching done by the CDN, and public static content is super easy to cache.
The distributed part makes things faster, but doesn’t necessarily save money.
It's annoying how you have to work around Cloudflare's weird Argo pricing like this. Paying per GB for already-cached data that takes the same route as non-Argo traffic makes no sense.
I believe it's also against Cloudflare's TOS to use it as an asset-hosting platform.
I believe it's also against Cloudflare's TOS to use it as an asset-hosting platform.
I've deployed similar stacks at scale multiple times and it is amazing how well it all works given how little it costs.
I often wonder why so many companies seem to spend tens of thousands of dollars per month on AWS services for their hosting. If most web apps nowadays are just CRUD apps, then why would you need to spend more than €300/month on hosting? All you'd need is data and the website itself, right?
I'm currently clicking through various European hosting services and they seem to offer great dedicated servers at good prices. I cannot wrap my head around these ridiculous costs I keep hearing about. A guy I know was telling me how they were spending tens of thousands of dollars per month on AWS at his company.
Is it because everyone is writing their stuff on node.js, putting 500kb of JS in every web page they serve, putting Docker everywhere when a chroot would suffice, microservices, Kubernetes, or hell even writing SAPs when we don't need them?
I am genuinely confused
I'm currently clicking through various European hosting services and they seem to offer great dedicated servers at good prices. I cannot wrap my head around these ridiculous costs I keep hearing about. A guy I know was telling me how they were spending tens of thousands of dollars per month on AWS at his company.
Is it because everyone is writing their stuff on node.js, putting 500kb of JS in every web page they serve, putting Docker everywhere when a chroot would suffice, microservices, Kubernetes, or hell even writing SAPs when we don't need them?
I am genuinely confused
> If most web apps nowadays are just CRUD apps, then why would you need to spend more than €300/month on hosting? All you'd need is data and the website itself, right?
If you're building a basic CRUD app and you have low traffic numbers, you don't need to spend tens of thousands of dollars on AWS.
But it's not as simple as picking a dedicated server, setting it up, and hoping for the best. At minimum you need periodic backups, testing and staging environments, solutions for rate-limiting your API, and so on.
And the "everything is just a CRUD app" meme is just that: a meme. Usually engineers who repeat this have only ever worked on simple CRUD apps, so they don't understand what it's like to work on anything different.
> I'm currently clicking through various European hosting services and they seem to offer great dedicated servers at good prices. I cannot wrap my head around these ridiculous costs I keep hearing about.
If you're working on the types of problems that are a good fit for setting up a single dedicated server on a random hosting provider, you're not working on the same types of problems that necessitate $10K AWS bills.
> I am genuinely confused
I've worked on projects with $10K+ monthly AWS bills. It wasn't node.js or Docker or Kubernetes. It was the sheer volume of connections we had to maintain and data we had to process.
But even if we could reduce our $10K/month AWS bill to $1000/month with a lot of engineering effort and manual management of our own servers, what would we gain? If I had to hire a single additional devops person or engineer to help manage this custom solution, the entire savings would be wiped out. And then some!
Wasting AWS resources isn't smart, but trying to DIY your solution to everything rarely makes financial sense when you look at how much engineering effort it takes and how much engineers cost. If I can spend $10K/month to avoid hiring 1 additional engineer, it's a financial win. I do not care if someone thinks they can do the same thing in $1K/month with endless amounts of custom setup and maintenance. I don't want it.
It also reduces the number of moving pieces that we have to manage manually, which reduces the on-call burden, which keeps people happier.
I really don't understand the anti-cloud hate on HN. It doesn't mirror the real engineering world at all.
If you're building a basic CRUD app and you have low traffic numbers, you don't need to spend tens of thousands of dollars on AWS.
But it's not as simple as picking a dedicated server, setting it up, and hoping for the best. At minimum you need periodic backups, testing and staging environments, solutions for rate-limiting your API, and so on.
And the "everything is just a CRUD app" meme is just that: a meme. Usually engineers who repeat this have only ever worked on simple CRUD apps, so they don't understand what it's like to work on anything different.
> I'm currently clicking through various European hosting services and they seem to offer great dedicated servers at good prices. I cannot wrap my head around these ridiculous costs I keep hearing about.
If you're working on the types of problems that are a good fit for setting up a single dedicated server on a random hosting provider, you're not working on the same types of problems that necessitate $10K AWS bills.
> I am genuinely confused
I've worked on projects with $10K+ monthly AWS bills. It wasn't node.js or Docker or Kubernetes. It was the sheer volume of connections we had to maintain and data we had to process.
But even if we could reduce our $10K/month AWS bill to $1000/month with a lot of engineering effort and manual management of our own servers, what would we gain? If I had to hire a single additional devops person or engineer to help manage this custom solution, the entire savings would be wiped out. And then some!
Wasting AWS resources isn't smart, but trying to DIY your solution to everything rarely makes financial sense when you look at how much engineering effort it takes and how much engineers cost. If I can spend $10K/month to avoid hiring 1 additional engineer, it's a financial win. I do not care if someone thinks they can do the same thing in $1K/month with endless amounts of custom setup and maintenance. I don't want it.
It also reduces the number of moving pieces that we have to manage manually, which reduces the on-call burden, which keeps people happier.
I really don't understand the anti-cloud hate on HN. It doesn't mirror the real engineering world at all.
> Is it because everyone is writing their stuff on node.js, putting 500kb of JS in every web page they serve, putting Docker everywhere when a chroot would suffice, microservices, Kubernetes, or hell even writing SAPs when we don't need them?
My response may come off as rude, but that's not the intention. From the above quote you've vastly over simplified all but the most simplistic environments. Someone could run all of the above for well under 10k month - those are not the reasons why costs are high
> putting 500kb of JS in every web page they serve
that's not much and CDNs serve that without issue or significant cost
> putting Docker everywhere when a chroot would suffice
Docker does not have much overhead over chroot with a large drop in security/isolation
> microservices
there are good and bad ways to do this - does not need to cost much. saying "microservices" is so broad to have no meaning in this context
> Kubernetes
think about why someone might run kube. Again, not that expensive unless you're really small where having master nodes would have a big impact on costs
> or hell even writing SAPs when we don't need them?
Do you mean SAP? SAP is the largest non-American software company by revenue. I don't think anyone likes working with SAP. You think people use it for without reason? There are reasons. Think about what those might be
But those are just details - when building anything of a decent size, nothing is a simple CRUD. There are always exceptions, limitation, biz logic, migration issues, schema issues. I don't care if you use Postgres, Mongo, Kafka or $SOMETHING_COOL
Then there's data retention. It's very very easy to keep PB of data in s3 or other data stores for biz or compliance needs
Then AWS gives IAM, which is very helpful in teams > 5
My response may come off as rude, but that's not the intention. From the above quote you've vastly over simplified all but the most simplistic environments. Someone could run all of the above for well under 10k month - those are not the reasons why costs are high
> putting 500kb of JS in every web page they serve
that's not much and CDNs serve that without issue or significant cost
> putting Docker everywhere when a chroot would suffice
Docker does not have much overhead over chroot with a large drop in security/isolation
> microservices
there are good and bad ways to do this - does not need to cost much. saying "microservices" is so broad to have no meaning in this context
> Kubernetes
think about why someone might run kube. Again, not that expensive unless you're really small where having master nodes would have a big impact on costs
> or hell even writing SAPs when we don't need them?
Do you mean SAP? SAP is the largest non-American software company by revenue. I don't think anyone likes working with SAP. You think people use it for without reason? There are reasons. Think about what those might be
But those are just details - when building anything of a decent size, nothing is a simple CRUD. There are always exceptions, limitation, biz logic, migration issues, schema issues. I don't care if you use Postgres, Mongo, Kafka or $SOMETHING_COOL
Then there's data retention. It's very very easy to keep PB of data in s3 or other data stores for biz or compliance needs
Then AWS gives IAM, which is very helpful in teams > 5
> Is it because everyone is writing their stuff on node.js, putting 500kb of JS in every web page they serve, putting Docker everywhere when a chroot would suffice
Complexity inflates HN readers CVs and keeps them employed.
Complexity inflates HN readers CVs and keeps them employed.
>If most web apps nowadays are just CRUD apps, then why would you need to spend more than €300/month on hosting? All you'd need is data and the website itself, right?
Because most web apps aren't simply CRUD apps. It's a meme said by armchair engineers who believe they could build twitter in weekend.
Because most web apps aren't simply CRUD apps. It's a meme said by armchair engineers who believe they could build twitter in weekend.
Here you confuse 'most apps' with Twitter and Facebook. Most apps are crud apps; there are some apps that are not. Those apps are the most used, but they are not most apps by app count.
Only people here on hn seem to think that everyone is slinging k8s and 1000 layers of microservices to build the most complex things on earth; the rest is earning their paycheck by building some forms in php.
Sadly many here are overengineering crud apps and then somehow making them out to be something more than crud apps because somewhere in the future they might be (probably not); as in, you can build 'a twitter' in 1 weekend and it will even probably handle good volume of users ( more than Twitter did at startup; it was often down or very slow) etc if you just take postgres with php on a few $/mo server. It is a crud app at that time (the flow for these user authored posts is built into every trivial and complex framework these days). Also it will most likely be bankrupt in a few months as that is what happens to startups; I rather spend money on building the business first and then scaling the tech; which is exactly what Twitter did. On hn it seems a lot of people work backward in that regard but that probably has also something to do with being a techie and VC interest in scalable tech.
Only people here on hn seem to think that everyone is slinging k8s and 1000 layers of microservices to build the most complex things on earth; the rest is earning their paycheck by building some forms in php.
Sadly many here are overengineering crud apps and then somehow making them out to be something more than crud apps because somewhere in the future they might be (probably not); as in, you can build 'a twitter' in 1 weekend and it will even probably handle good volume of users ( more than Twitter did at startup; it was often down or very slow) etc if you just take postgres with php on a few $/mo server. It is a crud app at that time (the flow for these user authored posts is built into every trivial and complex framework these days). Also it will most likely be bankrupt in a few months as that is what happens to startups; I rather spend money on building the business first and then scaling the tech; which is exactly what Twitter did. On hn it seems a lot of people work backward in that regard but that probably has also something to do with being a techie and VC interest in scalable tech.
I don't think that most workloads are webapps and many webapps are not just CRUD. HN is not the web either, in case HN formed such a viewpoint for you.
Until you're spending the salary of a large fraction of your engineers on hosting costs, there are much bigger things to worry about than hosting costs. AWS is user friendly and a lot of engineers are familiar with it, so developer productivity is strongly in its favor.
Also, you know what you're getting security-wise with AWS, and no one will blame you when your website / service goes down because AWS is down.
Also, you know what you're getting security-wise with AWS, and no one will blame you when your website / service goes down because AWS is down.
Wasn't that like IBM's business motto at some point? No one ever got fired for going with IBM.
Then IBM definitely lost a lot of business to cloud modernization. Maybe the next logical step is the cheaper CloudFlare, which another comment mentioned is actually pretty profitable and then I'm sure the next cheaper incumbent will come in with just slightly less trust and reliability.
The important thing here is caching/static delivery
I've had this argument too many times to count. Every page view does not need to be dynamic per user request. Creating a sane cache policy reduces origin resources, servers, cost, etc... and surprise gives a better user experience at the same time.
I've had this argument too many times to count. Every page view does not need to be dynamic per user request. Creating a sane cache policy reduces origin resources, servers, cost, etc... and surprise gives a better user experience at the same time.
Ironically, it's struggling to load the images for me.
How do 5M page views consume 80TB of traffic? That sounded like a lot, but calculates out to 1,600KB per page view. That's a lot of JS and images on every page :)
A few years ago when I hosted what felt to me like a popular site, I was serving serving > 150K page views a month from a 1.5Mbps adsl uplink. With that said, back then you could gzip everything and there was no letsencrypt or CloudFlare walls across the board (didn't exist yet).
Good thing bandwidth is easier to come by these days.
A few years ago when I hosted what felt to me like a popular site, I was serving serving > 150K page views a month from a 1.5Mbps adsl uplink. With that said, back then you could gzip everything and there was no letsencrypt or CloudFlare walls across the board (didn't exist yet).
Good thing bandwidth is easier to come by these days.
The website is for downloading 3D graphics assets, which are typically fairly large files.
I see.. was thrown off by:
> Running a massively popular website and asset resource while being funded primarily by donations has always been a core challenge of Poly Haven.
My brain went: poly.. fill.io or something
Thank you, sorry, am embarrassed.
> Running a massively popular website and asset resource while being funded primarily by donations has always been a core challenge of Poly Haven.
My brain went: poly.. fill.io or something
Thank you, sorry, am embarrassed.
> How do 5M page views consume 80TB of traffic? That sounded like a lot, but calculates out to 1,600KB per page view.
No, that's low by an order of magnitude. It is 16MB (16,000KB) per page view.
No, that's low by an order of magnitude. It is 16MB (16,000KB) per page view.
The site describes itself as, 'The Public 3D Asset Library'. I suspect that a good portion of this is downloading assets.
> asset resource
HDRIs, high res textures & 3D models
HDRIs, high res textures & 3D models
Did you even look at the article? They operate a digital asset store.
https://news.ycombinator.com/newsguidelines.html
'Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that."'
'Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that."'
I don't think they're a store, everything is free and Patreon supporters get early access. I do occasional 3D rendering and the HDRIs are great. The downloads are fast enough to make you ask "whoa, how is this free".
The origin website is currently down according to Cloudflare...
5M views per month is under 2 per second. In 2005, that was servable from a 16-bit pic microcontroller with a 10-base-t phy from microchip inc
It is annoying that the answer is basically "we paid someone [Cloudflare] to do it." In light of that I decided to run some of my own numbers and figure it out.
They are pushing about 245 Mbps out of Cloudflare (averaged over the month). Wholesale IP transit prices are anywhere from 10 cents to a dollar per Mbps depending on volume. Cloudflare dumps 40% of their traffic over peering, putting their price at $14.70/mo. Ignoring fixed capex of servers, Cloudflare is making about $25/mo on this customer.
Given the $11 Backblaze bill, I estimate about 2 TB of data.
A capable dedicated server with 2 TB of disk and 1 Gbps unlimited bandwidth will run about $30 in a major European metro, maybe double that for the US.
With a grand total of $370 vs. $60 worst case, they are spending 516% more to be "serverless."
Edit: Yes Cloudflare has more than one server. Double the price and put one in the US and you still come out ahead.
Edit 2: I'm not saying one way or the other is better. Just that the title is very clickbaity for a "put a credit card into a website" payoff.
They are pushing about 245 Mbps out of Cloudflare (averaged over the month). Wholesale IP transit prices are anywhere from 10 cents to a dollar per Mbps depending on volume. Cloudflare dumps 40% of their traffic over peering, putting their price at $14.70/mo. Ignoring fixed capex of servers, Cloudflare is making about $25/mo on this customer.
Given the $11 Backblaze bill, I estimate about 2 TB of data.
A capable dedicated server with 2 TB of disk and 1 Gbps unlimited bandwidth will run about $30 in a major European metro, maybe double that for the US.
With a grand total of $370 vs. $60 worst case, they are spending 516% more to be "serverless."
Edit: Yes Cloudflare has more than one server. Double the price and put one in the US and you still come out ahead.
Edit 2: I'm not saying one way or the other is better. Just that the title is very clickbaity for a "put a credit card into a website" payoff.
> A capable dedicated server with 2 TB of disk and 1 Gbps unlimited bandwidth will run about $30 in a major European metro, maybe double that for the US.
Not even close to comparable. I think you're ignoring all of the functions they're paying for. They're not just hosting a few files. Also, your "worst case" is literally the most optimistic best case for a standalone single server, which completely disregards any redundancy and assumes that the $30/month server is truly unlimited/unmetered in every way. That's not a good assumption.
Cloudflare is a global CDN that will be fast for everyone regardless of location and will soak up bursts of traffic with ease. It's also fast in a location-independent way, which won't be true for a single server in a single datacenter somewhere.
Finally, the amount of effort it takes to do this with Cloudflare is trivial. The amount of effort it would take to maintain and optimize a standalone solution is not negligible.
Cloudflare seems like a very good deal to me, unless you value your time at $0 and you have access to these truly unlimited, high-performance $30/month servers.
Not even close to comparable. I think you're ignoring all of the functions they're paying for. They're not just hosting a few files. Also, your "worst case" is literally the most optimistic best case for a standalone single server, which completely disregards any redundancy and assumes that the $30/month server is truly unlimited/unmetered in every way. That's not a good assumption.
Cloudflare is a global CDN that will be fast for everyone regardless of location and will soak up bursts of traffic with ease. It's also fast in a location-independent way, which won't be true for a single server in a single datacenter somewhere.
Finally, the amount of effort it takes to do this with Cloudflare is trivial. The amount of effort it would take to maintain and optimize a standalone solution is not negligible.
Cloudflare seems like a very good deal to me, unless you value your time at $0 and you have access to these truly unlimited, high-performance $30/month servers.
But they could deliver potentially ~300TB with a 1gbps unmetered link. Not that hard to come by, could even get to 3PB for $300/mo.
Back in the day we used Texan Colo data centers with DrFTPD to do just this at massive scale.
Bytes in, bytes out. Not a popular opinion here, but I firmly believe it's not necessary to play the CentralizedFlare game to get a winning outcome.
Back in the day we used Texan Colo data centers with DrFTPD to do just this at massive scale.
Bytes in, bytes out. Not a popular opinion here, but I firmly believe it's not necessary to play the CentralizedFlare game to get a winning outcome.
> But they could deliver potentially ~300TB with a 1gbps unmetered link. Not that hard to come by, could even get to 3PB for $300/mo.
Cloudflare Pro is $20/month per domain.
Trying to run everything through a single server in a single datacenter just doesn't compare for globally-accessed websites like this, even with a 1Gbps unmetered link.
Cloudflare Pro is $20/month per domain.
Trying to run everything through a single server in a single datacenter just doesn't compare for globally-accessed websites like this, even with a 1Gbps unmetered link.
I must be missing something, are you able to elaborate on how CentralizedPro Flare for $20 solves the OPs problems?
I've accessed plenty of sites physically located in the USA from various EU countries, not to mention AUS and I didn't notice much difference as long as they had a good interconnect.
Maybe not the best for YouTube, but there's only a few of those.
Don't get me wrong, I see the appeal and fell for CF early on but not a fan anymore since they've grown into such a behemoth.
Absolutely power corrupts absolutely.
I've accessed plenty of sites physically located in the USA from various EU countries, not to mention AUS and I didn't notice much difference as long as they had a good interconnect.
Maybe not the best for YouTube, but there's only a few of those.
Don't get me wrong, I see the appeal and fell for CF early on but not a fan anymore since they've grown into such a behemoth.
Absolutely power corrupts absolutely.
Are you referring to the linked article? It literally explains what Cloudflare does for them.
Everything might make more sense if you visit their website: https://polyhaven.com/ It's not just a dumb, static file server.
EDIT: The questions below are literally explained in the article, so I'm going to give up on this thread.
Everything might make more sense if you visit their website: https://polyhaven.com/ It's not just a dumb, static file server.
EDIT: The questions below are literally explained in the article, so I'm going to give up on this thread.
> Everything might make more sense if you visit their website: https://polyhaven.com/ It's not just a dumb, static file server.
At a glance, it still looks like mostly a highly cached static website + search. Its not exactly a high complexity backend.
Things are always more complex at scale, so there's probably more to it then what appears at first glance, but its not obvious what that is.
At a glance, it still looks like mostly a highly cached static website + search. Its not exactly a high complexity backend.
Things are always more complex at scale, so there's probably more to it then what appears at first glance, but its not obvious what that is.
[deleted]
Yes, and if they could pay 1/20th of their current bill to solve it, I'm sure they would.
I feel like we're heading into the weeds. Do you get the broader point I'm trying to make?
I feel like we're heading into the weeds. Do you get the broader point I'm trying to make?
Note: my comment is no longer relevant and looks insane because Badwolff made edits to completely change their message.
Good network connectivity makes a huge difference.
Back in the day when Joyent ran their public cloud you could beat Cloudflares free plan from Joyents Japan datacenter by a tiny amount in most cases.
Anyway, those days are gone and Cloudflare provides great value and probably the best quality network available. So if you have the problem they solve then you would be a fool to not put some serious thought into considering them.
Back in the day when Joyent ran their public cloud you could beat Cloudflares free plan from Joyents Japan datacenter by a tiny amount in most cases.
Anyway, those days are gone and Cloudflare provides great value and probably the best quality network available. So if you have the problem they solve then you would be a fool to not put some serious thought into considering them.
> Don't get me wrong, I see the appeal and fell for CF early on but not a fan anymore since they've grown into such a behemoth.
> Absolutely power corrupts absolutely.
This seems to capture the gist of the argument clearly. I hadn't realized cloudflare had grown large enough to fall into the "too big to not be evil" bucket already.
> Absolutely power corrupts absolutely.
This seems to capture the gist of the argument clearly. I hadn't realized cloudflare had grown large enough to fall into the "too big to not be evil" bucket already.
brace yourself before you read up on cf's origin
For the average person, why make it unnecessarily hard on yourself?
If it’s not your core competency and unless you possess the vast breadth and depth of skills to dig yourself out if something goes wrong, it seems like a distraction at best and a fatal mistake at worst, with the end result of saving 1-2 engineer hours worth of money per month.
For some self-hosting is a no-brainer, but I suspect that community is much smaller than many here would expect.
If it’s not your core competency and unless you possess the vast breadth and depth of skills to dig yourself out if something goes wrong, it seems like a distraction at best and a fatal mistake at worst, with the end result of saving 1-2 engineer hours worth of money per month.
For some self-hosting is a no-brainer, but I suspect that community is much smaller than many here would expect.
It seems disingenuous to count engineer hours at $200USD/hour for charitable donation-run projects. Is it charitable to funnel your donor money to CF? I'm sure CF is grateful for the donation..
Fact: Things break most often because of humans doing things - changes, like deployments and config modifications.
Set things up right to begin with, and in my experience you can leave them running for a long time without intervention.
Downvote all you like, but it's the truth.
Fact: Things break most often because of humans doing things - changes, like deployments and config modifications.
Set things up right to begin with, and in my experience you can leave them running for a long time without intervention.
Downvote all you like, but it's the truth.
> Is it charitable to funnel your donor money to CF? I'm sure CF is grateful for the donation..
Is the job getting done for the donors? I bet they don’t care about $300 if it means the website is highly available and the maintainer isn’t burned out from giving away $200/hr of opportunity cost all the time.
> Fact: Things break most often because of humans doing things - changes, like deployments and config modifications.
Totally agree, and sometimes success in your project forces your hand as you are pressed to add functionality or need to scale. If you can guarantee that you’re doing this work once, ignoring hardware failure or scaling, the scales may very well tip toward self-managed.
For a vast majority of projects, I think it just plainly makes sense to go this route, hence the success of these centralized hosts. It is the pragmatic option.
PS. For what it’s worth, I’m sorry you’re being downvoted simply for having an viewpoint that others disagree with.
Is the job getting done for the donors? I bet they don’t care about $300 if it means the website is highly available and the maintainer isn’t burned out from giving away $200/hr of opportunity cost all the time.
> Fact: Things break most often because of humans doing things - changes, like deployments and config modifications.
Totally agree, and sometimes success in your project forces your hand as you are pressed to add functionality or need to scale. If you can guarantee that you’re doing this work once, ignoring hardware failure or scaling, the scales may very well tip toward self-managed.
For a vast majority of projects, I think it just plainly makes sense to go this route, hence the success of these centralized hosts. It is the pragmatic option.
PS. For what it’s worth, I’m sorry you’re being downvoted simply for having an viewpoint that others disagree with.
I guess the question is what do you consider a long time?
RedHat usually break setups every 3-4 years which I personally consider way to frequent.
If I configure a server with automatic updates then I would like it to run with very little maintenance for at least a decade and a long time would be two plus decades.
RedHat usually break setups every 3-4 years which I personally consider way to frequent.
If I configure a server with automatic updates then I would like it to run with very little maintenance for at least a decade and a long time would be two plus decades.
While you are right CF provides a lot more, it really must be noted that Hetzner SB servers start at 28 EUR and they _do_ come with unlimited bandwidth and 2 x 3TB disks. For 30 EUR you can get a E3-1245 with 2x4TB disk. Just noting.
That would be a single node though, in contrast to cloudflare's edge, and would probably slow down download speeds for everyone outside europe
> Wholesale IP transit prices are anywhere from 10 cents to a dollar per Mbps depending on volume.
10 cents is far from the lower end for high quality IP transit.
Cloudflare doesn't pay anywhere near $14.70/mo for 80 TB/mo. Even I spend much less than that on 80 TB in a month. On top of that, peering makes it even cheaper for Cloudflare, as you said. Cloudflare is very likely using peering for much more than 40 % of the traffic now, so it's even cheaper for them.
The worst peering region for Cloudflare was North America in 2016 with 40 % peering according to Cloudflare blog posts.
10 cents is far from the lower end for high quality IP transit.
Cloudflare doesn't pay anywhere near $14.70/mo for 80 TB/mo. Even I spend much less than that on 80 TB in a month. On top of that, peering makes it even cheaper for Cloudflare, as you said. Cloudflare is very likely using peering for much more than 40 % of the traffic now, so it's even cheaper for them.
The worst peering region for Cloudflare was North America in 2016 with 40 % peering according to Cloudflare blog posts.
I was conservative in my estimates for IP transit because I don't know what CF pays, but the numbers only get better for them.
Cloudflare might not pay that on a per TB basis, but where are you getting 10 gige transit for under $1k??
Linode offers a 50 dCPU with 10Gbps out for $960. Or if you just need the bandwidth and don't need the CPU power, you can combine three $30 servers with 4Gbps each, and put them behind a $10 load balancer, for a total of 12Gbps out at $100/mo.
https://www.linode.com/pricing/
https://www.linode.com/pricing/
Plenty of providers, depending on how many 10G/40G links you're willing to buy, check out https://blog.telegeography.com/global-ip-transit-prices-decl... to see how much prices have gone down.
An actually capable dedicated bare metal server with a good amount of RAM, CPU and disk, and 1Gbps will be more like $250-300/mo USD.
This would be approximately equivalent in spec to what you could build for $2500 purchase cost if buying a 1U machine and colocating itself.
For 30 bucks a month you'll get something very old and weak.
This would be approximately equivalent in spec to what you could build for $2500 purchase cost if buying a 1U machine and colocating itself.
For 30 bucks a month you'll get something very old and weak.
The bulk of the traffic is serving static files. We figured out how to do this with "old and weak" hardware in the 90s.
Here is a 3.3 GHz Xeon, 24 GB of RAM, with the required disk and bandwidth for $33: https://oneprovider.com/order/item/dediconf/59
Here is a 3.3 GHz Xeon, 24 GB of RAM, with the required disk and bandwidth for $33: https://oneprovider.com/order/item/dediconf/59
> The bulk of the traffic is serving static files.
The bulk of the data transferring is static files because they're hosting large assets, but the rest of the article is about their API, database, etc.
A single server in a single datacenter isn't comparable to a global CDN. They made it clear in the article that they value global latency, and they're willing to pay more for it.
> Here is a 3.3 GHz Xeon, 24 GB of RAM, with the required disk and bandwidth for $33: https://oneprovider.com/order/item/dediconf/59
That has mechanical spinning hard disks and a CPU that was released literally a decade ago.
You're not going to replace Cloudflare, Firebase, their API server, and a global CDN with a single 10-year old machine serving files from a mechanical spinning hard disk, hosted by a company that almost nobody has heard of.
The bulk of the data transferring is static files because they're hosting large assets, but the rest of the article is about their API, database, etc.
A single server in a single datacenter isn't comparable to a global CDN. They made it clear in the article that they value global latency, and they're willing to pay more for it.
> Here is a 3.3 GHz Xeon, 24 GB of RAM, with the required disk and bandwidth for $33: https://oneprovider.com/order/item/dediconf/59
That has mechanical spinning hard disks and a CPU that was released literally a decade ago.
You're not going to replace Cloudflare, Firebase, their API server, and a global CDN with a single 10-year old machine serving files from a mechanical spinning hard disk, hosted by a company that almost nobody has heard of.
Sure you are for something like serving large image files (like, say 3D asset files?)... I serve >30TB/month off a Hetzner server with rotating rust drives for like $50/month doing pretty much that, multi-terabyte image datasets and gigabyte-sized DL models. Plus a few websites, which you've probably seen. Zero optimization, just rsync and nginx with defaults, works fine. (It'd be way cheaper except I wanted another 20TB or so of rust to get room to add more datasets to serve - after all, the server is barely breaking a sweat.)
> I serve >30TB/month off a Hetzner server with rotating rust drives for like $50/month
You could probably do it with Cloudflare for a similar price, but that's not what this article is about. The article says that their Cloudflare (not Argo) bill is only $40/month, and that's only because they have two domains. It's $20/month per domain.
The $400/month figure isn't just for static file hosting. It includes their API, database, and global CDN. They make a point to say that low-latency, global CDN is a priority for them, which is why they're paying extra for Cloudflare Argo.
I'm not sure why everyone is comparing what they're doing to a single fileserver hosted somewhere, serving up static files without regard to global latency. It's apples and oranges.
You could probably do it with Cloudflare for a similar price, but that's not what this article is about. The article says that their Cloudflare (not Argo) bill is only $40/month, and that's only because they have two domains. It's $20/month per domain.
The $400/month figure isn't just for static file hosting. It includes their API, database, and global CDN. They make a point to say that low-latency, global CDN is a priority for them, which is why they're paying extra for Cloudflare Argo.
I'm not sure why everyone is comparing what they're doing to a single fileserver hosted somewhere, serving up static files without regard to global latency. It's apples and oranges.
> They make a point to say that low-latency, global CDN is a priority for them
Which, franky, is dumb. File downloads depend on throughput, not time to first byte like a small javascript file. Once bits start flowing it doesn't matter if they travel half way around the globe.
It is like saying you are trying to optimize an ocean cargo ship for acceleration time.
Which, franky, is dumb. File downloads depend on throughput, not time to first byte like a small javascript file. Once bits start flowing it doesn't matter if they travel half way around the globe.
It is like saying you are trying to optimize an ocean cargo ship for acceleration time.
> Which, franky, is dumb. File downloads depend on throughput, not time to first byte like a small javascript file. Once bits start flowing it doesn't matter if they travel half way around the globe.
Round-trip latency has a significant impact on TCP throughput. It's not just about time to first byte.
They're also not a static file serving website.
If they were just serving static files and they didn't care about latency, they could pay $20/month for Cloudflare Pro and be done with it.
I don't understand why you continue to ignore the fact that this for running a high-traffic, global, dynamic website, not just a static file server.
Round-trip latency has a significant impact on TCP throughput. It's not just about time to first byte.
They're also not a static file serving website.
If they were just serving static files and they didn't care about latency, they could pay $20/month for Cloudflare Pro and be done with it.
I don't understand why you continue to ignore the fact that this for running a high-traffic, global, dynamic website, not just a static file server.
> Round-trip latency has a significant impact on TCP throughput. It's not just about time to first byte.
I've built three CDNs in my career, one at the Tbps scale. Latency only has two factors in throughput: time to scale window size, and retransmits. Modern TCP stacks handle the prior just fine, and the latter is only an issue with packet loss. You can also turn on HTTP/3 and remove your reliance on TCP entirely.
> I don't understand why you continue to ignore the fact that this for running a high-traffic, global, dynamic website, not just a static file server.
Because I actually looked at the site and read the article. The vast majority of the site is large static files. The frontend is a static JavaScript app that calls an API server. That API server is running on a $5/mo VM.
My offer still stands if you'd like a tour of a datacenter and see the sausage being made at scale. I'll throw in lunch and answer all your scalability questions if you like. But this thread is growing quite long and veering off topic.
I've built three CDNs in my career, one at the Tbps scale. Latency only has two factors in throughput: time to scale window size, and retransmits. Modern TCP stacks handle the prior just fine, and the latter is only an issue with packet loss. You can also turn on HTTP/3 and remove your reliance on TCP entirely.
> I don't understand why you continue to ignore the fact that this for running a high-traffic, global, dynamic website, not just a static file server.
Because I actually looked at the site and read the article. The vast majority of the site is large static files. The frontend is a static JavaScript app that calls an API server. That API server is running on a $5/mo VM.
My offer still stands if you'd like a tour of a datacenter and see the sausage being made at scale. I'll throw in lunch and answer all your scalability questions if you like. But this thread is growing quite long and veering off topic.
You can use different TCP congestion control algorithms, not just cubic. BBR2 has no issues with high RTT even with packet loss and is fair to other TCP streams.
It serves 25Mbps stream in real time to users in Japan and the US from a data-centre in Europe with no issues.
It serves 25Mbps stream in real time to users in Japan and the US from a data-centre in Europe with no issues.
Yahoo, Google, and eBay were built with gasp spinning hard disks.
But fine, if you don't believe it I will happily extend an invitation to come see in person my racks of spinning rust and 10 year old servers in Los Angeles or Fremont that are running services that are used by millions of end users. My email is in my profile.
But fine, if you don't believe it I will happily extend an invitation to come see in person my racks of spinning rust and 10 year old servers in Los Angeles or Fremont that are running services that are used by millions of end users. My email is in my profile.
> Yahoo, Google, and eBay were built with gasp spinning hard disks.
Which they ditched as soon as physically possible. The internet has changed a lot since then. That much should be obvious.
You can't reduce this article to "millions of end users" and then equate it with every other service. There's more to this article/service than you're suggesting, and there's more to a website than just users/month.
Which they ditched as soon as physically possible. The internet has changed a lot since then. That much should be obvious.
You can't reduce this article to "millions of end users" and then equate it with every other service. There's more to this article/service than you're suggesting, and there's more to a website than just users/month.
What's wrong with a spinning hard disk? The files are super easy to read, we're talking milliseconds, and once done it stays in the page cache in RAM... that completely negates any SSD advantage. SSDs have little to no benefit in serving static files.
If you're really strict on this it's trivial to preload these files in RAM after compiling your assets so you don't have to wait for a request.
If you're really strict on this it's trivial to preload these files in RAM after compiling your assets so you don't have to wait for a request.
Higher up someone calculated that they have 2TB of data, considering their $11/mo Backblaze bill. I calculate $11/$0.004/GB and get 2750 GB.
If you have 2.75TB of files to serve and have a bunch of simultaneous requests for different files:
- everything doesn't fit into the RAM page cache, unless you have 3TB of RAM in your server
- for files that aren't cached, you have to read from the drive
- when you have a bunch of different requests for different files, you're going to have a lot of disk seeks. Seeks are expensive on rotating media.
Serving a large collection of static files at scale can be quite challenging. At my previous e-commerce company, back in the early 2000's, we did our own image serving with RAID1 multiply-mirrored drives (before SSDs!) so we could multiply the seek capacity by the number of drives.
If you have 2.75TB of files to serve and have a bunch of simultaneous requests for different files:
- everything doesn't fit into the RAM page cache, unless you have 3TB of RAM in your server
- for files that aren't cached, you have to read from the drive
- when you have a bunch of different requests for different files, you're going to have a lot of disk seeks. Seeks are expensive on rotating media.
Serving a large collection of static files at scale can be quite challenging. At my previous e-commerce company, back in the early 2000's, we did our own image serving with RAID1 multiply-mirrored drives (before SSDs!) so we could multiply the seek capacity by the number of drives.
> What's wrong with a spinning hard disk? The files are super easy to read, we're talking milliseconds, and once done it stays in the page cache in RAM
They're running a dynamic website with an API and other dynamic functions.
It's not a static fileserver.
It might make more sense if you visit the website in question: https://polyhaven.com/
They're running a dynamic website with an API and other dynamic functions.
It's not a static fileserver.
It might make more sense if you visit the website in question: https://polyhaven.com/
24 GB of RAM means a lot of ability to cache data in RAM, which the OS does already by default. It of course depends on the type of content you have but generally traffic follows a power law, so most content can be served from RAM directly. The write side I'm a bit more worried about, as e.g. SQLite issues a sync request on each transaction (not sure about the other SQL implementations, should be similar there tho), and those might require seeking.
The article explains that they have database and API usage. You can't run a high-throughput database on spinning disks unless you disregard data integrity.
It's not just a static fileserver, despite some of the comparisons in this thread.
It's not just a static fileserver, despite some of the comparisons in this thread.
Like SSDs bring any new angle to this.
> You can't run a high-throughput database on spinning disks unless you disregard data integrity
Of course you can. That's the way it was and still is done. I'd say one can't run a database with any data integrity unless there are disks spinning somewhere very near.
I'd take another box for like $40 there just for the replica, which would sit mostly idle anyways.
But piling on CDNs, clouds and variuos other opex and indeterminate risks instead of some thinking out of this little cloud shoebox is just baffling.
> You can't run a high-throughput database on spinning disks unless you disregard data integrity
Of course you can. That's the way it was and still is done. I'd say one can't run a database with any data integrity unless there are disks spinning somewhere very near.
I'd take another box for like $40 there just for the replica, which would sit mostly idle anyways.
But piling on CDNs, clouds and variuos other opex and indeterminate risks instead of some thinking out of this little cloud shoebox is just baffling.
Spinning disks lose data integrity? You will be surprised where the cloud servers store your backups then. They are doing 80TB/month, you don't need 1GB/s nvme for that(assuming everything is uncached).
Why not? We did that for years if not decades.
How would you define actually capable? I can find servers with recent Xeon processors, good amounts of RAM and NVMe SSD with a gig uplink for around the 100 dollar mark.
These are pretty capable in my experience: https://www.netcup.eu/vserver/
I’m a happy customer of theirs. The UI for managing things is a disaster, but once you get ssh access, it doesn’t really matter.
I’m a happy customer of theirs. The UI for managing things is a disaster, but once you get ssh access, it doesn’t really matter.
Do tell where you found your 1Gbps unlimited bandwidth for $30. Sure it wasn't $300?
[deleted]
> But, we can get around that because of a partnership between Backblaze and Cloudflare they call the Bandwidth Alliance.
This reads like a paid marketing post for Cloudflare.
It's astonishing how many times the author conflates browser cache-able assets with cached assets on a CDN. When a browser downloads a static asset if the web server is configured properly those files will be cached and there won't be a need to re-download them for a very long time.
There is of course the issue with modern web frameworks like React generating a single massive js/css file that bundles everything all over again in a unique file busting all previous cached versions across all users' browsers just because a comma was added to a sentence.
Keep your js/css files small, serve them from your own web server and set a reasonable expire header, no need to pay Cloudflare $40/mo to continue gatekeeping the internet.
This reads like a paid marketing post for Cloudflare.
It's astonishing how many times the author conflates browser cache-able assets with cached assets on a CDN. When a browser downloads a static asset if the web server is configured properly those files will be cached and there won't be a need to re-download them for a very long time.
There is of course the issue with modern web frameworks like React generating a single massive js/css file that bundles everything all over again in a unique file busting all previous cached versions across all users' browsers just because a comma was added to a sentence.
Keep your js/css files small, serve them from your own web server and set a reasonable expire header, no need to pay Cloudflare $40/mo to continue gatekeeping the internet.
They provide 3D asset downloads, most folks will only be downloading an asset a single time. Therefore, browser side caching doesn't really help, they need to cache the files on a CDN.
I didn’t see it as an ad for Cloudflare unless I also then consider every single post about AWS in last 10 years on HN as ads.
If anything, I’m fine with a new competitor to BigCloud that’s been unchecked and increasingly hostile (cost-wise).
If anything, I’m fine with a new competitor to BigCloud that’s been unchecked and increasingly hostile (cost-wise).
Seems like they could pratically host everything on Cloudflare and save even more money.
Asset storage: Backblaze B2 – $11 (replace with Cloudflare R2)
Web hosting: Vercel – $20 (replace with Cloudflare Pages, $0 cost)
Database: Firestore – $100 (replace with Cloudflare)
API: Vultr – $5
Image hosting & optimization: Bunny.net – $27 (replace with Cloudflare)
Domains: Cloudflare – $4
Email fees: MXroute – $3
Asset storage: Backblaze B2 – $11 (replace with Cloudflare R2)
Web hosting: Vercel – $20 (replace with Cloudflare Pages, $0 cost)
Database: Firestore – $100 (replace with Cloudflare)
API: Vultr – $5
Image hosting & optimization: Bunny.net – $27 (replace with Cloudflare)
Domains: Cloudflare – $4
Email fees: MXroute – $3
I don't know if cloudflare image will be cheaper than bunnycdn. We migrated from pro to enterprise cloudflare because the imaging api costs basically put us at the price where upgrading gave us a lot of added benefits and cheaper additional costs during our peak seasons.
> Database: Firestore – $100 (replace with Cloudflare)
Does Cloudflare have a database offering? Do you just mean the Worker KVs, or is there a full relational database?
Does Cloudflare have a database offering? Do you just mean the Worker KVs, or is there a full relational database?
Firestore isn't a full relational DB, so why would a replacement need to be?
Good point. I'd assumed it was a full RMDB, but hadn't ever really spent much time with it.
They dead now? Btw 80TB/mo = 30 MB/s, 5M/mo = 2 views per second, not mind blowing
Do the Bunny images go through Argo?
If you were OK with your content only updating periodically you might be able to do away with Firestore, Argo, and Vultr.
You definitely do a great job of mitigating db hits. Consider publishing all the data to a CDN using something like Gatsby.
In this scenario your CI/CD would build a static website from all the assets and publish it to a static server.
You mentioned that only your view counts are the only really dynamic part. You could just estimate and emulate those or design them a different way.
Martin Fowler has an article on this: https://martinfowler.com/bliki/EditingPublishingSeparation.h...
If you were OK with your content only updating periodically you might be able to do away with Firestore, Argo, and Vultr.
You definitely do a great job of mitigating db hits. Consider publishing all the data to a CDN using something like Gatsby.
In this scenario your CI/CD would build a static website from all the assets and publish it to a static server.
You mentioned that only your view counts are the only really dynamic part. You could just estimate and emulate those or design them a different way.
Martin Fowler has an article on this: https://martinfowler.com/bliki/EditingPublishingSeparation.h...
how is 5M page views a month a lot? I just looked and my tiny old hetzner server serves at least 30M per month. Presumably hetzner gives 1Gbit for free so that should be enough for 80T (0.2Gbit/month, though i dont have that much traffic). It cost $60
I dont understand this entirely .
A lot of the article is spent talking about how great and fast Cloudflare is.
Then at the end he says he uses Bunny as the CDN for images. Why use Bunny instead of Cloudflare? One more moving part.
I presume it is cost related?
Then at the end he says he uses Bunny as the CDN for images. Why use Bunny instead of Cloudflare? One more moving part.
I presume it is cost related?
I too wish they'd spelled out that choice.
I speculate it's because Cloudflare's TOS prohibits you from using their CDN to serve "video or a disproportionate percentage of pictures, audio files, or other non-HTML content" on typical plans. See section 2.8 on https://www.cloudflare.com/terms/ . Since PolyHaven seems to be a purveyor purveyor of 3D assets and they use Bunny to host "all of our images shown on the website (thumbnails, renders, previews, etc.)", I'm guessing a lot of their assets exceed Cloudflare's TOS.
PolyHaven mentions using Bunny's image optimization service, so that'd factor into the decision too.
I speculate it's because Cloudflare's TOS prohibits you from using their CDN to serve "video or a disproportionate percentage of pictures, audio files, or other non-HTML content" on typical plans. See section 2.8 on https://www.cloudflare.com/terms/ . Since PolyHaven seems to be a purveyor purveyor of 3D assets and they use Bunny to host "all of our images shown on the website (thumbnails, renders, previews, etc.)", I'm guessing a lot of their assets exceed Cloudflare's TOS.
PolyHaven mentions using Bunny's image optimization service, so that'd factor into the decision too.
I don't believe Argo is still needed? Tiered Caching is now a free feature.
Yep was also going to mention this. Obviously, Argo still helps with latency and some other things probably, but if they're paying the additional $160 just for tiered caching, they should be able to switch off Argo.
As someone building on a mostly free service with lots of content this is very helpful, thank you! Will probably migrate from S3 to Blackblaze, seems like a great deal.
Depending on how much you access per month, Storj DCS[0] and Wasabi[1] can also be interesting choices.
- Storj has the lowest cost per TB, but charges for egress - Wasabi doesn't charge for egress, but has a fair use policy where if you egress more than you store they can kick you off their platform[2] - Wasabi is also a better fit only if you plan to keep your files around for 90+ days (they have a minimum object retention period) - The bandwidth alliance is only available for HTML related content unless you're on a specific paid plan[3]
[0] https://www.storj.io/ [1] https://wasabi.com/ [2] https://wasabi.com/paygo-pricing-faq/#free-egress-policy [3] https://www.cloudflare.com/terms/#28-limitation-on-serving-n...
*opinion is mine, not my employer's
- Storj has the lowest cost per TB, but charges for egress - Wasabi doesn't charge for egress, but has a fair use policy where if you egress more than you store they can kick you off their platform[2] - Wasabi is also a better fit only if you plan to keep your files around for 90+ days (they have a minimum object retention period) - The bandwidth alliance is only available for HTML related content unless you're on a specific paid plan[3]
[0] https://www.storj.io/ [1] https://wasabi.com/ [2] https://wasabi.com/paygo-pricing-faq/#free-egress-policy [3] https://www.cloudflare.com/terms/#28-limitation-on-serving-n...
*opinion is mine, not my employer's
How are Polyhaven funding their staff? Their Patreon income won't even cover a tenth. Do they have corporate sponsors or special agreements?
Awesome! I hope this means more competitors for YouTube soon, featuring videos that aren't just related to fringe politics.
Self shill: I'm working on https://non.io, a reddit-meets-patreon platform that supports up to 8k video. Planning on launching in the next couple of months (just about got the stripe integration working for auto-payouts). One of my main reasons for making it was I was annoyed at YouTube lately. Would love feedback as I'm aiming to be a competitor.
You can sign in with
Username: [email protected]
Password: hackernews
-
The site is currently in demo mode, and the db will be wiped before launch - feel free to sign in and poke around. Also it's hosted in Australia currently, so site may be a little slow for those in the US.
Source is here: https://github.com/jjcm/soci
You can sign in with
Username: [email protected]
Password: hackernews
-
The site is currently in demo mode, and the db will be wiped before launch - feel free to sign in and poke around. Also it's hosted in Australia currently, so site may be a little slow for those in the US.
Source is here: https://github.com/jjcm/soci
You think cloudflare will be more tolerant?
We have an incredibly similar stack for attic.city, aside from the application layer. We lean heavily on CF (edge caching, Argo and page rules). But we’ve recently changed our image assets from S3 to CF R2. Anyone else put that side by side with Bunny?
Worked on several sites with similar setup. Works great and is affordable. One thing to be aware of is that sometimes you can get at a lot of cache misses. Therefor the back-end services must not stop working completely if flooded with requests.
I like the idea of cloudflare as a OSPF style file cache, but what about putting 80TB in a colo-facility on a 1U box and paying $80/month to host that yourself? is it cheaper to use Amazon S3?
I think you’ve got the 80TB item confused with storage. They are talking about 80 TB of transfer over the internet in a month. I doubt they have 80TB of assets laying around on Vercel. That would cost them an arm and a leg with Vercel I’d expect.
80TB for 5M page views works out to 16.7MiB per page view. I assume you're serving images and other assets on the page otherwise you have room to optimize on the page weight.
Even if you are serving images that's very heavy for a single page. A 4K resolution jpeg sits at around 6-7 MiB.
Question:
I want to make a live video streaming website a la Twitch.tv. How much would CloudFlare charge me to stream 8 Mbps to ~80,000 viewers for 4 hours?
I want to make a live video streaming website a la Twitch.tv. How much would CloudFlare charge me to stream 8 Mbps to ~80,000 viewers for 4 hours?
Almost $20,000.
19,200,000 minutes at $1 per 1,000 minutes. https://support.cloudflare.com/hc/en-us/articles/36001645087...
19,200,000 minutes at $1 per 1,000 minutes. https://support.cloudflare.com/hc/en-us/articles/36001645087...
There's probably some bulk deals if you get in touch with their sales team?
Yes, you could. The last time someone I know spoke to them, the standard enterprise plans started at $5000/mo and got you some very nice freebies. Cloudflare's also announced managed WebRTC APIs (in closed beta) which look promising for creating meeting rooms ala Zoom.us. I imagine, these are already available to tech-shops on enterprise plans.
Though, mux.com is decent too, from what I've heard; for certain workloads, LiveStream and Vimeo might be cheaper.
There's Peer5, PeerTube, and BitTorrent in the P2P space (among many such solutions).
Though, mux.com is decent too, from what I've heard; for certain workloads, LiveStream and Vimeo might be cheaper.
There's Peer5, PeerTube, and BitTorrent in the P2P space (among many such solutions).
> mux.com
First time hearing of them (due to my own ignorance probably). Any details on them?
First time hearing of them (due to my own ignorance probably). Any details on them?
I have never used mux.com (I have used Vimeo, AWS Elemental, and Cloudflare Streams in the past), but they are fairly well-known as the founders created the very popular video.js library. One key thing that differentiates mux.com from others is they're multi-cdn unlike AWS Elemental / Cloudflare Streams, say.
I think this is the sort of thing you want to ask their sales team.
I thought about doing this a while ago, after I left my job with an ISP. I would just have bought transit from them (and maybe made them regret their 10Gbps for $1000/month plan ;)
I thought about doing this a while ago, after I left my job with an ISP. I would just have bought transit from them (and maybe made them regret their 10Gbps for $1000/month plan ;)
I want to pile up: Why is it that bandwidth today is so expensive for things like video? Is there some kind of a good-ol-boys network of companies that are preventing from making bandwidth so cheap its not worth metering? What does the future hold?
I am asking this because YouTube seems to have a massive monopoly both on technology as well as content/audience/network-effects. Even if you can get all the people of Youtube to shift over, you still need to solve the problem of bandwidth and the costs associated with that.
I am asking this because YouTube seems to have a massive monopoly both on technology as well as content/audience/network-effects. Even if you can get all the people of Youtube to shift over, you still need to solve the problem of bandwidth and the costs associated with that.
Bandwidth is only expensive through cloud providers, to try and squeeze you for it. If you go bare metal, dedicated, or just rack your own servers you can peer with a lot of providers and get essentially free bandwidth.
Isn't this like 2 requests per second?
That's massively overpriced, with current prices you should fit into a 5 dollar budget.
That's massively overpriced, with current prices you should fit into a 5 dollar budget.
Uh, so how does cloudflare do it, since the answer is basically "cloudflare".
They have sweet deals with the cloud providers?
They have sweet deals with the cloud providers?
Cloudflare has their own hardware and so pays a "market price" for bandwidth (which is much lower than what clouds (over)charge) and their business model relies on other value-add products rather than just bandwidth. I wouldn't be surprised if they actually lose money on this deal, but I guess it sometimes pays off to provide loss leaders.
Cloudflare subsidizes the small guys in hopes they'll turn into big guys. Bandwidth isn't that expensive in the first place though, that's just a myth perpetuated by AWS/Azure/Google.
Bandwidth Alliance, a consortium of companies that offer free (or steeply discounted) bandwidth between participants.
https://www.cloudflare.com/bandwidth-alliance/
https://www.cloudflare.com/bandwidth-alliance/
They don't have a cloud computing business to put a walled garden around (yet).
Also see their post about it https://blog.cloudflare.com/aws-egregious-egress/
Also see their post about it https://blog.cloudflare.com/aws-egregious-egress/
The biggest costs of server bandwidth are due to transmitting data over large distances (since the internet backbone operates on a sender-pays model when unbalanced traffic is involved in peering). By locating CDN nodes globally, Cloudflare can ensure data is sent near the last mile and thus inexpensive to send. Unless ISPs decide to shake it down the way they do with Netflix of course.
Cloudflare is currently operating at a close to break-even loss.
Sure, because we're hiring like crazy to build new things. As we said during our last earnings call, we intend to stay at break even as long as we can achieve extraordinary revenue growth and continue to deliver innovative new products.
The more relevant number for this comparison would be our gross margin — much we have to spend on things like bandwidth and servers to service our customers divided by the revenue those customers generate — which in Q3 2021 (our last reported quarter) was 78%. Which is… pretty good for a services business like ours.
I don't know the specifics of this customer, but I don't see anything that leads me to believe our margins would be out of the ordinary with them. There are a lot of scale economics in our business. In other words, we can definitely do things for less money because we service a lot of customers than any one customer could hope to do it on their own.
The more relevant number for this comparison would be our gross margin — much we have to spend on things like bandwidth and servers to service our customers divided by the revenue those customers generate — which in Q3 2021 (our last reported quarter) was 78%. Which is… pretty good for a services business like ours.
I don't know the specifics of this customer, but I don't see anything that leads me to believe our margins would be out of the ordinary with them. There are a lot of scale economics in our business. In other words, we can definitely do things for less money because we service a lot of customers than any one customer could hope to do it on their own.
I love HN and it’s always amazing to see the founders answer the questions themselves and most importantly care a lot about the product.
Do you see you offering more dynamic hosting in the future? Things like elastic databases, GPUs etc
Have anything in infrastructure/operations/Linux sysadmin type areas?
It's super nice that you're here and your transparency is appreciated, but don't waste time in people like GP. Just walk your way and don't mind the naysayers.
My two cents from a regular guy to a billionaire, :).
My two cents from a regular guy to a billionaire, :).
>don't mind the naysayers
Noting they are operating at a slight loss, during a period where they want to drive growth, isn't "naysaying". The question is whether the current pricing is tied at all to the "operating at a slight loss". He replied to say it mostly wasn't.
Remember, for example, when Uber rides were really cheap?
Noting they are operating at a slight loss, during a period where they want to drive growth, isn't "naysaying". The question is whether the current pricing is tied at all to the "operating at a slight loss". He replied to say it mostly wasn't.
Remember, for example, when Uber rides were really cheap?
You're right about that but in this case I think they have that covered. Bandwidth is not really that expensivea and they already have the network and scale effects on their favor; so, if anything, I believe their opex (per customer or w/e) will continue to go down.
Bandwidth is cheap. Cloud providers are insanely expensive for bandwidth. Cloudflare is peering in a lot of locations, making it even cheaper for them.
[deleted]
[deleted]
Your blog won't load after waiting for like 10 seconds.
Funny.
Hey look here guys it's cheap to do html !
Well, it's down now.
Foul play?
Foul play?
s/handle/handled
[deleted]
When I tried CloudFlare, they injected a JavaScript into my site that was bigger than my entire site. That was a couple of years ago, but I can't bring myself to trust them anymore. I also don't like their growing control over the web.
[deleted]
It cost less than $2K/Month.
The cloud is crazy expensive. Private servers are beasts, and they are cheap.
Of course, for this price, you don't have redundancy and horizontal scaling.
You also don't have to maintain and debug a system with redundancy and horizontal scaling.