Serving 39M Requests for $370/Month(trackchanges.postlight.com)
trackchanges.postlight.com
Serving 39M Requests for $370/Month
https://trackchanges.postlight.com/serving-39-million-requests-for-370-month-or-how-we-reduced-our-hosting-costs-by-two-orders-of-edc30a9a88cd
111 comments
It's apples to apples depending upon the needs of the project and business. From my experience few projects can go 100% serverless. Once you give that up, much of what you talk about is back into your view anyways. Beyond that, cloud vs physical servers again depends upon the requirements of the project and business.
We do a mix of cloud and physical servers, and I'm pushing our company to go more serverless (and cloud), but that could easily change based upon which projects gain traction. Some are high revenue like ecommerce, but others are more ad based and the cloud would probably dig into margins too much. We also aren't a VC backed startup nor are we a huge business so the needs are different. But at our scale, colocating, much less virtual servers, is not even a full time job.
And when I say requirements, I mean actual requirements. Many like to talk about 100% uptime. Then most seem to build their project to only work on a single region of AWS. Single region SLA of AWS is 99.95%. Guess they didn't need that 100% uptime. Of course, multi region is much harder, which is why they do it. But still, I wish people would stop talking about benefits they aren't actually taking advantage of. It makes it difficult to separate the signal from the bullshit.
We do a mix of cloud and physical servers, and I'm pushing our company to go more serverless (and cloud), but that could easily change based upon which projects gain traction. Some are high revenue like ecommerce, but others are more ad based and the cloud would probably dig into margins too much. We also aren't a VC backed startup nor are we a huge business so the needs are different. But at our scale, colocating, much less virtual servers, is not even a full time job.
And when I say requirements, I mean actual requirements. Many like to talk about 100% uptime. Then most seem to build their project to only work on a single region of AWS. Single region SLA of AWS is 99.95%. Guess they didn't need that 100% uptime. Of course, multi region is much harder, which is why they do it. But still, I wish people would stop talking about benefits they aren't actually taking advantage of. It makes it difficult to separate the signal from the bullshit.
> I can honestly say that to discredit the amount of time you spend on solving the above capabilities WELL is to probably toss more than 120% of the cost out the window.
Yes. Where I'm at, I like to believe we solve this reasonably well, and once you factor in all the redundancy for HA, DR & all the engineering around proper monitoring, security, log management... we find Lambda very attractive. Though in our case we tie Lambda back into our infrastructure, orgs that are expediently trying to solve a problem (greenfield projects) and don't have the staff to set all this stuff up, take a deep look at Lambda. At worse, you can always move your lambda functions into your own servers once you have the resources to devote to setting up all the stuff that goes for highly available setups.
Yes. Where I'm at, I like to believe we solve this reasonably well, and once you factor in all the redundancy for HA, DR & all the engineering around proper monitoring, security, log management... we find Lambda very attractive. Though in our case we tie Lambda back into our infrastructure, orgs that are expediently trying to solve a problem (greenfield projects) and don't have the staff to set all this stuff up, take a deep look at Lambda. At worse, you can always move your lambda functions into your own servers once you have the resources to devote to setting up all the stuff that goes for highly available setups.
Just my opinion, but I think API Gateway is massively over priced, considering a t2.nano could easily do the same job, and just slap that in a auto-scaling group to keep it running.
API Gateway doesn't even come close to EC2. I could see maybe 200%, but not like 5000% increase. 39M requests per month is basically nothing.
I'm just guessing, but maybe it's related to the inherent flaw that Lambda only processes with a concurrency of 1 (per func container), instead of utilizing Node / Golang concurrency capabilities.
API Gateway doesn't even come close to EC2. I could see maybe 200%, but not like 5000% increase. 39M requests per month is basically nothing.
I'm just guessing, but maybe it's related to the inherent flaw that Lambda only processes with a concurrency of 1 (per func container), instead of utilizing Node / Golang concurrency capabilities.
The issue is $370 per month is still a lot of money. While you can probably archieve the same performance on $5-10 vm or bare metal server.
well it depends on what you want.
High Availability comes with a cost.
well it's good that more and more vm providers like DO allow you to create your own network. But many still do, so you need some kind of vpn to securly create a network on top.
well it's good that more and more vm providers like DO allow you to create your own network. But many still do, so you need some kind of vpn to securly create a network on top.
I think cost do matter. Look at Wassap. They had only one bare metal server in production until they got acquired. The "Cloud" is sometimes deconnected from the reality.
It's not highly available, in fact Lambda just went down in two regions.
As someone who made one of the comparisons you're commenting on...
> A lot of the comments here are comparing bare metal/virtual servers to a combination of two+ products with literally dozens of built in capabilities, security controls, built in scaling, monitoring, logging, deployment mechanisms, and built in high availability. These are not apples to apples comparisons in the least.
Yes, I acknowledge that buying a pair of dedicated servers will not give me the things you mention however these things are coming at a cost and while they're nice to have, when you're running at a scale where a mere 2 baremetal servers can handle peak load, I'm dubious that you really need these things enough to pay ~2x to have them.
> What now do you do for: high availability, nearly instant scalability, monitoring, logging, alarming, deployment and security controls.
HA: OVH has load balancers too. Logging: Log to disk, rotate logs, compress them and send them to somewhere like S3 for further analysis with a cron job if desired. Monitoring + Alerting: You have competitors like NewRelic and DataDog, they're pretty trivial to set up. Alternatives can be self hosted when things get too expensive. Deployment: Depends on the app. Auto-updates on the servers and a private package repository could be enough. Security Controls: Depends entirely on the application.
Though not everyone needs all these things and AWS adds $200/month in additional expense relative to baremetal, which is a lot of savings to be had.
> Are you going to run all of your software on these 2-3 hosts? What happens when they go down? How quickly are you recovering?
Are you going to run all your software on AWS? What happens when AWS goes down? How quickly is AWS recovering?
You have to deal with these problems either way, AWS is no silver bullet and you're going to need to have disaster recovery plans either way. Most dedicated hosting providers I'm aware of will give you a basic SLA to start with and a better one can be purchased later. That covers hardware issues. Software issues are up to you but from the sound of things, the author's software is little more than a proxy.
> How about patching the OS, the software you run on it, your tools, your languages/packages?
Turn on unattended upgrades, pay a sysadmin a couple hundred bucks once every couple years to handle major upgrades if it's too scary to do yourself.
I'm under no illusions that paying AWS is always the wrong idea but the benefits of AWS come at high cost and not everyone is willing to pay it. I'm certainly not. I pay $60/mo for a colo (let's say $100/mo factoring in hardware) and to get the same capabilities on AWS, I'd be paying $23k for bandwidth, $1.4k for disk and $152 for CPU/RAM (an m3.2xlarge on a 3-year reserved plan), for a total of ~$24500/mo, or 245x what I'm paying now and more than enough to pay 2 full time engineers if I so wished.
> A lot of the comments here are comparing bare metal/virtual servers to a combination of two+ products with literally dozens of built in capabilities, security controls, built in scaling, monitoring, logging, deployment mechanisms, and built in high availability. These are not apples to apples comparisons in the least.
Yes, I acknowledge that buying a pair of dedicated servers will not give me the things you mention however these things are coming at a cost and while they're nice to have, when you're running at a scale where a mere 2 baremetal servers can handle peak load, I'm dubious that you really need these things enough to pay ~2x to have them.
> What now do you do for: high availability, nearly instant scalability, monitoring, logging, alarming, deployment and security controls.
HA: OVH has load balancers too. Logging: Log to disk, rotate logs, compress them and send them to somewhere like S3 for further analysis with a cron job if desired. Monitoring + Alerting: You have competitors like NewRelic and DataDog, they're pretty trivial to set up. Alternatives can be self hosted when things get too expensive. Deployment: Depends on the app. Auto-updates on the servers and a private package repository could be enough. Security Controls: Depends entirely on the application.
Though not everyone needs all these things and AWS adds $200/month in additional expense relative to baremetal, which is a lot of savings to be had.
> Are you going to run all of your software on these 2-3 hosts? What happens when they go down? How quickly are you recovering?
Are you going to run all your software on AWS? What happens when AWS goes down? How quickly is AWS recovering?
You have to deal with these problems either way, AWS is no silver bullet and you're going to need to have disaster recovery plans either way. Most dedicated hosting providers I'm aware of will give you a basic SLA to start with and a better one can be purchased later. That covers hardware issues. Software issues are up to you but from the sound of things, the author's software is little more than a proxy.
> How about patching the OS, the software you run on it, your tools, your languages/packages?
Turn on unattended upgrades, pay a sysadmin a couple hundred bucks once every couple years to handle major upgrades if it's too scary to do yourself.
I'm under no illusions that paying AWS is always the wrong idea but the benefits of AWS come at high cost and not everyone is willing to pay it. I'm certainly not. I pay $60/mo for a colo (let's say $100/mo factoring in hardware) and to get the same capabilities on AWS, I'd be paying $23k for bandwidth, $1.4k for disk and $152 for CPU/RAM (an m3.2xlarge on a 3-year reserved plan), for a total of ~$24500/mo, or 245x what I'm paying now and more than enough to pay 2 full time engineers if I so wished.
> I'm dubious that you really need these things enough to pay ~2x to have them.
Does your business have an accountant? Why not just get a high-school grad - after all, totting up numbers is simple maths, and it's much cheaper.
> You have competitors like NewRelic and DataDog, they're pretty trivial to set up.
Ha! Complain about the price and then say "just install NewRelic". It's pretty clear from that comment alone that you've just got an axe to grind with AWS.
Does your business have an accountant? Why not just get a high-school grad - after all, totting up numbers is simple maths, and it's much cheaper.
> You have competitors like NewRelic and DataDog, they're pretty trivial to set up.
Ha! Complain about the price and then say "just install NewRelic". It's pretty clear from that comment alone that you've just got an axe to grind with AWS.
> Why not just get a high-school grad - after all, totting up numbers is simple maths, and it's much cheaper.
I'd liken AWS more to KPMG, Deloitte, E&Y or PwC than an accountant. You don't need your accountant to have a risk management consultancy any more than you need your server host to be able to provide instant and infinite scalability.
> Ha! Complain about the price and then say "just install NewRelic".
DD and NR are still cheaper than the 2x cost of using AWS and I also suggested self hosting. InfluxDB, Telegraf and Grafana are pretty trivial to set up.
> you've just got an axe to grind with AWS.
I only object to using AWS for everything or blindly. People don't seem to see the costs of using AWS or any of the other big cloud providers.
I'd liken AWS more to KPMG, Deloitte, E&Y or PwC than an accountant. You don't need your accountant to have a risk management consultancy any more than you need your server host to be able to provide instant and infinite scalability.
> Ha! Complain about the price and then say "just install NewRelic".
DD and NR are still cheaper than the 2x cost of using AWS and I also suggested self hosting. InfluxDB, Telegraf and Grafana are pretty trivial to set up.
> you've just got an axe to grind with AWS.
I only object to using AWS for everything or blindly. People don't seem to see the costs of using AWS or any of the other big cloud providers.
> InfluxDB, Telegraf and Grafana are pretty trivial to set up.
No, they're actually not. They're trivial to get to a state of "the process is running on my machine", but to actually get a set of suitable metrics presented the way you want them, that takes a while, especially if you're not already a skilled sysadmin. It's certainly more than "a couple of hours of sysadmin time every couple of years" that others in this thread suggest.
Time and time again I've seen applications painted as "trivial to set up", then when you go to actually try it, the devil is in the details. Getting it running is only the start of it. Unless you have the simplest of use-cases, the stuff that comes out of the box is rarely enough.
Ultimately, this is all penny-pinching nonsense at these numbers anyway. If your shop is bigger than a handful of people, $400/mo is chickenfeed compared to other business expenses.
Your own use-case may indeed not be suitable for AWS - they do gouge you on traffic, and it looks like you're pushing a quarter-petabyte over the wire monthly, so that's certainly not a happy marriage.
No, they're actually not. They're trivial to get to a state of "the process is running on my machine", but to actually get a set of suitable metrics presented the way you want them, that takes a while, especially if you're not already a skilled sysadmin. It's certainly more than "a couple of hours of sysadmin time every couple of years" that others in this thread suggest.
Time and time again I've seen applications painted as "trivial to set up", then when you go to actually try it, the devil is in the details. Getting it running is only the start of it. Unless you have the simplest of use-cases, the stuff that comes out of the box is rarely enough.
Ultimately, this is all penny-pinching nonsense at these numbers anyway. If your shop is bigger than a handful of people, $400/mo is chickenfeed compared to other business expenses.
Your own use-case may indeed not be suitable for AWS - they do gouge you on traffic, and it looks like you're pushing a quarter-petabyte over the wire monthly, so that's certainly not a happy marriage.
> No, they're actually not. They're trivial to get to a state of "the process is running on my machine", but to actually get a set of suitable metrics presented the way you want them, that takes a while, especially if you're not already a skilled sysadmin. It's certainly more than "a couple of hours of sysadmin time every couple of years" that others in this thread suggest.
I said what I said not as speculation but out of experience. It's really not hard. Telegraf includes many of the metrics you'd want built in and others can be sent with minimal effort from most applications with StatsD.
I said what I said not as speculation but out of experience. It's really not hard. Telegraf includes many of the metrics you'd want built in and others can be sent with minimal effort from most applications with StatsD.
At $370/month, you're right: Operating all of these services is definitely costing more in human time than machine time.
I think the confusion stems from the fact that we don't really know how expensive a "request" is. Many applications can do 1000's of requests per second per machine (say 15K/sec).
Extrapolating the 15 requests per second = $370/month in the article, my hypothetical one machine application will cost $370K/month on lambda. For that, I can rent colo space, hire people to run all the services you mentioned, and gold plate the racks while I'm at it. That is crazy overhead for one machine worth of workload.
The GB*sec of a request in my extrapolated workload is certainly wrong, but I suspect a lot of people ran this calculation in their heads.
I don't really know what a request does in this context, except that version N-1 was 100x more expensive than the current one.
I think the right takeaway from the article is that spending some time to optimize workloads that are costing O(one engineer salary) to operate is often worth the effort, and, for low traffic stuff, it is hard to beat one-size-fits-all infrastructure like lambda.
I think the confusion stems from the fact that we don't really know how expensive a "request" is. Many applications can do 1000's of requests per second per machine (say 15K/sec).
Extrapolating the 15 requests per second = $370/month in the article, my hypothetical one machine application will cost $370K/month on lambda. For that, I can rent colo space, hire people to run all the services you mentioned, and gold plate the racks while I'm at it. That is crazy overhead for one machine worth of workload.
The GB*sec of a request in my extrapolated workload is certainly wrong, but I suspect a lot of people ran this calculation in their heads.
I don't really know what a request does in this context, except that version N-1 was 100x more expensive than the current one.
I think the right takeaway from the article is that spending some time to optimize workloads that are costing O(one engineer salary) to operate is often worth the effort, and, for low traffic stuff, it is hard to beat one-size-fits-all infrastructure like lambda.
It's true, "spring cleaning" from time to time can pay back in ways that people don't expect once they clean up months and years worth of additive work on top of something else. I agree that is a big take away.
On per request pricing this is something you can do with Lambda now semi-trivially. You can figure out average Lambda execution cost based on duration of execution and memory you have configured + API-GW related request costs. Depending on your DB you can potentially tie this back as well (harder with some DBs vs. others)
I'd say that while we see cost savings for most serverless platform customers, like an overwhelming amount honestly, we also see companies adopting it for the benefits of time to market, reduced operation burden, and the overall capabilities that they don't need to write themselves. Hard to ROI some of that. While this article did take on a cost based view I'd love to see more folks talk about how these "softer" benefits impact them.
Thanks for your reply!
- Chris
On per request pricing this is something you can do with Lambda now semi-trivially. You can figure out average Lambda execution cost based on duration of execution and memory you have configured + API-GW related request costs. Depending on your DB you can potentially tie this back as well (harder with some DBs vs. others)
I'd say that while we see cost savings for most serverless platform customers, like an overwhelming amount honestly, we also see companies adopting it for the benefits of time to market, reduced operation burden, and the overall capabilities that they don't need to write themselves. Hard to ROI some of that. While this article did take on a cost based view I'd love to see more folks talk about how these "softer" benefits impact them.
Thanks for your reply!
- Chris
[deleted]
> A lot of the comments here are comparing bare metal/virtual servers to a combination of two+ products with literally dozens of built in capabilities, security controls, built in scaling, monitoring, logging, deployment mechanisms, and built in high availability. These are not apples to apples comparisons in the least.
I'm not here to pick a fight against AWS. You guys have built a powerful ecosystem and it's definitely applicable to certain business cases.
Your points about building the same ecosystem as AWS on traditional hardware are absolutely correct, this is very difficult to get right. A lot of places simply don't. I have always said that AWS is great for loads where you need to start be able to easily scale up and down, something which is prohibitively expensive to do with your own servers.
But what you might not see in your position inside Amazon, is that for most companies it's much easier to hire developers/sysadmins for "traditional systems" than it is to find people with deep AWS knowledge.
Because AWS is such a large ecosystem the market of individuals with the domain specific AWS knowledge that your business needs is much smaller than regular old sysadmin/developers.
This means either it's very hard to find the right people, or you have to pay them multiples of what you would pay a standard developer/sysadmin. It's the difficulty of finding people to fill the role that vexes me most.
> What now do you do for: high availability, nearly instant scalability, monitoring, logging, alarming, deployment and security controls.
Ah, but you are assuming people who are using AWS are adhering to best practices and have all these set up, tested, and working in production. I've worked at a few companies with on site and AWS presence, and I can say that none of them followed best practices all the time...
I'm not here to pick a fight against AWS. You guys have built a powerful ecosystem and it's definitely applicable to certain business cases.
Your points about building the same ecosystem as AWS on traditional hardware are absolutely correct, this is very difficult to get right. A lot of places simply don't. I have always said that AWS is great for loads where you need to start be able to easily scale up and down, something which is prohibitively expensive to do with your own servers.
But what you might not see in your position inside Amazon, is that for most companies it's much easier to hire developers/sysadmins for "traditional systems" than it is to find people with deep AWS knowledge.
Because AWS is such a large ecosystem the market of individuals with the domain specific AWS knowledge that your business needs is much smaller than regular old sysadmin/developers.
This means either it's very hard to find the right people, or you have to pay them multiples of what you would pay a standard developer/sysadmin. It's the difficulty of finding people to fill the role that vexes me most.
> What now do you do for: high availability, nearly instant scalability, monitoring, logging, alarming, deployment and security controls.
Ah, but you are assuming people who are using AWS are adhering to best practices and have all these set up, tested, and working in production. I've worked at a few companies with on site and AWS presence, and I can say that none of them followed best practices all the time...
These are all true and valid points. True story: having AWS (or any cloud experience) on your resume today is a major boost to your money earning potential: A. because its rare to have several years experience. B. because the space is growing rapidly (again all of cloud). If you haven't started teaching yourself the basics of cloud computing platforms start today because your next job or the job after that WILL involve you running applications on public cloud.
In my experience from pre-AWS days hiring good Ops people for systems, networking, etc is hard no matter what. I'd argue that the developer's role/day to day changes little in the move from traditional "on-prem" type infrastructure to cloud, even with serverless platforms in play. Generally speaking though I'd aim to hire people who are hungry to learn and have enough fundamental knowledge of networks, operating systems, and hardware to be able to ramp up on public cloud technologies. If you learned it one way, you can learn it this way and there's a massive amount of people here at AWS at least to help you along the way (training folks, support folks, solutions architects) as well as 3rd parties like ACloudGuru, Linux Academy, and others with training/consulting capabilities. The future of technology is moving towards public cloud and I'd argue serverless application development models whether you want to get on board or not. No one cried for the last blacksmith in the town when cars replaced horses.
Best practices are hard for people to follow no matter the technology, and the easiest solution is rarely the "best" (see people leave SG's wide open, open S3 perms, etc). We at AWS know its our job to make security best practices easier to grok and implement and there are tools that help with that today (IAM testing/reviewing tools, trusted advisor as examples). There's more we can do here.
Thanks for your reply!
- Chris
In my experience from pre-AWS days hiring good Ops people for systems, networking, etc is hard no matter what. I'd argue that the developer's role/day to day changes little in the move from traditional "on-prem" type infrastructure to cloud, even with serverless platforms in play. Generally speaking though I'd aim to hire people who are hungry to learn and have enough fundamental knowledge of networks, operating systems, and hardware to be able to ramp up on public cloud technologies. If you learned it one way, you can learn it this way and there's a massive amount of people here at AWS at least to help you along the way (training folks, support folks, solutions architects) as well as 3rd parties like ACloudGuru, Linux Academy, and others with training/consulting capabilities. The future of technology is moving towards public cloud and I'd argue serverless application development models whether you want to get on board or not. No one cried for the last blacksmith in the town when cars replaced horses.
Best practices are hard for people to follow no matter the technology, and the easiest solution is rarely the "best" (see people leave SG's wide open, open S3 perms, etc). We at AWS know its our job to make security best practices easier to grok and implement and there are tools that help with that today (IAM testing/reviewing tools, trusted advisor as examples). There's more we can do here.
Thanks for your reply!
- Chris
> In my experience from pre-AWS days hiring good Ops people for systems, networking, etc is hard no matter what.
No argument here, good talent is hard to find. I guess what I meant is, mediocre talent doesn't hurt you as badly with traditional on prem because there are more guides someone can follow, or there is a lot of institutional knowledge around to help someone in areas they're weaker.
As you correctly pointed out, the cloud is going to be part of our lives going forward. We've already crossed the Rubicon when it comes to the cloud.
Any steps Amazon takes to make AWS configuration easier and more secure out of the box are always welcome.
> open S3 perms
Man it's like every other week we read about some company exposing sensitive data via S3 misconfiguration (NSA contractor data, GOP voter data...). Have you considered selling liability insurance along side it? ;-)
No argument here, good talent is hard to find. I guess what I meant is, mediocre talent doesn't hurt you as badly with traditional on prem because there are more guides someone can follow, or there is a lot of institutional knowledge around to help someone in areas they're weaker.
As you correctly pointed out, the cloud is going to be part of our lives going forward. We've already crossed the Rubicon when it comes to the cloud.
Any steps Amazon takes to make AWS configuration easier and more secure out of the box are always welcome.
> open S3 perms
Man it's like every other week we read about some company exposing sensitive data via S3 misconfiguration (NSA contractor data, GOP voter data...). Have you considered selling liability insurance along side it? ;-)
"Have you considered selling liability insurance along side it? ;-)"
Sounds like an idea for the next YC class!!
-munns
Sounds like an idea for the next YC class!!
-munns
Hmmm... 39M requests per month = 15 requests per second. "20.3MM out of 39MM" of those are hitting a cache, leaving 7.2 requests per second that actually need to be run. The article says requests typically take 2.4s but it's a scraping API and many of those seconds are going to be I/O rather than CPU time. Assuming that there's say 1.0s of CPU time used per request, you should only need an average of ~8 cores/hyperthreads for this.
I suspect you could fit this into a pair of dedicated server s from somewhere like OVH or Hetzner for <$150/month (two servers to provision for peak load, which I'm assuming is ~3x as high as the troughs).
Is there any reason why this wouldn't work?
I suspect you could fit this into a pair of dedicated server s from somewhere like OVH or Hetzner for <$150/month (two servers to provision for peak load, which I'm assuming is ~3x as high as the troughs).
Is there any reason why this wouldn't work?
The transactions are likely very peaky though, so it's difficult to map transactions/month to transactions/second without knowing more. For sure, a straight average isn't useful.
WordPress is notorious for non-cached requests being very heavy as well. Hundreds of PHP files and database queries for a single page isn't uncommon. And the PHP opcode caches suffer from an avalanche problem at cache expiration.
WordPress is notorious for non-cached requests being very heavy as well. Hundreds of PHP files and database queries for a single page isn't uncommon. And the PHP opcode caches suffer from an avalanche problem at cache expiration.
It isn't Serverless, it doesn't use any of the current buzzwords, and you can't run it on the browser side (who doesn't want to run a backend scraping API on the browser), so you'll get zero buy-in from any of the current load of entrepreneurs.
I see only negatives, don't think about doing it that way.
I see only negatives, don't think about doing it that way.
All of your negatives are about the perception of server/serverless, not about the actual benefits. Who knows if this company has investors or if it is actually self funding (judging by the costs, maybe not hard?).
I suspect Aeolun's comment was somewhat sarcastic, no?
While there are benefits, the biggest one the OP pointed out was reduced costs. The GP post was pointing out you could still likely achieve the scale the OP posted about for half of what they're paying on traditional hardware.
While there are benefits, the biggest one the OP pointed out was reduced costs. The GP post was pointing out you could still likely achieve the scale the OP posted about for half of what they're paying on traditional hardware.
This is some budget thinking, but 20 $5 instances, 10 from Vultr and 10 from Linode in 4 different locations and 2 load balancing servers for tops $30 with DDOS protection etc. to send the requests to proper servers would easily handle this and scale without the problem. Would that work? We would be left with $130 easy to scale, huge bandwidth and computing power network.
What would you use for load balancing, where would you put it and how would you run it (say, DNS wise)? It's been a few years since I looked at it, but I couldn't find a cheap, reliable way to load balance a web app between providers.
Tbf, AWS can't do that inter-region well either which is effectively what we are talking about. They rely on DNS-based solutions and so do many CDNs.
Everyone does DNS load balancing unless you're ready to do BGP and announce/withdraw your net blocks based on health.
Anycast is nice, but niche.
Anycast is nice, but niche.
CloudFlare?
I get about 150/requests/sec on a $25/month google cloud instance. Each request queries or updates a mysql database. The database is about $200/month though. Although the database doesn't break a sweat until about 1000/requests/sec
Yeah...my takeaway from this article is that they shouldn't be using AWS Lambda. If you wanted to go through all of caching and load balancing responsibly you could serve their load for <$100/month.
I'm sure it would work, but you wouldn't get a lot of the benefits like a global distributed cache (API Gateway uses Cloudfront as it's cache), and you're obviously dependant on support from a very low cost supplier in the event of a problem with the hardware.
Would it be worth introducing that support dependency for ~$200 a month? Maybe, I guess it depends on what you're making from the service.
Would it be worth introducing that support dependency for ~$200 a month? Maybe, I guess it depends on what you're making from the service.
So using a CDN? I wouldn't list that as a 'benefit' of a cloud-based solution, anyone can use a CDN.
you still have to run the cache from somewhere, and you probably want to be able to deploy during peak load times.
Not sure about the price, to me $75 for a decently beefy server seems low? Compared to the cheap stuff that most people use because most people are only at like 1 or 2 r/s.
Also serverless might sound silly but isn't it nice to not have to do ops works? Surely worth $200/month.
Not sure about the price, to me $75 for a decently beefy server seems low? Compared to the cheap stuff that most people use because most people are only at like 1 or 2 r/s.
Also serverless might sound silly but isn't it nice to not have to do ops works? Surely worth $200/month.
> Not sure about the price, to me $75 for a decently beefy server seems low?
You can argue whether Hetzner is reputable hosting or not, but $75/month will get you a v3 Xeon with 32GB of ECC RAM.
> Also serverless might sound silly but isn't it nice to not have to do ops works? Surely worth $200/month.
Hosting something on Amazon doesn't absolve you of operations work, it's just a different kind of operations. Instead of managing the OS/software updates, you need to manage S3, IAM, CloudWatch, optimise time/memory consumption to reduce cost, etc.
Maybe we just have different definitions of "ops works" but it bothers me when people claim that you don't need to have anyone minding AWS because it's serverless. You still need to have people monitoring, managing permissions, optimising to reduce cost, pushing updates, etc.
You can argue whether Hetzner is reputable hosting or not, but $75/month will get you a v3 Xeon with 32GB of ECC RAM.
> Also serverless might sound silly but isn't it nice to not have to do ops works? Surely worth $200/month.
Hosting something on Amazon doesn't absolve you of operations work, it's just a different kind of operations. Instead of managing the OS/software updates, you need to manage S3, IAM, CloudWatch, optimise time/memory consumption to reduce cost, etc.
Maybe we just have different definitions of "ops works" but it bothers me when people claim that you don't need to have anyone minding AWS because it's serverless. You still need to have people monitoring, managing permissions, optimising to reduce cost, pushing updates, etc.
> Maybe we just have different definitions of "ops works" but it bothers me when people claim that you don't need to have anyone minding AWS because it's serverless. You still need to have people monitoring, managing permissions, optimising to reduce cost, pushing updates, etc
Thank you. Instead of being able to find a server ops guy from a large pool of experienced folks, you now have to find someone specifically with AWS-ops experience - a growing but much smaller pool of people, ime. Getting IAM permissions correct for complex stuff is just not simple, for example, and there's some use cases IAM stuff doesn't (or didn't) cover.
Had a client who couldn't give me EC2 access because - at least at the time - they'd have to give me access to all their EC2 instances, not just the ones I needed. Now... this may have been IAM-inexperience, although, at the time, my digging seemed to indicate it was the correct conclusion.
Thank you. Instead of being able to find a server ops guy from a large pool of experienced folks, you now have to find someone specifically with AWS-ops experience - a growing but much smaller pool of people, ime. Getting IAM permissions correct for complex stuff is just not simple, for example, and there's some use cases IAM stuff doesn't (or didn't) cover.
Had a client who couldn't give me EC2 access because - at least at the time - they'd have to give me access to all their EC2 instances, not just the ones I needed. Now... this may have been IAM-inexperience, although, at the time, my digging seemed to indicate it was the correct conclusion.
I keep recommending people look at places like Hetzner over AWS. People keep wanting to do AWS anyway, and then end up paying 2x-3x as much for the servers and more for the ops work. I shouldn't complain - people who opt for AWS even when it doesn't make sense are what my profit margin is made of.
(I have clients I recommend AWS to as well, but reducing ops time and/or reduced cost are rarely if ever part of those discussions; AWS is the luxury option)
(I have clients I recommend AWS to as well, but reducing ops time and/or reduced cost are rarely if ever part of those discussions; AWS is the luxury option)
out of curiosity: What makes you recommend AWS in those cases?
It would typically be if they have use cases where they either want to do large batch jobs, or have other needs where "add-on" services AWS provides is worthwhile. Often it's a matter of urgency, and AWS will be a good first approximation, and we can start migrating onto a hybrid or fully dedicated environment depending on what's most cost effective down the road when they know more about what they actually need.
Sometimes it's a matter of politics and trst - AWS is trusted, and if their hosting cost doesn't add up to much of their total costs, then that trust is sometimes more worthwhile than squeezing the cost down. E.g. I'm working on a project for a VC now, and their yearly hosting costs are likely to be below what I'm charging them for the initial development of their system, so spending time selling them on a cheaper provider is just not worth it for either them or me.
Sometimes it's a matter of politics and trst - AWS is trusted, and if their hosting cost doesn't add up to much of their total costs, then that trust is sometimes more worthwhile than squeezing the cost down. E.g. I'm working on a project for a VC now, and their yearly hosting costs are likely to be below what I'm charging them for the initial development of their system, so spending time selling them on a cheaper provider is just not worth it for either them or me.
> Instead of being able to find a server ops guy from a large pool of experienced folks, you now have to find someone specifically with AWS-ops experience
Yes, and too many companies don't realize that by moving to AWS, they are not only narrowing the pool of developers/operations people they can hire from, but the salary of these people is higher.
I'm not against AWS, there are absolutely businesses where AWS makes sense (as the article describes). However it's more than just a question of dollars and cents to move to AWS, you also have to factor in the HR consequences of losing access to dime a dozen developers/sysadmins.
For most businesses I have worked with, I would argue losing access to developer/sysadmin talent outweighs their perceived benefits of moving to serverless.
Yes, and too many companies don't realize that by moving to AWS, they are not only narrowing the pool of developers/operations people they can hire from, but the salary of these people is higher.
I'm not against AWS, there are absolutely businesses where AWS makes sense (as the article describes). However it's more than just a question of dollars and cents to move to AWS, you also have to factor in the HR consequences of losing access to dime a dozen developers/sysadmins.
For most businesses I have worked with, I would argue losing access to developer/sysadmin talent outweighs their perceived benefits of moving to serverless.
replying to myself - the client at the bottom had a "move to AWS" mandate from management, they moved, then found out that at least some of their use cases weren't supported. As silly as it sounds, research a tool before you commit to it. Some things are certainly not discoverable until midway in to a project - life happens - but moving ops to AWS in ... 2014, 2015, 2016 - there's enough "known" caveats and gotchas already documented that basic things shouldn't be a surprise. The cost of working around those gotchas often surpasses any first year savings from a migration, but as I've learned - "that's someone else's budget" is sometimes a valid excuse (from some perspectives anyway - as a shareholder, I wouldn't want that justification).
OK well here's the sort of ops work that serverless kinda absolves you of:
- making sure you don't run out of inodes
- having to manage DB backups
- making sure your cache didn't stop working
- updating your box for security updates (beyond just your application requirements)
It's a tradeoff (architecturally as well), and for a lot of stuff you'll end up wanting full-ish control of your box. But for getting the ball rolling it's nice to not have to first spend a bunch of time getting postgres to start consistently on boot.
You still have to actually manage deploys themselves, of course. And keep an eye on costs, too. But a lot of these services are getting close to "the code that you are running locally, but in the cloud". A far cry from "OK, first lets get apache up, then set up some proxy servers, then set up supervisor" etc etc.
- making sure you don't run out of inodes
- having to manage DB backups
- making sure your cache didn't stop working
- updating your box for security updates (beyond just your application requirements)
It's a tradeoff (architecturally as well), and for a lot of stuff you'll end up wanting full-ish control of your box. But for getting the ball rolling it's nice to not have to first spend a bunch of time getting postgres to start consistently on boot.
You still have to actually manage deploys themselves, of course. And keep an eye on costs, too. But a lot of these services are getting close to "the code that you are running locally, but in the cloud". A far cry from "OK, first lets get apache up, then set up some proxy servers, then set up supervisor" etc etc.
> OK well here's the sort of ops work that serverless kinda absolves you of
I wasn't trying to list every bit of ops work for server/serverless. My point is that as a business you cannot expect that once you move to serverless you don't need anyone to manage it anymore, or that the time to manage it drops to zero.
Yes, serverless does have its distinct advantages over rolling your own, there is no question about that. But serverless still requires experienced people to configure/manage/monitor it. I've seen way too many managers get giddy at the thought of having their developer/ops time completely free to perform other tasks, and that's not the case. You still have to budget for people and time to manage serverless architecture.
I wasn't trying to list every bit of ops work for server/serverless. My point is that as a business you cannot expect that once you move to serverless you don't need anyone to manage it anymore, or that the time to manage it drops to zero.
Yes, serverless does have its distinct advantages over rolling your own, there is no question about that. But serverless still requires experienced people to configure/manage/monitor it. I've seen way too many managers get giddy at the thought of having their developer/ops time completely free to perform other tasks, and that's not the case. You still have to budget for people and time to manage serverless architecture.
I use linux on my desktop and have for almost 20 years. Managing a server is a piece of cake for me. AWS is much more difficult for me.
We use Azure, when you deploy it actually goes down, like proper down-down, timeout errors. Long periods of down-time.
That doesn't happen when I deploy on dedicated servers.
It really does depend on your setup and cloud guarantees nothing.
Also, 2.37s for parsing a HTML doc and then making it more readable is awfully slow, assuming it gets sent the HTML doc.
That doesn't happen when I deploy on dedicated servers.
It really does depend on your setup and cloud guarantees nothing.
Also, 2.37s for parsing a HTML doc and then making it more readable is awfully slow, assuming it gets sent the HTML doc.
Do you mean if you do an in-place upgrade? or a VIP swap? A inplace should just go through the update domains so if you have your extra 20% provisioned as normal it should be fine and a VIP should be pretty quick and painless.
Do you mean Azure goes down or your application?
It would be my impression that 1s of cpu time used per request is a very very high value for the given problem. Am I incorrect?
For whatever reason lambda cpu time is much slower then test on your laptop time. Something that takes your laptop 5ms may take lambda 100ms. Some may be oversubscription, some could be cache thrashing...
Takes time to bring the container up to execute.
No, I also suspect it shouldn't use nearly that much CPU time. I just wanted to make sure it was high enough that I wasn't underestimating it.
I guess there will also be some cost reduction if you can skip the server management. You will still have to monitor how your code is doing but no longer have to patch them.
The requests are not uniformly distributed throughout the day, so you can safely triple the calculations you just made.
You realize he actually did take that into account, and tripling his estimate is exactly what he did?
[deleted]
A slightly older article, but here's 40M requests a day for $10/m
http://reviewsignal.com/blog/2014/06/25/40-million-hits-a-da...
http://reviewsignal.com/blog/2014/06/25/40-million-hits-a-da...
This needs an explanation why the requests are so expensive.
39M Requests for $370/Month is an order of magnitude more expensive then the average web application.
I read through the article and it did not give any explanation except that it hints at a big database: "database had grown to store a massive slice of the internet".
Without actual numbers and a description of the workload, it's hard to say if the described solution is good or bad.
39M Requests for $370/Month is an order of magnitude more expensive then the average web application.
I read through the article and it did not give any explanation except that it hints at a big database: "database had grown to store a massive slice of the internet".
Without actual numbers and a description of the workload, it's hard to say if the described solution is good or bad.
Just for context at 39M requests at $370 you are spending ~0.001 cent per request. That's only counting the API. That's not counting their Database costs (which if they are archiving a large part of the internet would be sizable).
For reference I've had times where I was getting ~1000 requests/day/site on a $17/year VM. This was done with Cloudflare caching and caching PHP output.
This comes out to...
That means my request, on a crappy VPS serving Wordpress, some custom static sites, and a few other services (PayPal API handling, emailing, uptime monitoring, etc), cost only 30% as much as the requests described in this post!
PHP-FPM, Lighttpd, and MySQL's small config are magical.
For reference I've had times where I was getting ~1000 requests/day/site on a $17/year VM. This was done with Cloudflare caching and caching PHP output.
This comes out to...
* `ls /var/www/ -1 | wc -l`: 17
* (1000 * 30 days) = 30000req/month/site = 30k * 17/month
* 17 / 12 = $1.5/month
That means my request, on a crappy VPS serving Wordpress, some custom static sites, and a few other services (PayPal API handling, emailing, uptime monitoring, etc), cost only 30% as much as the requests described in this post!
PHP-FPM, Lighttpd, and MySQL's small config are magical.
Did you read the second sentence in the article? You'll get a kick out of it.
>> How We Reduced Our Hosting Costs by Two Orders of Magnitude
So it was costing them that much more, before. Wow.
>> How We Reduced Our Hosting Costs by Two Orders of Magnitude
So it was costing them that much more, before. Wow.
That was the first thing that jumped out at me too. Without sharing the portion of the cost that was from having a massive database and associated expenses (storage, redundancy, backups, disk I/O, maintenance) and then not having one at all on the new setup, the savings come across as disingenuous. I wouldn't be shocked if the DB was at least 2/3 of it.
That title is a little deceptive about scale? 39m requests a month works out to 14/s, then half of that is cached? Why did it cost over $10,000 to begin with? Seems like a simple workload. 2 small vms could handle that
I feel like every article about "X request rate for Y cheap price" ends up being "And then we added caching, and it reduced our load by 80%!".
It's definitely not quite an apples-to-apples comparison. The article mentions the old API used to do things like storing the results in a database which "[...] had grown to store a massive slice of the internet", while the new service appears to be stateless, aside from the cache. Not needing a couple of large database servers alone is going to save a lot of money.
That's not to say Lambda was the wrong choice here, the overall cost is pretty low and it's hard to argue that maintaining their own servers would end up being less than that.
That's not to say Lambda was the wrong choice here, the overall cost is pretty low and it's hard to argue that maintaining their own servers would end up being less than that.
> Rewrite the Readability Parser API
That's at least the third rewrite of the parser. The initial version was a js bookmarklet, I did the rewrite into Python in 2008 which kind of sat around and then got turned into the app by a different group about a year later. Fun to see it go back to JS.
That's at least the third rewrite of the parser. The initial version was a js bookmarklet, I did the rewrite into Python in 2008 which kind of sat around and then got turned into the app by a different group about a year later. Fun to see it go back to JS.
One key takeaway here is that in a typical serverless API setup, the AWS API Gateway will actually end up costing more than the Lambda compute. According to the breakdown in this post the Mercury Web Parser API isn't an exception.
I don't want to be the guy saying that 39 million requests isn't a lot, but especially if your API serves trivial responses to a lot more requests, API Gateway can end up being the opposite of cheap.
I don't want to be the guy saying that 39 million requests isn't a lot, but especially if your API serves trivial responses to a lot more requests, API Gateway can end up being the opposite of cheap.
Lambda itself is already ~300% more expensive than EC2, but in comparison API Gateway is indeed much much more than Lambda for processing the requests.
Agreed that the API Gateway is overpriced, but what is your alternative - build a Lambda that somehow is your API gateway?
There is no truly serverless alternative, although since the API for invoking Lambda functions synchronously is free, you can possibly roll your own API Gateway (assuming you are OK with having servers for that).
But my point was more about simply being cognizant of the API Gateway cost. Too many Serverless tutorials/testimonials assume no significant request volume and end up ignoring this aspect of AWS billing.
But my point was more about simply being cognizant of the API Gateway cost. Too many Serverless tutorials/testimonials assume no significant request volume and end up ignoring this aspect of AWS billing.
The genius of AWS lies in a pricing structure that is easy to get approval for, and where developers once they've started using it are rarely aware of how much it costs...
I see so many cases where people are bewildered at how they've come to spend as much as they do because nobody understands the cost model.
I see so many cases where people are bewildered at how they've come to spend as much as they do because nobody understands the cost model.
This is very much one of those YMMV (quite wildly depending on the project) but one option might be to write the files to S3. You can configure a Lambda to run for each new file written, and then get the result back into a separate bucket.
The low cost is really impressive, especially given that the requests are relatively long-running (2-3s). Color me surprised that their use-case could benefit so much from Lambda's pricing.
Obviously YMMV. Does anybody else have anecdotes about Lambda's pricing helping or hurting their application?
Obviously YMMV. Does anybody else have anecdotes about Lambda's pricing helping or hurting their application?
I wouldn't call it a low price.
Maybe I'm just spoiled because I can put crawl/parse tasks in a work queue for each url without much concern for latency but I've gotten 1000 r/s (on async calls) out of $100/€100 dedicated servers.
Their setup really only makes sense for WAN facing public architecture that can spike because you have no control over the load. Otherwise, I'd suggest you just get some leased dedicated server(s) to meet your workload.
Maybe I'm just spoiled because I can put crawl/parse tasks in a work queue for each url without much concern for latency but I've gotten 1000 r/s (on async calls) out of $100/€100 dedicated servers.
Their setup really only makes sense for WAN facing public architecture that can spike because you have no control over the load. Otherwise, I'd suggest you just get some leased dedicated server(s) to meet your workload.
Maybe if you compare it to existing AWS pricing. Otherwise, these are not low prices.
They mentioned the cost of storing all the articles in a database at the start. I wonder how much of the cost decrease was from removing that?
Many of the comments make a good case for why requests per month is a close to meaningless metric. My blog, hosted on a single Amazon spot instance for about $11/month, serves close to that measure at times.
How big is each request? How peaky is it? Etc.
How big is each request? How peaky is it? Etc.
Is it common to quote requests in requests per month?
I had to read a big chunk of the article before I realized what units were and did the math to get the 15 rps number.
I had to read a big chunk of the article before I realized what units were and did the math to get the 15 rps number.
No it's not you, that really threw me off as well. I've worked on fairly large data mining software and we almost always talked about requests on a per second basis (sometimes requests per day if we were discussing aggregate projects and total output). Using requests per month would be...astronomical. Hell, one product was running 100M requests per day (though to be fair not every request was 1:1 bound to a corresponding database read or write op).
That brings me to my next observation: I don't want to demean the author, but 15 rps is really not extraordinary, so I'm wondering why these requests are so expensive. I assume there is a good reason they need to focus on "scaling" that many, I just can't figure out exactly what it is. They might be doing a lot of very inconsistent throughput with hard to predict peaks, I guess...
That brings me to my next observation: I don't want to demean the author, but 15 rps is really not extraordinary, so I'm wondering why these requests are so expensive. I assume there is a good reason they need to focus on "scaling" that many, I just can't figure out exactly what it is. They might be doing a lot of very inconsistent throughput with hard to predict peaks, I guess...
I'm a huge fan of Lambda, but these prices really need to go down. If anyone thinks API Gateway is cheap, just do the math, this is a trivial amount of traffic for even the smallest of EC2 instances.
Head over to http://serverlesscalc.com/ and enable the "HTTP Requests" radio.
Head over to http://serverlesscalc.com/ and enable the "HTTP Requests" radio.
If you do well, you can get 39M requests per month for less than $30 on App Engine.
It's worth noting, that they could use route53, and parallel deployments to get lower latency as well, with regional scaling. Not sure what this would do to request times, already over a second on average.
On an aside, I do like their Mercury Reader extension.
You find some website that's utterly polluted with content, click the Mercury Reader and you've got a very clean readable article (similar to Firefox's Reading mode).
I keep thinking, it'd be nice to have a Lynx Browser-like proxy that reformats existing webpages into readable, but minimum markup. Sort of like AMP-everywhere perhaps. The purist in me only wants to download the 2kB of actual information from most sites, and not the extra few Meg for tracking, ads, pointless styles, frameworks etc...
You find some website that's utterly polluted with content, click the Mercury Reader and you've got a very clean readable article (similar to Firefox's Reading mode).
I keep thinking, it'd be nice to have a Lynx Browser-like proxy that reformats existing webpages into readable, but minimum markup. Sort of like AMP-everywhere perhaps. The purist in me only wants to download the 2kB of actual information from most sites, and not the extra few Meg for tracking, ads, pointless styles, frameworks etc...
While several other commenters have pointed out how expensive API Gateway/Lambda is. It is highly dependent on your workload.
For example: We have a fairly consistent load of very small, short api requests (Billions a month). These requests can be easily served by a couple of small servers. We calculated API Gateway + Lambda to cost 30 times more than the solution we rolled on Elastic Beanstalk.
We have other services which run for nothing on API Gateway + Lambda. Most of them aren't doing very much though (< 1M requests a month).
For example: We have a fairly consistent load of very small, short api requests (Billions a month). These requests can be easily served by a couple of small servers. We calculated API Gateway + Lambda to cost 30 times more than the solution we rolled on Elastic Beanstalk.
We have other services which run for nothing on API Gateway + Lambda. Most of them aren't doing very much though (< 1M requests a month).
> Our previous costs also included database expenses, which we chose to forego in our serverless setup, opting instead for short-term caching.
Most web applications require a higher degree of persistence.
Most web applications require a higher degree of persistence.
This is actually good use-case for Lambda, majority of other attempts that I see are mostly not great for Lambda. I personally wouldn't use it without really good need and would be very cautious because costs can spike without you having a limit on them with Amazon.
It is good but some more traditional way would also be fine as well. I kind of expected to see Elixir or Go in the backend handling the load more efficiently.
It is good but some more traditional way would also be fine as well. I kind of expected to see Elixir or Go in the backend handling the load more efficiently.
This is 15 requests per second, which should be easily handled with just about anything connected to the internet.
Back in 2014, I worked on an in-memory DB API that handled 1m http requests per hour with ease on a desktop machine with 10G card.
DPDK + DMA aware programming in C++
DPDK + DMA aware programming in C++
>39 Million Requests ... $/Month
39 days/Month
80K sec/day
1M req/day ==> 12 req/s
39 days/Month
80K sec/day
1M req/day ==> 12 req/s
There's not 39 days in a month.
If you can't make $370/month from 39M requests, maybe you shouldn't be making those requests in the first place?
Sounds like they can just well.
They mentioned orders (plural) of magnitude difference in cost. Which means it was closer to $37,000/mo previously.
They mentioned orders (plural) of magnitude difference in cost. Which means it was closer to $37,000/mo previously.
TLDR: amazon serverless (lambda) free tier.
Pretty amazing. Looking at their numbers...
39 million requests X 2.39 seconds per request = 93.21 million seconds of compute time performed each month. That works out at 2.95 years of computation for $370.
Although he mentioned that about half the traffic comes from cached results. So maybe it is more like 1.5 years of actual new computing each month. Compare the $370 against the AWS cost of running a VM for 1.5 years to do the same workload and I think your saving a lot of money.
39 million requests X 2.39 seconds per request = 93.21 million seconds of compute time performed each month. That works out at 2.95 years of computation for $370.
Although he mentioned that about half the traffic comes from cached results. So maybe it is more like 1.5 years of actual new computing each month. Compare the $370 against the AWS cost of running a VM for 1.5 years to do the same workload and I think your saving a lot of money.
It's pretty explicit in the article:
> In the last month, 52% (20.3MM out of 39MM) of our API requests were served from the cache, meaning less than half (18.7MM requests) required invoking our Lambda function. That $14 saves us around $240 a month in Lambda costs.
> In the last month, 52% (20.3MM out of 39MM) of our API requests were served from the cache, meaning less than half (18.7MM requests) required invoking our Lambda function. That $14 saves us around $240 a month in Lambda costs.
It's worth pointing out that AWS allocates CPU shares based on the memory size selected for a function. The CPU performance you'd get with a function running on 256MB would probably be significantly lower than a core of a c4.large instance.
It's unlikely that you'd get cheaper raw compute from any cloud vendor when using Lambda-like products, compared to their IaaS offerings. The savings are in lower ops costs, easier auto-scaling, integration with tons of other AWS products, no minimum cost for services with a small amount of requests, etc.
It's unlikely that you'd get cheaper raw compute from any cloud vendor when using Lambda-like products, compared to their IaaS offerings. The savings are in lower ops costs, easier auto-scaling, integration with tons of other AWS products, no minimum cost for services with a small amount of requests, etc.
> That works out at 2.95 years of computation for $370.
It is 2.95 years of computation on less than 1 CPU core.
http://docs.aws.amazon.com/lambda/latest/dg/lambda-introduct...
> AWS Lambda allocates CPU power proportional to the memory by using the same ratio as a general purpose Amazon EC2 instance type, such as an M3 type. For example, if you allocate 256 MB memory, your Lambda function will receive twice the CPU share than if you allocated only 128 MB.
https://aws.amazon.com/ec2/instance-types/
> m3.large 2 7.5 1 x 32
> Each vCPU is a hyperthread of an Intel Xeon core except for T2 and m3.medium.
1 Core per 7.5GB RAM. Or ~1/28th of 1 core.
(2.95*365.25)/28 = ~39 Core Days.
That is really only impressive when you are comparing AWS prices to AWS prices which is a trap people seem to frequently fall into imo.
It is 2.95 years of computation on less than 1 CPU core.
http://docs.aws.amazon.com/lambda/latest/dg/lambda-introduct...
> AWS Lambda allocates CPU power proportional to the memory by using the same ratio as a general purpose Amazon EC2 instance type, such as an M3 type. For example, if you allocate 256 MB memory, your Lambda function will receive twice the CPU share than if you allocated only 128 MB.
https://aws.amazon.com/ec2/instance-types/
> m3.large 2 7.5 1 x 32
> Each vCPU is a hyperthread of an Intel Xeon core except for T2 and m3.medium.
1 Core per 7.5GB RAM. Or ~1/28th of 1 core.
(2.95*365.25)/28 = ~39 Core Days.
That is really only impressive when you are comparing AWS prices to AWS prices which is a trap people seem to frequently fall into imo.
The computation costs for Lambda were $174, not the full $370.
Using AWS Lambada created that cost structure. Using an alternative to AWS would not.
$150 (at most) dedicated server for instance.
$150 (at most) dedicated server for instance.
We're all assuming content comes in constantly, but that's not the case. I imagine they spike heavily at peak times, perhaps 5-6 fold (judging by my own site statistics). Serverless might be good for them because they can handle small loads with small costs and large loads with no effort.
> We're all assuming content comes in constantly, but that's not the case. I imagine they spike heavily at peak times, perhaps 5-6 fold (judging by my own site statistics). Serverless might be good for them because they can handle small loads with small costs and large loads with no effort.
No. I've pushed 1000 r/s (approx) on a price tracking tool at work for keeping track of competitors on one of those boxes. Their largest spike (even if you assume its 10x their average) would be at most 15% of that.
No. I've pushed 1000 r/s (approx) on a price tracking tool at work for keeping track of competitors on one of those boxes. Their largest spike (even if you assume its 10x their average) would be at most 15% of that.
Wow, I wonder if it's location based. Mine was heavily situated in one location, whereas yours was perhaps global? So it was always prime time for someone? Here is a graph of my usage for the past week (note lows of 30-40 and highs of 650 odd):
https://puu.sh/wrAn1/5816d3f7d7.png
https://puu.sh/wrAn1/5816d3f7d7.png
> Wow, I wonder if it's location based. Mine was heavily situated in one location, whereas yours was perhaps global? So it was always prime time for someone? Here is a graph of my usage for the past week (note lows of 30-40 and highs of 650 odd):
A) I'm not sure "one location" is a city, country, or province.
B) Yes, it was global as we had offices from China to the EU.
C) Given the OP is handling 15 r/s, 150 r/s is not going to be pushing 500 mbps unless something is very poorly designed.
D) Even at spikes of 40x, it would still only be 600 r/s which is still quite doable with a commodity server.
The only serious difference is it was very large batches of requests (20 * SKUs) so people were fine with waiting a few minutes.
A) I'm not sure "one location" is a city, country, or province.
B) Yes, it was global as we had offices from China to the EU.
C) Given the OP is handling 15 r/s, 150 r/s is not going to be pushing 500 mbps unless something is very poorly designed.
D) Even at spikes of 40x, it would still only be 600 r/s which is still quite doable with a commodity server.
The only serious difference is it was very large batches of requests (20 * SKUs) so people were fine with waiting a few minutes.
As an aside (and a noob question), how many requests can a single server handle? I know frameworks such as Node can do handle many more connections when compared to something like Django but what would be a reasonable ballpark?
As a concrete example, say you have an ecommerce app. Read requests take 80ms while write takes 200ms. Any good article/book I can read to get more insight regarding this?
As a concrete example, say you have an ecommerce app. Read requests take 80ms while write takes 200ms. Any good article/book I can read to get more insight regarding this?
I've personally handled hundreds of requests per second using Django behind Nginx on a m3.medium EC2 instance. The trick to scaling like this is to take full advantage of caching, and the cores available to you. With Django (and Node), this means spinning up multiple instances on a single server (one per available core).
With even moderate caching (say, 1m), you can saturate your VM's bandwidth before running out of CPU or memory.
Ultimately, you will always have a bottleneck. All you can do is identify the bottleneck (via load testing) and address it. If it's network (and you're in Amazon), you have to scale vertically until they remove that limit, or horizontally if you're already on unrestricted instances. If it's the CPU, you'll need to be more aggressive at caching completed computations, or work in a language with less overhead. If it's memory, start evaluating the tradeoffs between CPU and memory. If it's your DB, start investigating why it's slow (it's probably a poor index or configuration; DBs on decent hardware can handle millions of requests per second before degrading).
I don't have any book recommendations; I just recommend creating something and find its limits with a simple load test tool. Then figure out your bottleneck and fix that. Rinse and repeat, and learn.
With even moderate caching (say, 1m), you can saturate your VM's bandwidth before running out of CPU or memory.
Ultimately, you will always have a bottleneck. All you can do is identify the bottleneck (via load testing) and address it. If it's network (and you're in Amazon), you have to scale vertically until they remove that limit, or horizontally if you're already on unrestricted instances. If it's the CPU, you'll need to be more aggressive at caching completed computations, or work in a language with less overhead. If it's memory, start evaluating the tradeoffs between CPU and memory. If it's your DB, start investigating why it's slow (it's probably a poor index or configuration; DBs on decent hardware can handle millions of requests per second before degrading).
I don't have any book recommendations; I just recommend creating something and find its limits with a simple load test tool. Then figure out your bottleneck and fix that. Rinse and repeat, and learn.
Its too application centric to be honest to give good generics unless you've built something similar before.
> As a concrete example, say you have an ecommerce app. Read requests take 80ms while write takes 200ms.
An ecommerce site can be nearly 100% cached except for cart/session/search data. This gives it a massive advantage over everything else (since you can load the non-cached data via js and its relatively tiny).
I think the last time I benchmarked one of our front end servers it was ~500 r/s on 2 cores for our search page.
You really aren't going to know until you try it and benchmark it yourself tho.
> As a concrete example, say you have an ecommerce app. Read requests take 80ms while write takes 200ms.
An ecommerce site can be nearly 100% cached except for cart/session/search data. This gives it a massive advantage over everything else (since you can load the non-cached data via js and its relatively tiny).
I think the last time I benchmarked one of our front end servers it was ~500 r/s on 2 cores for our search page.
You really aren't going to know until you try it and benchmark it yourself tho.
A lot of the comments here are comparing bare metal/virtual servers to a combination of two+ products with literally dozens of built in capabilities, security controls, built in scaling, monitoring, logging, deployment mechanisms, and built in high availability. These are not apples to apples comparisons in the least.
Fwiw I've run my own metal at some fairly large websites(can go find me on Linkedin) and I can honestly say that to discredit the amount of time you spend on solving the above capabilities WELL is to probably toss more than 120% of the cost out the window. People time to manage what goes on the hardware is almost always higher than the cost of the hardware. I've spent 5+ years working with everyone from startups up through massive enterprises and only the top 10% does any of this well when doing it themselves.
Taking some of the other examples here, let's say you do try and solve this with 2 virtual/physical servers from a traditional hosting provider. What now do you do for: high availability, nearly instant scalability, monitoring, logging, alarming, deployment and security controls. Those last few all scale on their own too (oh you've never filled a harddrive with logs before?). Are you going to run all of your software on these 2-3 hosts? What happens when they go down? How quickly are you recovering? What impact is there to your customers during this time? What value do you place on that downtime? How many of your tools just failed? Lost data?
How about patching the OS, the software you run on it, your tools, your languages/packages? In busier environments where I ran my own logging, monitoring, metrics, alarming, backups, software, firewalls, etc + the operating systems those things ran on + the modules/packages most developers use these days in languages such as node.js and python, you are talking about whole days in a week to track, test, and understand these changes and the risks associated.
I'm a #serverless zealot by trade sure, and I am happy to be called out on that, but these comparisons of a very high level platform that essentially removes most of the day to day care and feeding of your own "iron" can not be compared to running your own iron at just that iron's cost.