Handling 350k Requests for $3 using Lambda(levelup.gitconnected.com)
levelup.gitconnected.com
Handling 350k Requests for $3 using Lambda
https://levelup.gitconnected.com/handling-350k-requests-for-3-using-lambda-c258ace3da58
85 comments
Yep, that is exactly the architecture that I use to watch over 600k Github repo changelogs for https://changelogs.md
Lambda generates static HTML in the background, puts it in S3, and the static HTML get served via CloudFront
The Lambda costs are a whopping 26 cents per month, for over 2 million Lambda invocations per month. If anyone is interested in the architecture, I've developed this website as an open source project here, for people to learn from: https://github.com/aws-samples/aws-cdk-changelogs-demo
Lambda generates static HTML in the background, puts it in S3, and the static HTML get served via CloudFront
The Lambda costs are a whopping 26 cents per month, for over 2 million Lambda invocations per month. If anyone is interested in the architecture, I've developed this website as an open source project here, for people to learn from: https://github.com/aws-samples/aws-cdk-changelogs-demo
> No Lambda costs as it would be covered by the monthly free tier of 1M requests.
That's far from the full picture. AWS Lambdas are charged by units of computational resources that are expressed as multiples of 64MB of RAM used per 100ms, each rounded up to the next value and with a minimum charge of 128MB of RAM used. So you are only charged a fixed fee per request if all your requests are short-lived and barely use any computational resources. Long-lived processing tasks that require a bit of RAM are charged multiple times the value of a single request.
That's far from the full picture. AWS Lambdas are charged by units of computational resources that are expressed as multiples of 64MB of RAM used per 100ms, each rounded up to the next value and with a minimum charge of 128MB of RAM used. So you are only charged a fixed fee per request if all your requests are short-lived and barely use any computational resources. Long-lived processing tasks that require a bit of RAM are charged multiple times the value of a single request.
You’re right, I should have mentioned that as well.
I didn’t go into those details because I was strictly talking about the project in the article and the compute time limit would not be exceeded for this project either.
400,000 GB-s is free every month, and even if the Lambda function ran for 2,592,000 seconds (equals to a month, way more than enough) using 128 MB of RAM (again, more than enough for a task like this), it would only use 324,000 GB-seconds.
I didn’t go into those details because I was strictly talking about the project in the article and the compute time limit would not be exceeded for this project either.
400,000 GB-s is free every month, and even if the Lambda function ran for 2,592,000 seconds (equals to a month, way more than enough) using 128 MB of RAM (again, more than enough for a task like this), it would only use 324,000 GB-seconds.
> I would simply trigger a Lambda function once a minute (or every X minutes) using CloudWatch to fetch the latest articles and save them to an S3 bucket which I would expose and cache using CloudFront or any other CDN service.
Lot of upsides to this design, and this pretty much outlines a toned-down version of a very large, high-throughtput, low-latency globally distributed configuration system with strict write-ordering but near-realtime write-propagation guarantees a sister team worked on (though, I hear, they're redesigning it for reasons not relevant in this context). There is much to like about it.
Fetching items from S3 (fronted by a CDN or not) would require managing credentials at the client-side, though? Do-able but may require additional code for an auth-service (AWS Cognito or AWS STS or...)?
Lot of upsides to this design, and this pretty much outlines a toned-down version of a very large, high-throughtput, low-latency globally distributed configuration system with strict write-ordering but near-realtime write-propagation guarantees a sister team worked on (though, I hear, they're redesigning it for reasons not relevant in this context). There is much to like about it.
Fetching items from S3 (fronted by a CDN or not) would require managing credentials at the client-side, though? Do-able but may require additional code for an auth-service (AWS Cognito or AWS STS or...)?
There are many different ways to do it.
You can simply whitelist the IP addresses of the CDN (many of them provide them in their documentation or provide an API for it) in your bucket policy. It’s important to schedule a Lambda to run every now and then to check whether there are changes to the IP addresses and update the policy accordingly.
Another way would be to set a custom header with a token on the CDN to be sent in requests to the origin, which you can, again, whitelist in your bucket policy.
You can simply whitelist the IP addresses of the CDN (many of them provide them in their documentation or provide an API for it) in your bucket policy. It’s important to schedule a Lambda to run every now and then to check whether there are changes to the IP addresses and update the policy accordingly.
Another way would be to set a custom header with a token on the CDN to be sent in requests to the origin, which you can, again, whitelist in your bucket policy.
Why use s3 and cache? Why not just cache the output of the lambda directly? Suspicious.
It's like 0.13qps on average. Assuming a constant load and no spikes, a raspberry pi should also be able to handle this load, it would be nice to compare the performance
My Raspiberry PI model 3 would do roughly[1] ~250 req/s with an Elixir server I wrote. 350k @ $3 requests seems kind of expensive to me to be honest.
Is your pi at an AWS data center? How much to co-host it at an AWS-grade data center?
Sure a self-hosted pi is cheaper, but that's an apples vs oranges comparison.
Sure a self-hosted pi is cheaper, but that's an apples vs oranges comparison.
The question wasn't about a PI in a data center was it? It was about how does this compare to a raspberry pi.
I'd even question whether or not the application needs to be in an "AWS-grade datacenter" since it does 1 query every 10 seconds but sure...
I'd be more than willing to bet hertzner's cheapest VPS (~$3.25?) would do over a million[1] requests per month pretty easily and they're inside a data center which is a pretty apples to apples comparison.
My point is that the cost per request is high, regardless of data center or not. That's a lot of money for such a low volume of requests.
[1] Which is still only a request like every 2 seconds.
I'd even question whether or not the application needs to be in an "AWS-grade datacenter" since it does 1 query every 10 seconds but sure...
I'd be more than willing to bet hertzner's cheapest VPS (~$3.25?) would do over a million[1] requests per month pretty easily and they're inside a data center which is a pretty apples to apples comparison.
My point is that the cost per request is high, regardless of data center or not. That's a lot of money for such a low volume of requests.
[1] Which is still only a request like every 2 seconds.
I don't understand why you're getting downvoted. If you're serving simple content, the bottleneck is going to be the network card or disk, every time. In this case, you have static content: a well-written web server (not nginx, but mongoose or similar) can easily handle enough requests to saturate a gigabit uplink on a first gen Raspberry Pi.
Not that it has a gigabit uplink. But handling 100k req/s is trivial for a well-written server.
Not that it has a gigabit uplink. But handling 100k req/s is trivial for a well-written server.
You cannot ignore infrastructure while saying X is cheaper. AWS Lambda is a "batteries included" solution vs a Pi being "some assembly required".
Hertzner is closer, but even then, it's not AWS Lambda. Setup and maintenance of a VPS is not "free".
Hertzner is closer, but even then, it's not AWS Lambda. Setup and maintenance of a VPS is not "free".
What leads you to believe it's an apples-to-oranges comparison? From OP's example the AWS monthly bill would be close to 10% of the total hardware cost to serve the same volume of requests.
How about hardware problems if they happen? Or just rebooting it sometimes to apply kernel patches? Multiple fiber links for a case of bad network weather? DDoS mitigation?
This is what you pay for in the datacenter. The request handling itself is much simpler.
This is what you pay for in the datacenter. The request handling itself is much simpler.
Seems like a better argument for not using FaaS after reading it all.
It's also filled with "once you've done X Y is simple!" like:
> Once you have the template ready, you can simply deploy your code by running: serverless deploy
which is true for all automatable tasks. The same can be said for any VM or shared hosting.
It's also filled with "once you've done X Y is simple!" like:
> Once you have the template ready, you can simply deploy your code by running: serverless deploy
which is true for all automatable tasks. The same can be said for any VM or shared hosting.
That completely misses the point of FaaS.
Devs don't want to manage VMs. Patching, gold loads, SSH configs, k8s, so on. You write code, you deploy, and you pay only when the function executes. It scales as much as you want it to, as much as it needs to, and quicker even than EC2.
There are very clear benefits to FaaS. It is certainly not for everyone, esp. if you already have a well functioning deployment pipeline.
Devs don't want to manage VMs. Patching, gold loads, SSH configs, k8s, so on. You write code, you deploy, and you pay only when the function executes. It scales as much as you want it to, as much as it needs to, and quicker even than EC2.
There are very clear benefits to FaaS. It is certainly not for everyone, esp. if you already have a well functioning deployment pipeline.
> Devs don't want to manage VMs. Patching, gold loads, SSH configs, k8s, so on
Which devs?
I'm a developer and I enjoy most of these things. Working with k8s is a pleasure, managing VMs can be quite simple (and more often than not, is). I don't do much patching, but I "SSH" into our k8s pods quite frequently. I've never thought, gee I wish I had a lambda to simplify this. I know I _also_ don't represent all devs, but I'm never seen this comment made outside of it being an argument supporting FAAS.
Which devs?
I'm a developer and I enjoy most of these things. Working with k8s is a pleasure, managing VMs can be quite simple (and more often than not, is). I don't do much patching, but I "SSH" into our k8s pods quite frequently. I've never thought, gee I wish I had a lambda to simplify this. I know I _also_ don't represent all devs, but I'm never seen this comment made outside of it being an argument supporting FAAS.
K8s can be great but at a largish retailer (billions in yearly revenue) it took a team of engineers on the better end of the company's talent pool to manage the clusters. Those salaries can buy a lot of serverless service usage and according to uptime metrics better reliability. On top of it the teams adopting serverless generally had higher velocity. At scale there were a few use cases that needed to economize but... Largely not.
I would argue "most devs other than you".
I work in the finance industry. Most devs don't seem to know or care about IAM, proper network security, or keeping VMs and container images up to date.
IAM in AWS is by no means "easy" but when you get it right, it's pretty damned good.
I work in the finance industry. Most devs don't seem to know or care about IAM, proper network security, or keeping VMs and container images up to date.
IAM in AWS is by no means "easy" but when you get it right, it's pretty damned good.
I would add that the hard parts of working with Kubernetes is setting up ingress controllers and persistent volumes, but that's precisely the type of problems that any Kubernetes service provider takes care for you.
I think the more important part is the laundry list of frameworks, cloud services, etc. to manage, not the pleasure you draw from working with the services themselves.
If you get minimal returns for managing 5 extra services, it doesn't make sense most of the time.
I do agree though that the cost should not be the thing the poster brags about with this architecture. It's the tradeoff between simplicity, cost, and functionality.
If you get minimal returns for managing 5 extra services, it doesn't make sense most of the time.
I do agree though that the cost should not be the thing the poster brags about with this architecture. It's the tradeoff between simplicity, cost, and functionality.
> Seems like a better argument for not using FaaS after reading it all.
To me the main beneficiaries of all FaaS offerings are actually the service providers themselves, as they can make better use of their existing capacity and at the same time charge a hefty premium for the service. I'm yet to see a single convincing usage of AWS Lambdas that makes economical and technical sense.
To me the main beneficiaries of all FaaS offerings are actually the service providers themselves, as they can make better use of their existing capacity and at the same time charge a hefty premium for the service. I'm yet to see a single convincing usage of AWS Lambdas that makes economical and technical sense.
3$ for 350k request is VERY expensive
Couple points here in this specific case
1) Lambda costs were shown as actually 50 cents for all this. 2) API Gateway was the balance of the cost, and actually isn't needed in this customer's use case. Lambdas can be placed behind application load balancers which are significantly less expensive.
Author also mentions other downsides such as "vendor lock-in," however his lambdas were written in NodeJs and it would be trivial to move his code to some other NodeJs delivery mechanism such as Fargate or ECS cluster.
The benefit here is his deployment with the ServerLess framework got him his MVP very quickly on a very scalable environment with minimal management.
1) Lambda costs were shown as actually 50 cents for all this. 2) API Gateway was the balance of the cost, and actually isn't needed in this customer's use case. Lambdas can be placed behind application load balancers which are significantly less expensive.
Author also mentions other downsides such as "vendor lock-in," however his lambdas were written in NodeJs and it would be trivial to move his code to some other NodeJs delivery mechanism such as Fargate or ECS cluster.
The benefit here is his deployment with the ServerLess framework got him his MVP very quickly on a very scalable environment with minimal management.
API Gateway is the definite gotcha with serverless architectures. Whenever someone tells me how cheaper serverless is, I figure they haven't actually dived in to see how many AWS resources besides Lambda are required for a production deployment.
It is convenient in several use cases, but for an endpoint or even an entire API that is going to receive a lot of load, managing servers is the more economical approach.
If you need to get a one-off endpoint built that won't receive any significant amount of load, I think a serverless solution is great. But I wouldn't use it as my primary API architecture. Like most things in tech, there's no silver bullet. Context matters, and the best approach depends on many factors.
I still recall this article by the creator of ipify on how he's able to receive upwards of 30 billion requests per month on Heroku for a fraction of the cost of API gateway: https://blog.heroku.com/scaling-ipify-to-30-billion-and-beyo...
It is convenient in several use cases, but for an endpoint or even an entire API that is going to receive a lot of load, managing servers is the more economical approach.
If you need to get a one-off endpoint built that won't receive any significant amount of load, I think a serverless solution is great. But I wouldn't use it as my primary API architecture. Like most things in tech, there's no silver bullet. Context matters, and the best approach depends on many factors.
I still recall this article by the creator of ipify on how he's able to receive upwards of 30 billion requests per month on Heroku for a fraction of the cost of API gateway: https://blog.heroku.com/scaling-ipify-to-30-billion-and-beyo...
I run 8 different production API endpoints, all on Lambda, directly through Application Load Balancer instead of API Gateway, and they served 510 million requests last month for about $200.
Those API endpoints took me about a day to deploy using serverless deployment tools. They economically scale between extremely varied workload requirements. There are no complaints on my end.
Those API endpoints took me about a day to deploy using serverless deployment tools. They economically scale between extremely varied workload requirements. There are no complaints on my end.
I don't know how I'm only hearing about this now, since it's been available since late 2018, but that sounds awesome - unlike API Gateway, which is a horrible, buggy mess that doesn't work well for anyone I've spoken to.
> and they served 510 million requests last month for about $200.
In EC2, which is not cheap at all, that's between the monthly cost of one m2.2xlarge and m3.2xlarge instance, or two m2.xlarge instances, or 3 t3.large, or 7 t3.medium.
In other words, you can operae a small cluster on AWS for that price tag.
In EC2, which is not cheap at all, that's between the monthly cost of one m2.2xlarge and m3.2xlarge instance, or two m2.xlarge instances, or 3 t3.large, or 7 t3.medium.
In other words, you can operae a small cluster on AWS for that price tag.
> Author also mentions other downsides such as "vendor lock-in," however his lambdas were written in NodeJs and it would be trivial to move his code to some other NodeJs delivery mechanism such as Fargate or ECS cluster.
I find it amusing that your examples to which one could move the project and avoid “vendor lock-in” are both from the same vendor as Lambda.
I get the point that it’s not Lambda specific outside the entry point, it’s just funny that the presented alternatives are still cutting a check to the same vendor.
I find it amusing that your examples to which one could move the project and avoid “vendor lock-in” are both from the same vendor as Lambda.
I get the point that it’s not Lambda specific outside the entry point, it’s just funny that the presented alternatives are still cutting a check to the same vendor.
Well, the variability in this case is how much system administration you want to do. Right?
Application load balancing is VERY expensive compared to other solutions on the low end. Minimum ALB price is $16/month.
The trade off with API Gateway is around 5 million requests (I wrote an article on it: https://ldoughty.com/2019/12/comparing-aws-api-gateway-to-lo... )
ALB is probably the best solution > 5-6 million and up.. or perhaps HTTP API Gateway between 5-6 and 15 million... But it needs to be near 20 million requests/month before ALB is the "best" solution (though there are still some other considerations that might factor in if your involving a lot of AWS Data transfer which is waived when you are not binding to a VPC, which ALB requires)
The trade off with API Gateway is around 5 million requests (I wrote an article on it: https://ldoughty.com/2019/12/comparing-aws-api-gateway-to-lo... )
ALB is probably the best solution > 5-6 million and up.. or perhaps HTTP API Gateway between 5-6 and 15 million... But it needs to be near 20 million requests/month before ALB is the "best" solution (though there are still some other considerations that might factor in if your involving a lot of AWS Data transfer which is waived when you are not binding to a VPC, which ALB requires)
Seriously. Something like an AWS nano instance only costs $4/month and can easily handle that many requests an hour with a slow dynamic language.
The scary thing I saw is that while it's cool that Lambda is $2/month right now, which is negligible, it's $2/month for 900 users. If it scales linearly (and I don't know that it does, because I don't know exactly how the system works), then that means you're one good blog post somewhere about your extension away from being $200-$2000/month.
That said, my gut says there's a good chance that since the author is surprised at the rate of requests, there's almost certainly an optimization opportunity that wouldn't even take much effort.
That said, my gut says there's a good chance that since the author is surprised at the rate of requests, there's almost certainly an optimization opportunity that wouldn't even take much effort.
Especially the author has experience with docker so deploying a container for his app on Digital Ocean would be super quick and easy. In addition, the list of disadvantages seems longer than the advantages, with scary points like the difficulty to debug things (that could easily takes hours to figure out if something don’t run properly).
However in the case when they both get 0 requests, the Lambda solution costs $0 and the Micro instance still costs $4.
That's a really nice advantage for starting a new project where you're confident in unit economics but have unknown scale.
That's a really nice advantage for starting a new project where you're confident in unit economics but have unknown scale.
Is it two different arguments you're posting? How's $0 on zero requests is a better solution when the range would be 0 to $2000 for what's a fixed $4 without Lambda?
The $0 argument is also flowed because $4 is a fixed cost for unlimited $0 to $2000 solutions through hedging. If you have 10 solutions, what's the likelihood of all of them still doing $0, or at the very least below $4?
The unit economics argument is likewise hard to see in practice, because we'd be talking quite different numbers on a per-unit cost with Lambda vs. without, which may easily make the difference between making a profit or taking a loss, especially if it's a lifestyle business we're talking about.
The $0 argument is also flowed because $4 is a fixed cost for unlimited $0 to $2000 solutions through hedging. If you have 10 solutions, what's the likelihood of all of them still doing $0, or at the very least below $4?
The unit economics argument is likewise hard to see in practice, because we'd be talking quite different numbers on a per-unit cost with Lambda vs. without, which may easily make the difference between making a profit or taking a loss, especially if it's a lifestyle business we're talking about.
Pretty sure you'd have to pay at least $3.5 if you wanted to expose your Lambda function using AWS API Gateway (which I assume most would want to do).. The author mention $1.11 but that's for "HTTP APIs (Preview)" rather than "REST APIs" - in either case it wouldn't be 0. At least that's how I read it.. AWS pricing is such a mess with so many hidden fees (https://aws.amazon.com/api-gateway/pricing/)..
Is there no variable charge for network bandwidth?
Lambda and EC2 have the same network bandwidth charge.
It may be very cheap or very expensive depending on what the requests are doing. There's no point making an overly generalized statement.
>>It may be very cheap or very expensive
It is very expensive. Very. No need to ignore this fact. You can get a VM with 2 vCPU and 2GB of RAM for that amount with some cloud service providers like Hetzner.
It is very expensive. Very. No need to ignore this fact. You can get a VM with 2 vCPU and 2GB of RAM for that amount with some cloud service providers like Hetzner.
And manage it yourself forever.
I’m not saying this example isn’t expensive. But you didn’t post an apples to apples.
I’m not saying this example isn’t expensive. But you didn’t post an apples to apples.
> And manage it yourself forever.
We're not in 1990 anymore. Nowadays all deployments are automatic and ephemeral. I have multiple services up and running with zero downtime for close to a year in a single VM instance that I haven't intervened directly since I launched it, and during that time I lost track of how much time I had to spend fiddling with AWS lambdas and AWS API Gateways.
We're not in 1990 anymore. Nowadays all deployments are automatic and ephemeral. I have multiple services up and running with zero downtime for close to a year in a single VM instance that I haven't intervened directly since I launched it, and during that time I lost track of how much time I had to spend fiddling with AWS lambdas and AWS API Gateways.
No, you can't get any Hetzner VM for $3/month.
The CX11 is 2.49 euro without tax, which is 2.75 USD right now.
> No, you can't get any Hetzner VM for $3/month.
So Hetzner's CX11 instances, that go for €3.01/month with unlimited bandwidth, don't exist in your world?
So Hetzner's CX11 instances, that go for €3.01/month with unlimited bandwidth, don't exist in your world?
That's more than $3. Sure you get unlimited bandwidth and other stuff, but if you don't need it, why waste more money?
My experience with UniProt.org where some requests generate responses dynamically generated gigabytes in size. This is roughly 400x more expensive, assuming response equivalence.
Agree. We consider non-cached requests costly if they extend beyond around $1.50 / million. That said, we do about 2-3% of requests non-cached. As a content network, that means that a million web requests can be as low as $.30 to .40. This does have a lot of benefit to scale, but ~$10 / million seems high. What stops you from putting a ttl on these and caching them for a couple mins. Probably would lower that number substantially.
It would be a fraction of that without API gateway, not sure what it is needed for. I deploy directly to Lambda@Edge and it is an order of magnitude cheaper.
well authhor did list high cost as disadvantage.
If you meant for your comment to be constructive you'll need to expand on it.
Based on my experience here, figuring out how to run your service very cheaply is quite simple:
1. Set up a somewhat convoluted, definitely not efficient, but kind-of cheap setup
2. Write a blog post bragging about it
3. Submit on HN
4. Take notes as HN users inevitably tell you how to run your setup for much cheaper, while even providing double-checked rate estimates (you wouldn't want someone commenting that your estimates were wrong, would you?)
1. Set up a somewhat convoluted, definitely not efficient, but kind-of cheap setup
2. Write a blog post bragging about it
3. Submit on HN
4. Take notes as HN users inevitably tell you how to run your setup for much cheaper, while even providing double-checked rate estimates (you wouldn't want someone commenting that your estimates were wrong, would you?)
It's just a specific implementation of https://meta.m.wikimedia.org/wiki/Cunningham%27s_Law
(This comment is unrelated to the use of lambda, and more of a note to the author)
The author comments that they prefer the light mode to their own dark mode. I think that the dark mode would benefit greatly from being designed (color wise) bottom-up as a dark mode, versus just trying to toggle certain things to be darker.
The big tell is that the lines separating each row of content appear to be the same color in both areas. This tells me that the color of the line is not determined based on the color behind it (either via transparency, via color manipulation manually, or via dynamic color manipulation from a tool like sass (ie, opacity(), darken(), etc...)
I do agree w/ the author that light mode looks better. I also tend to prefer dark mode UIs. I think that by mindfully revisiting color a lot of the visual noise will go away and result in a more pleasant experience.
tl;dr dark mode is not just inverting certain light colors to dark, it should be done with more care.
The author comments that they prefer the light mode to their own dark mode. I think that the dark mode would benefit greatly from being designed (color wise) bottom-up as a dark mode, versus just trying to toggle certain things to be darker.
The big tell is that the lines separating each row of content appear to be the same color in both areas. This tells me that the color of the line is not determined based on the color behind it (either via transparency, via color manipulation manually, or via dynamic color manipulation from a tool like sass (ie, opacity(), darken(), etc...)
I do agree w/ the author that light mode looks better. I also tend to prefer dark mode UIs. I think that by mindfully revisiting color a lot of the visual noise will go away and result in a more pleasant experience.
tl;dr dark mode is not just inverting certain light colors to dark, it should be done with more care.
I have a silent low-power ex-thin-client box sitting in my living room that is used to perform batch jobs. It's been running for about 2.5 years, cost about $26 to buy the hardware second hand, and maybe costs around $20 / yr in electricity. Internet is $0 as I'd pay for that anyway and it doesn't need much bandwidth. It executes about 85,000 batch jobs every fortnight, so call that about 5.5 million jobs total. So that's about 72k jobs to the dollar. Handwaving 1 request as being comparable work to 1 batch job, the lambda setup described here is doing about 117k requests to the dollar. I'm massively over-spending!
The box is currently using about 10% cpu (postgres processes) and has 2 gb of memory available. I guess I over-provisioned and should go looking for some customers to onboard onto my "cloud" to get better utilisation and help amortise my operating costs.
The box is currently using about 10% cpu (postgres processes) and has 2 gb of memory available. I guess I over-provisioned and should go looking for some customers to onboard onto my "cloud" to get better utilisation and help amortise my operating costs.
Cloudflare is $0.50 per million.
$5 minimum though with 128MB RAM and upto 10m reads and 1m writes to their globally distributed KV store, plus free bandwidth and upto 50ms of CPU time (not wall-time). Also, something like API Gateway for routing requests isn't necessary with Cloudflare but options like RateLimiting and Firewall incur additional charges (which, imo, are quite expensive but might be necessary if one hits a certain scale, at which point, one might run their own cluster, instead: It isn't as difficult anymore to setup servers worldwide behind anycast).
IMHO this is poorly researched article. Couple of things mentioned as disadvantages of serverless are already solved in the market.
1. You can use serverless framework or other framework available in market instead of doing everything in GUI
2. Its easy in most cases to invoke locally by using serverless invoke local --function functionName.You don’t have to redeploy.
3. For monitoring you can create different Dashboards on AWS. Also a great solution is integration to Datadog or you can create alerts for specific number of errors or number of invocations and forward it to specific email address.
I dream, that, maybe, sites which have something like "news feed" could export content not only in human-readable HTML/CSS/JS but, maybe, some machine-readable format too?
And such plugins, software products and extensions, as Devo will consume this unambiguous and common format, and will not need custom APIs, APIs keys, backends, HTML scraping & parsing and all custom work for each site they support, but one common way for all sites?
Is it possible on our technological level, or it is wild dream?
OH, WAIT! RSS!
And such plugins, software products and extensions, as Devo will consume this unambiguous and common format, and will not need custom APIs, APIs keys, backends, HTML scraping & parsing and all custom work for each site they support, but one common way for all sites?
Is it possible on our technological level, or it is wild dream?
OH, WAIT! RSS!
If the author is already making an extension, could they have the extension set the security policy for these sites and simply display them in iframes? No cost at all.
They're not actually displaying the websites; they're pulling the top stories via each site's API and rendering them locally.
[deleted]
I could tell you how I have handled millions of requests on a free AWS tier ubuntu system, but its not interesting enough. Just like this...
[deleted]
That’s just 8 requests per minute. Can do that easily on a $2.50 vultr VPS. Beat that :D
What you will do, if you are on vacations (or it is your sleep time, whatever time you prefer to sleep) and there are 100-fold increase in requests die to some reasons?
There are three ways:
(1) You use $2.50 VPS (or Raspbery Pi on your home link, it, really, will be enough for 8 rpm too) and you are Ok that everything collapses and your product/service will be unavailable for long (by current Internet standards) time: from several hours to several days.
(2) You use $40-$100 VPS or dedicated server and pay every month for 8 rpm :-)
(3) You sue something serverless and pay much more than $2.50 only for this surge.
Each way have its own upsides and downsides, and it is impossible to say which is better based only on «8 rpm» information without whole context.
There are three ways:
(1) You use $2.50 VPS (or Raspbery Pi on your home link, it, really, will be enough for 8 rpm too) and you are Ok that everything collapses and your product/service will be unavailable for long (by current Internet standards) time: from several hours to several days.
(2) You use $40-$100 VPS or dedicated server and pay every month for 8 rpm :-)
(3) You sue something serverless and pay much more than $2.50 only for this surge.
Each way have its own upsides and downsides, and it is impossible to say which is better based only on «8 rpm» information without whole context.
Can the extension not make the http requests to get this info? Am I missing the need for a backend for this type of work?
As mentioned in the article, some of the data sources use API keys which the author didn't want to make available to end users.
API keys I believe
Title needs a rate ... 350k requests/month.
Ok: the rest of the article was great. Thanks
Ok: the rest of the article was great. Thanks
[deleted]
Is there anyone but me who uses the premium article on medium.com as a filter for what is NOT worth my time?
I believe in the tech space the majority of readers who write in depth worth to read articles are not incentivised by a few dollars kickback from their hard work from medium.com. They often write it for personal profile enhancement or company blog marketing reasons and want to reach maximum audience and will keep it free for all.
The problem nowadays though is that medium.com is often able to trick them by making the article premium through deceptive ui even when that is not their intention. Just look at the googlecloud medium blog and you constantly see premium articles there which seems bizarre.
I believe in the tech space the majority of readers who write in depth worth to read articles are not incentivised by a few dollars kickback from their hard work from medium.com. They often write it for personal profile enhancement or company blog marketing reasons and want to reach maximum audience and will keep it free for all.
The problem nowadays though is that medium.com is often able to trick them by making the article premium through deceptive ui even when that is not their intention. Just look at the googlecloud medium blog and you constantly see premium articles there which seems bizarre.
I haven't subscribed to Medium but there are certainly some articles/guides I have read there over the years that have been helpful.
> Is there anyone but me who uses the premium article on medium.com as a filter for what is NOT worth my time?
I humbly suggest that your filter might be broken.
After years of being irritated by Medium's pay wall and reflexively clicking away from blockaded articles, I recently decided to pay the $5, because there was some article I really wanted to read (can't even remember what it was), and because I was tired of paying the mental tax of constantly weighing whether to click on a Medium link because I didn't know if the article would be paywalled or not.
I've since read dozens of high-quality, in-depth articles on a variety of technical topics and other areas of interest.
I find it well worth $5/month and I'll probably continue to subscribe.
I humbly suggest that your filter might be broken.
After years of being irritated by Medium's pay wall and reflexively clicking away from blockaded articles, I recently decided to pay the $5, because there was some article I really wanted to read (can't even remember what it was), and because I was tired of paying the mental tax of constantly weighing whether to click on a Medium link because I didn't know if the article would be paywalled or not.
I've since read dozens of high-quality, in-depth articles on a variety of technical topics and other areas of interest.
I find it well worth $5/month and I'll probably continue to subscribe.
readers-> authors
Huh, 350K for $3 is good?
I'm handling Millions for $0 :/ with my https://github.com/amark/gun .
I'm handling Millions for $0 :/ with my https://github.com/amark/gun .
Does that mean you don't run any peer servers yourself? Is that recommended?
Not any that I have to pay for yet! :)
I would simply trigger a Lambda function once a minute (or every X minutes) using CloudWatch to fetch the latest articles and save them to an S3 bucket which I would expose and cache using CloudFront or any other CDN service.
This would lead to:
- No Lambda costs as it would be covered by the monthly free tier of 1M requests.
- No storage costs as the size of the stored data would be extremely small.
- Really fast responses as the “response” would actually be a static file cached at the CDN.
- The only parameter defining your cost would be your CDN of choice, which would cost somewhere between free and as low as $10/TB. For a project like the one in the article, that’s hundreds of millions of requests for just $10.