A serverless architecture for high performance financial modelling(aws.amazon.com)
aws.amazon.com
A serverless architecture for high performance financial modelling
https://aws.amazon.com/blogs/hpc/a-serverless-architecture-for-high-performance-financial-modelling/
40 comments
Personally, I'm moving more and more away from anything AWS. Too many foot guns and its just not worth it.
Yea, as I get better educated and more experienced at my job, I'm thinking more and more that the lessons of the past 5-10 years are less "serverless is the future" an more "the real gains are things like immutability for deployments and reproducibility". As a general rule, I try to reduce as many "moving parts" as possible.
I found migrating to AWS serverless and managed services to not be so bad - that approach has some merit, especially when it comes to minimizing admin overhead. Could you elaborate on your experience?
OP said "personally", and I feel what they mean is, for an individual developer or a small tech shop, AWS makes for a less enticing proposition. I concur. As for me, several reasons:
1. Novelty: It is no longer cool, shiny, hip.
2. Paradox of choice: Just too many things. Though, I prefer AWS Lightsail and do use its services.
3. DevEx: Prefer easier tooling and simpler billing.
4. Realisation: Boring tech works: https://archive.is/sXPN9 (https://twitter.com/jeanqasaur/status/1455589141299675139)
1. Novelty: It is no longer cool, shiny, hip.
2. Paradox of choice: Just too many things. Though, I prefer AWS Lightsail and do use its services.
3. DevEx: Prefer easier tooling and simpler billing.
4. Realisation: Boring tech works: https://archive.is/sXPN9 (https://twitter.com/jeanqasaur/status/1455589141299675139)
I really think it comes down to what you already know, and what you want to do.
Sure, I can run my personal website on a Pi, but then I have to do the hoops to try and serve it off port 80/443 since my ISP blocks it. I could spend an extra $100/month for business class, but why do that if I can get a $5 VM on lightsail that can host half a dozen small WordPress sites and a DB..
And I know S3 and CloudFront, so I can cut the costs lower and serve the static page for pennies...
I'd pay AWS $0.10/month to avoid the hassles of my ISP and Dynamic DNS..
Sure, I can run my personal website on a Pi, but then I have to do the hoops to try and serve it off port 80/443 since my ISP blocks it. I could spend an extra $100/month for business class, but why do that if I can get a $5 VM on lightsail that can host half a dozen small WordPress sites and a DB..
And I know S3 and CloudFront, so I can cut the costs lower and serve the static page for pennies...
I'd pay AWS $0.10/month to avoid the hassles of my ISP and Dynamic DNS..
plot twist:
Static website hosted in Netlify, Github/Gitlab pages or Cloudflare.
No point using AWS - 1000 pound gorilla with dozens of services and features PLUS _unbounded_ billing - if you have only a basic use case such as static page
Static website hosted in Netlify, Github/Gitlab pages or Cloudflare.
No point using AWS - 1000 pound gorilla with dozens of services and features PLUS _unbounded_ billing - if you have only a basic use case such as static page
> PLUS _unbounded_ billing
Disclaimer: I work at AWS, but not involved in billing. The below is my personal experience having dealt with AWS outside of my current role there.
I agree its not worth it if you're just hosting a static page, but if you're using it for other development it can be worthwhile. (I have to admit that Cloudflare/R2 is looking promising for static hosting, although workers seem a bit more limited than lambda. Definitely an interesting ecosystem to keep an eye on)
Even thought it has unbounded billing, in the case of mistakes or unexpected surges, they're often willing to assist and even nix charges. Ultimately, they'd rather you continue using the service and pay future bills than have you quit while they try to collect on a bill you can't pay.
On my personal account, I have billing alerts configured low, and its billed via SEPA from a separate bank account that I use just for cloud providers (N26 makes it easy to configure a separate account with a distinct IBAN). In the unlikely event goes over the amount I have in that account, it'll fail to charge me and I can look into disputing the charges.
Disclaimer: I work at AWS, but not involved in billing. The below is my personal experience having dealt with AWS outside of my current role there.
I agree its not worth it if you're just hosting a static page, but if you're using it for other development it can be worthwhile. (I have to admit that Cloudflare/R2 is looking promising for static hosting, although workers seem a bit more limited than lambda. Definitely an interesting ecosystem to keep an eye on)
Even thought it has unbounded billing, in the case of mistakes or unexpected surges, they're often willing to assist and even nix charges. Ultimately, they'd rather you continue using the service and pay future bills than have you quit while they try to collect on a bill you can't pay.
On my personal account, I have billing alerts configured low, and its billed via SEPA from a separate bank account that I use just for cloud providers (N26 makes it easy to configure a separate account with a distinct IBAN). In the unlikely event goes over the amount I have in that account, it'll fail to charge me and I can look into disputing the charges.
I'm sure billing is not a problem 99% of times
but if aws support decides not to help, or won't clear as much as expected
you'd have a credit to AWS which they could possibly try to collect by other means besides direct debit, such as debt collection
but if aws support decides not to help, or won't clear as much as expected
you'd have a credit to AWS which they could possibly try to collect by other means besides direct debit, such as debt collection
Aren't #1 and #4 at odds? I like that these days it's not cool and hip, which means kind of akin to something like Spring Boot, everything's been done before and there's way less custom fiddling.
The paradox of choice doesn't apply here, you don't have to choose between every service offered. You can pick the one or few that work for you and reuse, reuse, reuse. I find it great for small operations; free tiers for really tiny projects, and the choice of managed services for a higher fee but way less maintenance - especially handy for small businesses who don't have the people to be worrying about architecture every day.
The paradox of choice doesn't apply here, you don't have to choose between every service offered. You can pick the one or few that work for you and reuse, reuse, reuse. I find it great for small operations; free tiers for really tiny projects, and the choice of managed services for a higher fee but way less maintenance - especially handy for small businesses who don't have the people to be worrying about architecture every day.
> Aren't #1 and #4 at odds?
They are at odds (but I allow myself to use a few shiny, new things that I like [0]), but they are also part of the reason I don't use AWS.
> The paradox of choice doesn't apply here, you don't have to choose between every service offered.
Well: https://nitter.net/QuinnyPig/status/1504118950019284995
[0] https://news.ycombinator.com/item?id=9291215
They are at odds (but I allow myself to use a few shiny, new things that I like [0]), but they are also part of the reason I don't use AWS.
> The paradox of choice doesn't apply here, you don't have to choose between every service offered.
Well: https://nitter.net/QuinnyPig/status/1504118950019284995
[0] https://news.ycombinator.com/item?id=9291215
You don't really need AWS for personal weekend type of projects or small startups but for large companies, they seem to be accelerating their dependence on AWS and its not a bad thing imho, it means a standard is emerging and companies are voting with their money.
kubernetes on the other hand....think will be replaced by fargate/ecs, I still struggle to know why startups with a dozen container services need kubernetes. Not having opinionated standard creates so much complexity with endless amount of configurations.
kubernetes on the other hand....think will be replaced by fargate/ecs, I still struggle to know why startups with a dozen container services need kubernetes. Not having opinionated standard creates so much complexity with endless amount of configurations.
AWS: proprietary APIs implemented by one vendor
K8S: many vendors collaborating on a API they all implement
I'd call kubernetes the emerging standard.
K8S: many vendors collaborating on a API they all implement
I'd call kubernetes the emerging standard.
Companies (even small startups) working in regulated spaces can benefit from using the public cloud for the compliance aspect. More than once I've had a regulator ask me for X or Y security certifications, very easy to point to the list that ECS has, much harder to do if running on IaaS or in bare metal.
exactly, yes you have to pay the piper but there are real contractual liability shift away from the operator by going with aws
> While it would certainly have been possible to lift and shift this workload to EC2
Unfortunately I think that the article doesn't make a good case of why this was discarded right off the bat. In particular it is not clear why this wasn't a lift and shift with the NAS storage replaced by S3 and Celery workers placed under an autoscale policy.
Looking at the conclusions the first two are consequences of having on demand compute power while the third one is just surpring (was 70% of the code dedicated to celery infrastructure?).
Unfortunately I think that the article doesn't make a good case of why this was discarded right off the bat. In particular it is not clear why this wasn't a lift and shift with the NAS storage replaced by S3 and Celery workers placed under an autoscale policy.
Looking at the conclusions the first two are consequences of having on demand compute power while the third one is just surpring (was 70% of the code dedicated to celery infrastructure?).
I suppose the authors assumed the reader is familiar with the benefits of the serverless model.
When running functions as a service, you don't have to administer the OS, you just request the computation, provide an input event, and wait for the results to be computed.
If you use EC2 you need to handle the work submission, the scaling and registration of the workers, and the security and life-cycle of the OS yourself.
It seems to me they didn't get all the benefits of serverless, because they had to write thier own custom workload management code. But probably it would still be simpler than managing the workload across thousands of EC2 instances.
When running functions as a service, you don't have to administer the OS, you just request the computation, provide an input event, and wait for the results to be computed.
If you use EC2 you need to handle the work submission, the scaling and registration of the workers, and the security and life-cycle of the OS yourself.
It seems to me they didn't get all the benefits of serverless, because they had to write thier own custom workload management code. But probably it would still be simpler than managing the workload across thousands of EC2 instances.
> When running functions as a service, you don't have to administer the OS, you just request the computation, provide an input event, and wait for the results to be computed.
You don't have to administer any OS when you run them with a container orchestration system as a short-lived job.
It's also perplexing how suddenly "managing an OS" is depicted as such a blocker, specially in a time when treating servers as cattle is the norm.
You don't have to administer any OS when you run them with a container orchestration system as a short-lived job.
It's also perplexing how suddenly "managing an OS" is depicted as such a blocker, specially in a time when treating servers as cattle is the norm.
Yes that’s what serverless is. It’s running compute jobs via a scheduling layer. That’s all it is.
> Yes that’s what serverless is. It’s running compute jobs via a scheduling layer. That’s all it is.
I'm not sure you got the point. The whole point is that you don't need to throw "serverless" buzzord nonsense around to be able to launch long-running processes in the cloud. Al you need is a way to launch processes, and that just happens to be the main responsibility of container orchestration systems, which just so happen to support launching them transparently in your own cluster.
I'm not sure you got the point. The whole point is that you don't need to throw "serverless" buzzord nonsense around to be able to launch long-running processes in the cloud. Al you need is a way to launch processes, and that just happens to be the main responsibility of container orchestration systems, which just so happen to support launching them transparently in your own cluster.
Does anyone know the equivalent of lambda without the 15 mins restriction. Firecracker would fit the bill if it didn't have the bare metal requirements.
Yep you’re looking for https://aws.amazon.com/fargate/
The problem with Fargate is the cold startup time and lack of support for GPUs. It's also quite new and there's all sorts of issues with it but hopefully will get resolved.
I think ECS is a better fit if you require GPU or you can do with not scaling to zero. Obviously once Fargate catches up and polishes its kinks it would stand on its own but given the way Cognito is, I have very little faith that things will improve quickly (theres an open ticket in cognito that is over 3 years old).
I think ECS is a better fit if you require GPU or you can do with not scaling to zero. Obviously once Fargate catches up and polishes its kinks it would stand on its own but given the way Cognito is, I have very little faith that things will improve quickly (theres an open ticket in cognito that is over 3 years old).
Cold start would be just as much of a problem on lambda, no? Which is to say, it would or wouldn't be a problem depending on your needs modulo language/runtime choice. If we expect to compute for >15 minutes, cold start is probably not a big deal.
you get 0.5~1+ second delay when you use containers in lambda, think its worse on Fargate unless somebody can update with real data here recently.
if you dont use containers in lambda, its super fast unless you start using C#/Java
if you dont use containers in lambda, its super fast unless you start using C#/Java
You could use AWS batch. It is like lambda but for more resource intensive uses.
I remember as part of the AWS certification exams this use was explicitly mentioned.
You can trigger it from an s3 upload, for example.
https://aws.amazon.com/batch/
You can trigger it from an s3 upload, for example.
https://aws.amazon.com/batch/
I stood up openfaas for that a few years ago. Set an absurdly high global timeout and something more reasonable for each function. Setup was a twidge finicky (spend a day and rtfm), but I made a couple templates for each language the company used, and people generally had a much easier time deploying via that than anything else we had at the time, even for languages they'd formerly never touched.
Cloud Functions on GCP.
60 minutes for HTTP functions.
10 minutes for event-driven functions.
https://cloud.google.com/functions/quotas
60 minutes for HTTP functions.
10 minutes for event-driven functions.
https://cloud.google.com/functions/quotas
I want to like Google Cloud but so many horror stories I've read from people having their production environment shut down and locked out of their accounts because the over zealous fraud detection found an issue or non-chalant account managers.
Not sure why you would need 60 minutes for HTTP invoked functions, that alone should be a signal to offload it to another queue or asynchronous process.
Not sure why you would need 60 minutes for HTTP invoked functions, that alone should be a signal to offload it to another queue or asynchronous process.
People hear horror stories about Paypal, but it is still massive. I stopped living in fear of everything and decided to adopt stuff that does the job. HN loves to be an echo chamber, especially hating on things. If you only listen to the loud voices, you will end up missing out on things that are actually quite good.
I've used GCP / AppEngine for about a decade now. Built one biz that did $80m gross in its first year. Never even a small threat of being shut down. Recently had a billing issue with GCP and after explaining the problem, they credited me far more than I was even asking for.
I currently have 20k+ servers hitting Cloud Functions (golang) and Cloud SQL 24/7, which is a constant 50 requests/sec... costing me about $100 a month total. It was easy to set up, documentation is well written with clear examples, deployments are all through CI on Github. It just works.
Sorry, but I'm a fan.
To answer your question... https://cloud.google.com/tasks/docs/creating-http-target-tas...
I've used GCP / AppEngine for about a decade now. Built one biz that did $80m gross in its first year. Never even a small threat of being shut down. Recently had a billing issue with GCP and after explaining the problem, they credited me far more than I was even asking for.
I currently have 20k+ servers hitting Cloud Functions (golang) and Cloud SQL 24/7, which is a constant 50 requests/sec... costing me about $100 a month total. It was easy to set up, documentation is well written with clear examples, deployments are all through CI on Github. It just works.
Sorry, but I'm a fan.
To answer your question... https://cloud.google.com/tasks/docs/creating-http-target-tas...
We have a small ETL process (no other ETL in the apps stack so adding something else was silly) that runs on cloud run. It gets a http request to start, puts a lock,status record in the database, forks a worker and returns the id to the http client. The work runs for 10-30 minutes to finish the work and exits and then cloud run shuts us down till the next request.
Like I said we did it this way to avoid adding a proper queue and worker pool to this app so not recommended for heavy traffic but it works
Like I said we did it this way to avoid adding a proper queue and worker pool to this app so not recommended for heavy traffic but it works
Hey have you heard about https://cloud.google.com/run/docs/create-jobs?
Disclaimer I work for Google on Cloud Run.
Disclaimer I work for Google on Cloud Run.
I always find it interesting that GCP employees respond to these sorts of threads with generally helpful comments, but AWS employees rarely respond at all.
Yet at the same time, HN loves to hate on GCP. Keep up the good work @yegle.
Yet at the same time, HN loves to hate on GCP. Keep up the good work @yegle.
Fargate would be the closest.
> Does anyone know the equivalent of lambda without the 15 mins restriction.
You can run short-lived jobs in a container in any container orchestration system or even straight up Docker.
You can run short-lived jobs in a container in any container orchestration system or even straight up Docker.
Or Cloud Run Jobs https://cloud.google.com/run/docs/create-jobs
Google’s Cloud Run with 60 minutes, reasonable cold starts and even always allocated vCPU which are priced much cheaper until request are received to further help with warm starts.
[deleted]
You might check out fly.io if you're looking to do this at the edge. Cloudflare has a similar-looking service in closed beta.
maybe Google App Engine or AWS Beanstalk
unless you're ok to build your own docker images, then fargate already mentioned should be good
unless you're ok to build your own docker images, then fargate already mentioned should be good
I wonder if AWS will ever increase this limit. I would love to know why they haven't yet or why they never will.