A lot of AWS services (especially SageMaker) aren’t very good in customer experience. People buy them for nominal capabilities and AWS core bread and butter — short-term and long-term reliability.
Most of these startups (AI and others) have to offer a compelling product before even being notable.
Besides, AWS top level doesn’t care if you use sagemaker or not. There’s a premium but if you’re still using EC2 via another startup, they’re still capturing lions share of value.
This is a guest engineering blog post from Ray contributors at Ant Group, discussing how Ant Group implemented scalable Ray Serving architecture atop Ray, deploying 240,000 cores for model serving, scaling by 3.5x from previous year, and reaching 1.37 million TPS during peak times.
(resubmission, URL redirected to /Engineering last time)
This is a guest engineering blog post from Ray contributors at Ant Group, discussing how Ant Group implemented scalable Ray Serving architecture atop Ray, deploying 240,000 cores for model serving, scaling by 3.5x from previous year, and reaching 1.37 million TPS during peak times.