Ant Group – scaling to 1.37M QPS on Ray(anyscale.com)

3 pointsby george_123قبل 4 سنوات1 comments

https://www.anyscale.com/blog/how-ant-group-uses-ray-to-build-a-large-scale-online-serverless-platform

1 comments

george_123قبل 4 سنوات

This is a guest engineering blog post from Ray contributors at Ant Group, discussing how Ant Group implemented scalable Ray Serving architecture atop Ray, deploying 240,000 cores for model serving, scaling by 3.5x from previous year, and reaching 1.37 million TPS during peak times.

(resubmission, URL redirected to /Engineering last time)