High Performance Distributed Inference with Ray Serve LLM(anyscale.com)

https://www.anyscale.com/blog/high-performance-distributed-inference-ray-serve-llm-vllm-google-kubernetes-gke