LLM Engine Orchestration for Performance(anyscale.com)

https://www.anyscale.com/blog/ray-serve-faster-first-token-custom-routing