LLM Inference with Ray: Expert parallelism and prefill/decode disaggregation(anyscale.com)
anyscale.com
LLM Inference with Ray: Expert parallelism and prefill/decode disaggregation
https://www.anyscale.com/blog/ray-serve-llm-anyscale-apis-wide-ep-disaggregated-serving-vllm
0 comments
—