LLM Microserving: a new RISC-style approach to design LLM serving API(blog.mlc.ai)
blog.mlc.ai
LLM Microserving: a new RISC-style approach to design LLM serving API
https://blog.mlc.ai/2025/01/07/microserving-llm-engines
1 comments
Scale LLM serving with programmable cross-engine serving patterns, all in a few lines of Python