HackerTrans
トップ新着トレンドコメント過去質問紹介求人

sorrow17

no profile record

投稿

VLLM: The High-Throughput and Memory-Efficient Serving Engine for LLMs

vllm.ai
1 ポイント·投稿者 sorrow17·6 か月前·0 コメント

コメント

sorrow17
·7 か月前·議論
Yeah, for a lot of users who control the exported source code, rewriting model to use control flow ops, or simply removing the control flow code is a viable option and solvable. For some other users who want to export the model as-is, the option is either using the (deprecated) torchscript, or just move on and use torch.compile and run your model in Python.