CUDA-Free Inference for LLMs(pytorch.org)

https://pytorch.org/blog/cuda-free-inference-for-llms/