akawry·11 か月前·議論Take a look at ik_llama.cpp: https://github.com/ikawrakow/ik_llama.cppCPU performance is much better than mainline llama, as well as having more quantization types available
akawry·12 か月前·議論ik_llama.cpp is a popular fork of mainline llama.cpp started last year containing many new SOTA quants and improved CPU performance.On 07/20, GitHub deleted both the repo as well as the account without explanation. More discussion at https://huggingface.co/ikawrakow/Qwen3-30B-A3B/discussions/2
CPU performance is much better than mainline llama, as well as having more quantization types available