llama.cpp to get 115 tok/s on RTX 4090 with Qwen3.6-27B. For example in Windows the latest CUDA variant llama-b9678-bin-win-cuda-13.3-x64.zip and Unsloth UD-Q4_K_XL MTP gguf:
Note that this does not use kv cache quants as in my case quants offload to CPU and tanks performance. Also keep in mind this almost maxes VRAM usage so any additional browsers or other programs that use VRAM should be closed.
For chat go to http://localhost:8080/ and minimize the window to maximize perf as the web page UI draw itself consumes a lot of GPU perf via constant context switching.
Can try bigger than -c 75000 until perf gets lower than 100 tok/s - that means something is off as windows starts paging out memory or other issues. -c 50000 seems sweetspot if running browsers and stuff that consume 2GB VRAM. If wanting more than -c 140000 then likely need to use a bit smaller model quant.
CPU usage should be near zero, maybe 1 core load. If you see 8+ core load then settings are off and something is offloaded to CPU (for example kv cache). GPU load should be about 100%, meaning it utilizes work optimally in this case.
-t 16 can be omitted or set to the amount of physical cores, not important in this dense model that is 100% in GPU.
Can be pushed to 125 tok/s with that model if using --spec-draft-n-max 4 but VRAM usage also increases, so context needs to be smaller.
If speed is not important and want max context length then remove the draft-mtp parameters and also might need to use k and v quants like --cache-type-v q8_0, leave k f16 if possible to keep quality.
Yes, disassembling and tinkering with microwave is almost the only easily available thing I would not recommend at all because it seems so ordinary but actual danger is way too high and hidden. Even if you know exactly the dangers, it takes only one distracted move, just being tired or too complacent to end really badly. Like even some medium time disconnected and expectedly safe units can have real nasty shock in them when discharge resistor is faulty, that failure mode does not usually kill but not recommended.
For sure seems they are likely deliberately riding on the fame of the movie. I too instantly thought it is some kind of Flow movie animation collaboration similarily like Flow is represented in Blender 4.4 splash screen or is even their mascot.
Interesting what power do microwaves in US deliver. In EU the regular micros are about 900W range give or take 200W (that is 1500W from wall because microwaves are not efficient machines) but kettles have usually range 2200..3000.
So it is obvious why everyone uses kettles. I myselft have 3kW one and it heats 0.35 litres to boiling in 51 seconds. To have similar microwave it would have to be 3000W micro that is 4500W from wall and have not seen that kind of monsters, also regular 1 phase EU socket delivers max 3600W so that limits it also.
llama-server.exe --host 0.0.0.0 --alias "Qwen3.6-27B-MTP" -m "F:\Qwen3.6-27B-UD-Q4_K_XL-MTP.gguf" -c 75000 -ngl 99 --metrics --temp 0.6 --top-p 0.95 --min-p 0.00 --top-k 20 --presence-penalty 0.0 --no-mmap -t 16 --spec-type draft-mtp --spec-draft-n-max 3 --reasoning on -fa on --parallel 1 -lv 4
Note that this does not use kv cache quants as in my case quants offload to CPU and tanks performance. Also keep in mind this almost maxes VRAM usage so any additional browsers or other programs that use VRAM should be closed. For chat go to http://localhost:8080/ and minimize the window to maximize perf as the web page UI draw itself consumes a lot of GPU perf via constant context switching.
Can try bigger than -c 75000 until perf gets lower than 100 tok/s - that means something is off as windows starts paging out memory or other issues. -c 50000 seems sweetspot if running browsers and stuff that consume 2GB VRAM. If wanting more than -c 140000 then likely need to use a bit smaller model quant.
CPU usage should be near zero, maybe 1 core load. If you see 8+ core load then settings are off and something is offloaded to CPU (for example kv cache). GPU load should be about 100%, meaning it utilizes work optimally in this case.
-t 16 can be omitted or set to the amount of physical cores, not important in this dense model that is 100% in GPU.
Can be pushed to 125 tok/s with that model if using --spec-draft-n-max 4 but VRAM usage also increases, so context needs to be smaller.
If speed is not important and want max context length then remove the draft-mtp parameters and also might need to use k and v quants like --cache-type-v q8_0, leave k f16 if possible to keep quality.