Vaskivo · HackerTrans

Yeah, having even a little bit in the CPU tanks the t/s...

But thanks. I've learned a few more configurations to tinker with.

Vaskivo

2 ay öncediscuss

How can you get it to run at 41 t/s? I also have a single 3090 and even with MTP can't break 20 t/s.

HEre's my setup:

  llama-server
  --port 9999
  --model /MODELS/LLMs/Qwen3.6-27B-UD-Q4_K_XL.gguf
  --ctx-size 128000
  --threads 12
  --flash-attn on
  --device CUDA0
  --jinja
  --gpu-layers 52
  --mmproj /MODELS/LLMs/Qwen3.6-27B-mmproj-F16.gguf
  --cache-type-k q8_0
  --cache-type-v q8_0
  --temp 0.6 --top-k 20 --top-p 0.95 --min-p 0.0 --repeat-penalty 1.0 --presence-penalty 0.0
  --spec-type draft-mtp --spec-draft-n-max 2

(I'm not filling out 100% of the VRAM, as I have other stuff I need it for.)

Vaskivo

7 yıl öncediscuss

Absolutely.

I hate library/playlist oriented stuff. I have everything organized in folders and files, with good filenames. Since I have in my collection a lot of unkown, old or rare media, most programs are not able to find out what it it or even tag them wrong!

I still use foobar. On my kodi setup, I navigate almost everything via file browser. I spent a lot of time finding a decent folder oriented audio player for android.

comments