Mostlygeek·vor 8 Monaten·discussqwen3-coder-30B-A3B supports FIM and should be faster than the 7B if you got the vram.I use bartowkski’s Q8 quant over dual 3090s and it gets up to 100tok/sec. The Q4 quant on a single 3090 is very fast and decently smart.
Mostlygeek·letztes Jahr·discussI loved my pebble and still sad it died. Please use inductive charging. The metal connectors on my last one shorted when I was in a salt water pool.
I use bartowkski’s Q8 quant over dual 3090s and it gets up to 100tok/sec. The Q4 quant on a single 3090 is very fast and decently smart.