Since no one else posted it... I have open-webui pointed at a linux box with 128 gig of ram and an RTX Pro 6000, and after a couple of runs on trivia, had it do one of Open WebUI's conversation starters: "Show me a code snippet of a website's sticky header in CSS and JavaScript."
72.06 t/s. That's the full Qwen 3.6 27B model BF16, using MTP, running on Ollama. Yes I know I should bite the bullet and get vllm running on that box.
That was, also, at a 570 watt limit: I normally run a little less, but when I first tried this I actually forgot I had set the limit to 300 (it's a hot day, I figured why fight the A/C?), and at 300 watts the same question came back at 69.38 t/s. (The extra power matters more for compute bound things, the difference in generating LTX2.3 videos is considerably higher... but still not linear.)
Not a shareholder, but on first try, it won't do it because it recognizes Iger's name. And clearly the deal is fresh because it balked at Mickey Mouse too. But it has no trouble with just, "mouse": https://sora.chatgpt.com/p/s_693ae0d25bbc819188f6758fce3f90c...
It's likely that I'm seeing this from my deep into ComfyUI bubble. My impression was that AUTOMATIC1111 and Forge and the like, were fading as ComfyUI was the "what people ended up on" no matter which AI generation framework they started with. But I don't know that there are any real stats on usage of these programs, so it's entirely possible that AUTOMATIC1111/Forge/InvokeAI are being used by more people than ComfyUI.
Did you test some local image gen software in that you installed the Python code on the github page for a local model, which is clearly a LOT for a normal user... or did you look at ComfyUI, which is how most people are running local video and image models? There are "just install this" versions, which eases the path for users (but it's still, admittedly, chaos beneath the surface).
72.06 t/s. That's the full Qwen 3.6 27B model BF16, using MTP, running on Ollama. Yes I know I should bite the bullet and get vllm running on that box.
That was, also, at a 570 watt limit: I normally run a little less, but when I first tried this I actually forgot I had set the limit to 300 (it's a hot day, I figured why fight the A/C?), and at 300 watts the same question came back at 69.38 t/s. (The extra power matters more for compute bound things, the difference in generating LTX2.3 videos is considerably higher... but still not linear.)