While I think that local LLMs are the future, i think these setups are insane. You shouldn't be trying to push the SOTA, most people underestimate how much you can get out of small LLMs.
Why ask FABLE 5000 to "summarize this email thread" when a tiny model can do the job?
Sure Codex3000 can oneshot your backlog, but why not use a subsidized subscription to do it for now? We're clearly not at the peak of these model's capabilities yet.
I tried making one, but I couldn't make through the camera position tracking bit, software was super unintuitive. Very interested in gaming applications for this tech, but still waiting for it to be more approachable from a layman's point of view.
So I get the whole performance-per-watt spiel, but if they're targeting the same 10W as with Ice Lake Y-series[1], it's gonna be hot during continuous workloads like screen sharing, since they've seemingly decided to get rid of even the "chassis fan" they've had on 2019 Air.
Why ask FABLE 5000 to "summarize this email thread" when a tiny model can do the job?
Sure Codex3000 can oneshot your backlog, but why not use a subsidized subscription to do it for now? We're clearly not at the peak of these model's capabilities yet.