fwiw because of the relatively few activated params offloading to system RAM is quite feasible, you can see the endless amount of people doing this on r/localllama with qwen3.6 35a3b
imo being digital native means that migrating to any machine should be basically trivial. working with the flow of the machines rather than customizing and ricing them because your a cool computer person or whatever
i just want my computer to work. any config i have on my machine can be rebuilt by just doing the work i need to do.
my primary work machine was stolen last year so i was forced to go through this quite literally with a new machine rather than hypothetically or by my own will
im more surprised that more people don’t treat their computer as disposable anyway.
that it could just be wiped at any moment and it wouldn’t matter. shit happens, could be stolen, broken, whatever. the computer should be able to be thrown out the window and continue to live life.
to be clear, i don’t think upgrading and disposable in this way is good, but it being wiped at any moment shouldn’t be a concern
i grew up wiping my machine every year anyway, so i guess it’s just a habit
Wondering similar. It certainly can run beyond 30 seconds but at some point I believe the output should degrade
Plus you could do actual batch inference instead. Or if you must carry forward the context you could still do it linearly, but the mem usage shouldn’t just explode
I just don't have the bandwidth to run another project, maintaining Handy is hard enough on it's own, especially for free!
I didn't just dismiss for no reason, I am a human! I have needs and I can't just sleeplessly stay in front of the computer putting out code. If I had more time I would, but alas.
Someone could easily vibe code an iOS version in a few hours. I could do the same but I do not have time to support it.
I don’t think it’s about literally shrinking the models via quantization, but rather training smaller/more efficient models from scratch
Smaller models have gotten much more powerful the last 2 years. Qwen 3.5 is one example of this. The cost/compute requirements of running the same level intelligence is going down
That wasn’t the point. You’re a person who runs arch, that means most likely your requirements for a computer are VERY different than the target for this Mac. There’s always some other computer you can buy, but most people will just buy the Mac