should run perfect for 12-16gb with maybe 10-20k context
seems intelligent enough that I would recommend this as a daily driver for friends who just want a local ai that can do most things relatively quickly (getting 10 tps on my m2 air)
the HITL (human in the loop) is basically the single point...AI is a mirror..
it only "exists" when you talk to it.. much like your reflection in the mirror is only there when you're in view.
models can never be self-improving because it can never have "self". it can only mirror the appearance of self.
what's actually happening is "symbiotic group improvement".
our brains are resonant.. for those of use who are brilliant, getting leverage with ai just means that our innovative ideas become louder and more physically real every day.
eventually everything worth building will be built for free and made readily available.. no more "profiteering"
good, the point is that now you have free mental bandwidth to use on building something that truly interests you using AI to help actualize your goal. build something cool for your kiddos idk?
at this point I trust software companies less and less..
being able to build the stack and create bespoke solutions with llms's is incredible..
idk why people get mad about vibe-coding. if ur little brother can make a Spotify clone with Claude in a day, shouldn't that mean that you as a dev should be able to create something 100x better that makes Spotify obsolete?
good ideas / feasible novel architecture design will be the only thing valuable..
I run the latest 20b-30b models on a MacBook Air... running inference with an MoE (25 tps) for like 2 hours is like 10% battery.. (look me up on huggingface to download my models)
also you gotta realize frontier models have massive "system prompts" that clog up the context window with garbage.
being able to write your own system prompts gives you a MASSIVE edge..