With the recent release of Qwen-3 Omni I've decided to put together my first local machine. As much as I just want to pick up a beelink and flash it with Omarchy I think I want a bit more horsepower.
However, the internet seems littered with "clever" loca ai monstrosities that gang together 4-6 ancient nVidia GPU's (priced today to seem like overpriced e-waste) to get lackluster performance from piles of nVidia m60's and P100's? In 2025 this kind of seems like a waste or just bad advice to use hardware this old?
Curious if this find seems like a good source of info regarding staying away from Intel and AMD GPU's for local inference? Might do some training but right now more interested in light RAG and maybe some local coding.
Hoping to build something before the holiday season to keep my office warm with GPU's :).
Hi, I'm looking to transition from renting GPU's from RunPod to hosting some models locally - specifically qwen-2.5 and some lightweight VLM's like Moondream. It looks like the RTX 3060 12gb is a relatively good option but I don't necessarily have a lot of experience with pc hardware, let alone used hardware.
Curious if anyone here has a similar config of 1-4 RTX 3060s? Trying to decide if picking up a few of these is a good value or if I should just continue renting cloud GPU's?
I've been toying with the idea of building a dedicated machine to continue working on my FPV flight control model.
So far I've just been renting gpu time but it's getting expensive. I also occasionally use qwen3-coder in my homelab but my headless "inference" machine is quite limited with an RTX 3080.
Curious if this build from llamabuilds.ai looks good? In terms of gpu etc? I'm maybe considering buying more GPUs in the future - hard to tell if stacking RTX 6000 ADA's is worthwhile!
I think this is going to change really fast - not in favor of ai companies or investors.
Initially, ChatGPT was kind of a cool toy that felt like something new but not really a threat. Now, even people who aren't engineers see the threat this has to destroy the dignity and financial stability of millions of people.
I like Ai - but at the same time I hope our country picks people over corporate profits... unfortunately I don't see that being likely.
I honestly don't know what I would do if I was fresh out of school today or over 40.
That said, grinding through middling startup jobs also sucks.
The ability to get vc attention seems incredibly cliquey - not a game I really want to play again. On top of just wanting a normal / decent salary as a founder.
My favorite elixir project of all time now written in Rust.
Love to see it.
Have you guys added the ability to cleanse / watch command strings for sensitive items like secrets, keys etc? Seems easier than ever with advances in lightweight LLMs.
My guess is they don't really know how to price it in yet - but also they seem to massively reduce headcount in areas where you can loftily make assumptions that "ai is boosting productivity"
Seems like hand waving and layoffs will have to stop before we get real data
Any idea what triggered this in Southeast Asia? I think Vietnam just did the same thing with over 80M accounts. Seems like the IMF accused them all of harboring mass money laundering or offshoring schemes?