I did mention that in my case I had the 4080, but you do not need the 4080 whatsoever. You can run the same model with less context on a single £200 V100 or with the same amount of context on two of these.
Building computers is very easy. I would suggest watching a YouTube video to get the general gist, and then once you buy the parts, just Google for whatever doesn’t go well.
I built my first Pentium 4 one when I was like six, so I’m sure someone much older that’s into tech can do it without an issue.
There are also tons of Discord communities that are willing to help you live if you encounter any issues.
It thinks this is AI:
“I bought a datacenter GPU that doesn’t even have a normal PCIe connector, stuck it in my gaming PC with an adapter, and now I have 32GB of VRAM across two GPUs running a 27 billion parameter model at 32 tokens per second.”
There’s nothing AI about that. Not all SOTA LLMs agree, hell, none of them do. The same exact example I sent here gives me 0% in some, 10% in others, 100% in GPTzero.
I have written the blog post. I know empirically that I have used 0% AI while writing it.
I also know LLM detectors are total BS and they don't really work. I have tried a couple on this exact blog post, and QuillBot, for example, gave me 0% AI detected on it.
I have then used a blog post of mine from 2021. QuillBot gave me 8%...
The King James version of the Bible came out at almost 100% AI generated a while ago. It was the HN front page.
Stop thinking that if someone writes in a way that is fun or looks like what you would think an AI writes, then it is AI generated. Loads of the time it is, but sometimes it's not, and it really hurts those like me.