cybertim · HackerTrans

I own a dedicated 48vCPU with even 160GB RAM.. its not that expensive, check ebay, maybe now with mem prices it will be a bit more steep but as a hobby it's not crazy to think one owns such a piece of hardware. My dual GPU setup was more expensive I think.

cybertim

19 giorni fadiscuss

I bought two RTX3080s with 20GB during my holiday in china (set me back 700euros) I'm getting 800-1000 input tps and 60-100tps output with Qwen 3.6 27b Q8 (MTP, P2P, 200k context) this feels like opus4.5 level while coding (pi harness). Also easy to just host your own openai compatible api from home this way and still use your MacBook as dev station.

cybertim

29 giorni fadiscuss

Ok, i went through the P2P rabbithole, and incase anyone reads this, just use Fedora Rawhide, and run that repo install.sh mentioned in the blog post :) Now i have:

  GPU0 GPU1

 GPU0 X OK

 GPU1 OK X

cybertim

29 giorni fadiscuss

Graphene is NOT a jailbroken/rooted OS, its a real secure unrooted, bootloader locked OS, and MS Authenticotor works just fine. If anything does not work its related to dependency of the App maker on a certain attestation google play services grapheneos.org/articles/attestation-compatibility-guide

cybertim

mese scorsodiscuss

I'm using bazzite on my ai-rig just because it has the gpu-optimized things setup (also nvidia-open). Looking at P2P seems to be available only for 90-versions of the nvidia rtx gpu line, not 80, and some versions of 50xx? (apparently the 5080?). Anyways, i downloaded that uncensored model and tweaked those kv settings etc. still getting 60-80tk/s but im able to get my context on 180224 now, used to be 131072 which gave me some trouble, this is already a win :)

cybertim

mese scorsodiscuss

$ nvidia-smi topo -p2p r

GPU0 GPU1

GPU0 X CNS

GPU1 CNS X

i guess not, i use llama.cpp with:

--spec-draft-n-max 3 --spec-type draft-mtp --split-mode tensor --tensor-split 1,1

and my (gen) tk/s are between 60-80 tk/s

will test this uncensored model and ngram added as well this weekend

btw, i also set my powerlimit to 220watt per card (with nvidia-smi) that will cost you around 1 tk/s but safe you a LOT of power and heat :)

cybertim

mese scorsodiscuss

I bought two 3080/20gb and one of those MACHINIST X99 mainboards as well (one with two full x16 pcie slots) those boards come with a xeon cpu included (for the pcie lane support) it set me back 800 euros total (had a spare psu, ssd and mem in a drawer) and now im also happily running 80tk/s Qwen 3.6 Q8 (MTP).

cybertim

anno scorsodiscuss

l'm curious if Bartosz will actually contact him for that final casting, i know i would definitely love to own one. And mainly because I love mechanical watches, especially what's inside, but I don't like to wear or really use them, this is a great solution for this "problem".

cybertim

4 anni fadiscuss

How does bitwarden’s backend being build in a gc language matter? There is also an api compatible project in rust called Vaultwarden if gc matters. Is 1password and lastpass’s backend better in that regard? would be nice if he could elaborate on that point.

cybertim

4 anni fadiscuss

And to add to this, i learned to not use the 'bluetooth' icon to manage the AirPods, instead use the 'triangle with broadcast signal'-icon found next to the volume in the control centre or on many other different locations on your ipad, iphone or apple tv.

Just turn off the auto-connect feature and start using that icon, all the mentioned 'annoyances' will go away :)

comments