Show HN: I trained a 65B LLM on my texts to talk to myself (details inside)(airic.serveo.net)
airic.serveo.net
Show HN: I trained a 65B LLM on my texts to talk to myself (details inside)
https://airic.serveo.net/
7 comments
Cool. But please tell me that your childhood phone number wasn't 708-642-9XXX (censored in case it's real) because it just told me that "fact"... and we were chatting about trivia night!! lol
It also told me that your mom was an astronaut and that you were born in space. I love this thing!
It also told me that your mom was an astronaut and that you were born in space. I love this thing!
This is so interesting. I've been wanting to do this, not to chat, but to generate coherent answers to real chats - like pre-baked answers to quickly reply. Looks pretty complicated, though. I have no idea where to start.
According to this, your boss Matt Cain is the most annoying person you know. This would be a much more exciting chat if it hadn’t been scrubbed haha
It is a cool app but it might spill secrets? It told me some things like someone’s name who you know, but it could be making it up.
It's probably making it up. I scrubbed the PII vs a list of fake names and info.
I posted this to Reddit and had several hundred people talking to it. Salient points from that discussion:
LLAMA 1 65b
Rank 128
5 epochs
Batch size 1, 256 cutoff
Trained in the Oobabooga suite using bitsandbytes 4-bit quantization for the lora
Loss around 1.5 seems to give the most coherent results
Trained on raw text dumps that is then parsed by a crappy Blazor Server app I threw together in a few hours. Text format is just "Sender:The Message\n"
Trained on 2x 3090
Training took about 16 hours at a 90% power cap on the 3090's
Trained on ~30k texts (I talk a lot, that was just 2 years)
There's nothing telling it that it's a robot, though it sometimes seems to know
It was largely inspired by the Unreal Engine lora tutorial
I generated a list of fake names and addresses, pulled a list of my contacts, and then scripted out swapping the names and addresses for fictitious PII. I don't really send other sensitive data through text and my account is so thoroughly associated with my real name/location that the data leakage risk is manageable for the short period of time I'll have this available. It tends to halucinate fake PII as well which I think is partially a side effect of the data scrubbing. You'll notice it says things like that I live at 420 Ligma.
I'll need to mix in some actual assistant tasks to the dataset before it will actually be useful as an assistant. Right now it's largely just for idle conversation.
It's pretty ADHD and will randomly go off on its own tangents. I don't think it's the model. I think I just talk like that.
Let me know if you have any questions or comments. I built it for myself, but figured I'll let the communities that have taught and entertained me so much play with it a little, too.
Note: it says some pretty unhinged stuff. There's absolutely no guardrails. It also tends to talk like you're already friends with history.