HackerTrans
TopNewTrendsCommentsPastAskShowJobs

controversy187

55 karmajoined 9 lat temu

Submissions

I built a large language model "from scratch"

brettgfitzgerald.com
91 points·by controversy187·w zeszłym roku·16 comments

comments

controversy187
·12 godzin temu·discuss
[flagged]
controversy187
·6 miesięcy temu·discuss
https://brettgfitzgerald.com

General learnings and findings about technology and personal projects.
controversy187
·7 miesięcy temu·discuss
I'm writing https://github.com/controversy187/kerbalbook, which aims to be a hobbyist's guide to orbital mechanics, spaceflight, and maybe astrophysics. All this through the lens of a Kerbal Space Program walkthrough.
controversy187
·12 miesięcy temu·discuss
I'm building https://adeptli.org with the goal of making personalized learning available to anyone with an internet connection. It's pay-as-you-go, and everyone starts with free credits to check it out and not incur costs if they don't find it useful. I'm still very early in development and very open to feedback!
controversy187
·w zeszłym roku·discuss
I wrote the blog post, and I did it on my own freewill, and am receiving no compensation. The main reason I wrote it was to help cement my own learnings from the book. I've heard that the best way to learn something is to teach it, so I wanted to see how much I could regurgitate on my own. Turns out, not a whole lot. It was hastily written, and more of a "brain dump" than anything else. I'm entering a new-to-me field, and wanted a place to document the things I'm learning. If anyone finds it interesting, great. If not, no big deal.

As for the specifics of the model I trained, I would be hard pressed to recall the specifics off the top of my head. I believe I trained a small model locally, but after completing that as a PoC, I downloaded the GPT-2 model weights, then trained / fine-tuned those locally. That is what the book directed. All the steps are in my github repo, which (unsurprisingly) like the author's repo. His repo actually has more explanation. Mine is more or less just code.