C++ GPT-2 inference engine(github.com)
github.com
C++ GPT-2 inference engine
https://github.com/a1k0n/a1gpt
7 comments
Indeed it does, thanks for posting this.
From the same twitter thread, a dense implementation in less than 100 lines of plan C:
https://github.com/davidar/eigenGPT/tree/c
https://github.com/davidar/eigenGPT/tree/c
I found this extremely helpful even though it's not in C++: https://jaykmody.com/blog/gpt-from-scratch/.
Yes it's very nice from the author to provide this very readable text.
I saved this text to read it later.
This is pretty cool. I had the same idea but in zig: https://github.com/EugenHotaj/zig_gpt2
Not fully finished yet, haven't gotten around to implementing bpe encoding/decoding and only some ops use BLAS.
Not fully finished yet, haven't gotten around to implementing bpe encoding/decoding and only some ops use BLAS.
Well, I downloaded and compiled it (cool! Thanks!) but no matter what prompt I give it, it just prints out gibberish....where do I go now to learn how to properly use it?
"I reimplemented GPT-2 from scratch in C++ as an exercise to really understand the nuts and bolts of LLMs. GPT2-117M isn't a super great model, but it's extremely satisfying to get it to generate basically the same thing as other reference implementations."
"I" refers to the guy that wrote this, I, version_five have nothing to do with it, I just thought it looked cool.