Show HN: LLaMA Nuts and Bolts, A holistic way of understanding how LLMs run(github.com)
github.com
Show HN: LLaMA Nuts and Bolts, A holistic way of understanding how LLMs run
https://github.com/adalkiran/llama-nuts-and-bolts
0 comments
—
https://github.com/adalkiran/llama-nuts-and-bolts
—
I’m so excited to show my latest open-source project here.
I know it’s in a very niche technical domain, but hope you will like my project. Because using Go on Machine Learning and Large Language Models is an interesting experience for me. Please check it out and I’d love to read your thoughts!
A holistic way of understanding how LLaMA and its components run in practice, with code and detailed documentation. "The nuts and bolts" (practical side instead of theoretical facts, pure implementation details) of required components, infrastructure, and mathematical operations without using external dependencies or libraries.
The goal is to make an experimental project that can perform inference on the LLaMa 2 7B-chat model completely outside of the Python ecosystem (using the Go language). Throughout this journey, the aim is to acquire knowledge and shed light on the abstracted internal layers of this technology.
This journey is an intentional journey of literally reinventing the wheel. While reading my journey in the documentation, you will see the details of how Large Language Models work, through the example of the LLaMa model.
If you are curious like me about how the LLMs (Large Language Models) and transformers work and have delved into conceptual explanations and schematic drawings in the sources but hunger for deeper understanding, then this project is perfect for you too!
You will not only find the details of the LLaMa architecture but will find explanations of a wide variety of related concepts in the documentation directory. From reading a Pickle, a PyTorch model, a Protobuf, and a SentencePiece tokenizer model files at byte-by-byte level, to internals of BFloat16 data type, implementation from scratch of a Tensor structure and mathematical operations including linear algebraic computations.
This project was initially started to learn what an LLM does behind by running and debugging it and was made for experimental and educational purposes only, not for production use.
I will be happy if you check out it and comments are welcome!
https://github.com/adalkiran/llama-nuts-and-bolts