I'm focused on finals at the moment but I'd be more than happy to share what I've been looking at so far (I plan to look into this a lot more in the summer).
When it comes to the theory, 3blue1brown released some really nice videos that solidified a lot of my current understanding, especially with the attention mechanism. I think I am also going to do a mixture of reading papers/watching youtube videos on things that are interesting (ex. Qlora for fine-tuning or diffusion models for image generation) and trying to build out a simple implementation myself and see where that takes me. Maybe I'll start out with the Karpathy nano-gpt videos but try and do it with a different data set.
But for people like myself who lack the math background, data, and compute to be able to train very strong LLMs, I think it is also a good idea to try and build some projects/apps that use a fine-tuned LLM, or just call the OpenAI API.
I'm still a bit lost myself, but in 2 weeks when I'm done with exams, I'm more than happy to keep exchanging resources with you.
When it comes to the theory, 3blue1brown released some really nice videos that solidified a lot of my current understanding, especially with the attention mechanism. I think I am also going to do a mixture of reading papers/watching youtube videos on things that are interesting (ex. Qlora for fine-tuning or diffusion models for image generation) and trying to build out a simple implementation myself and see where that takes me. Maybe I'll start out with the Karpathy nano-gpt videos but try and do it with a different data set.
But for people like myself who lack the math background, data, and compute to be able to train very strong LLMs, I think it is also a good idea to try and build some projects/apps that use a fine-tuned LLM, or just call the OpenAI API.
I'm still a bit lost myself, but in 2 weeks when I'm done with exams, I'm more than happy to keep exchanging resources with you.