> how in explanations of attention the Q, K, V matrices always seem to be pulled out of a hat after being motivated in a hand-wavy metaphorical way.
Justin Johnson's lecture on Attention [1] mechanisms really helped me understand the concept of attention in transformers. In the lecture he goes through the history and and iterations of attention mechanisms, from CNNs and RNNs to Transformers, while keeping the notation coherent and you get to see how and when in the literature the QKV matrices appear. It's an hour long but it's IMO a must watch for anyone interested in the topic.
You got to give it to the pytorch team, they're really great at bringing complex optimization schemes (mixed-precision, torch.compile, etc) down to a simple to use API. I'm glad I moved from TF/Kerasto Pytorch around 2018-2019 and never looked back. I'm eager to try this as well.
Not sure if my experience is relevant, but I did a couple of internships in web dev during my bachelors degree in CS and quickly realized it wasn't for me. I then did a masters and now a PhD in medical imaging where I extensively use machine learning (design and train my own models, doing both supervised and RL) but I wouldn't say I am a researcher in AI/ML.
Because I am still in the academic process, I had the opportunity to take a couple of classes on the subject. Three books that I would recommend going over to make sure your foundation in ML and mathematics are solid are
-Pattern recognition and machine learning by Christopher Bishop
-Mathematics for Machine Learning by Peter Deisenroth
-Deep Learning by Courville, Bengio and Goodfellow
All three are legally available online in some form. I can't say I have any experience in finding a job related to ML though.
I have a similar schedule, although a slightly more organized. I try to be available from 9 to 4, atleast through emails if I'm not in my office, because that's when other people also work and might need me. I often leave mid-day to go running or come in late because I went bouldering in the morning. If I don't feel inspired to write or work, I often instead go for a walk and read books tangentially to my research topics. Then some days I'll work 12 hours straight.
I mostly work when it feels right, yes, and do something else when it doesn't. But I am doing my PhD and (in Canada) it absolutely does not pay enough to live comfortably.
Justin Johnson's lecture on Attention [1] mechanisms really helped me understand the concept of attention in transformers. In the lecture he goes through the history and and iterations of attention mechanisms, from CNNs and RNNs to Transformers, while keeping the notation coherent and you get to see how and when in the literature the QKV matrices appear. It's an hour long but it's IMO a must watch for anyone interested in the topic.
[1]: https://www.youtube.com/watch?v=YAgjfMR9R_M