Then, where to read more about this? People who built autograd and other frameworks like Pytorch, mxnet, etc. should have learnt them in details somewhere. Where? AFAIK mxnet came out of academia (probably CMU).
Surprised to see these introductory courses haven't been mentioned yet.
These courses [0] [1] are on EdX and are taught by UBC Professor Gregor Kiczales. The explanations are so lucid it made recursion click for me. You can audit these courses without paying a single dime. They are based on the book How to Design Programs [2] which has much more stuff than the courses.This book is used in UBC, UWaterloo, NorthEastern and many other places.
Along the same lines there is another book for beginners called A Data-Centric Introduction to Computing [3]. It is used in Brown for their introductory courses.
These two part course rearranged my brain about what really programming is about. It is about working with data and the way the data is arranged determines how most of the program/functionality (pun intended) is written.
Programming is not a series of instructions to be executed by the computer. That was when assembly language programming was the main stay of programming computers. Not now.
I feel pure joy when I see HtDP mentioned anywhere on HN. This book made programming click for me. Turned me from fidling with code to thinking it through in a systematic way.
I hope the authors extend this someday to imperative programming too and compare and contrast.
I have done my bachelors and masters in electrical and communication engineering. I know calculus, matrix algebra and prob stats on a level of engineering. Not in much depth.
https://arxiv.org/abs/2304.06035