Ask HN: As a professional, does it worth it to learn the math behind NN?
In 2021, as a (junior) professional working directly with ML/AI, does it worth it to learn what's going on in terms of mathematics behind Neural Networks? Or should we just focus on the pratical side of things, like daily applications using frameworks/libraries and metrics/performance?
17 comments
My personal feeling is that it's always "worth it" to learn more maths. Not saying it's absolutely required, and I'm a big fan of (generally speaking) prioritizing the pragmatic / practical / hands-on over the purely theoretical. But, even within that belief system, I would say that - all things being equal - any opportunity to learn more maths is worthwhile, at least up to a certain point. Figuring out the details of the "certain point" is left as an exercise for the reader though.
I recommend learning multi variable calculus and linear algebra. You can take courses online for free. Personally, I didn’t really understand what was going on until I refreshed my math.
It won’t make you an ML expert, but it’s like the difference between knowing how to write a sort function vs. only knowing how to call a sort function. Ideally, you understand both. Not saying you need to understand how every sort function works internally and write a proof for it… but a basic understanding is helpful.
It won’t make you an ML expert, but it’s like the difference between knowing how to write a sort function vs. only knowing how to call a sort function. Ideally, you understand both. Not saying you need to understand how every sort function works internally and write a proof for it… but a basic understanding is helpful.
thats a great advice and makes total sense. thank you!
Since you're working in the field - yes, obviously. It's not even that much math - the building blocks are fairly straightforward linear algebra and calculus.
Here's a nice, fairly detailed, summary - https://explained.ai/matrix-calculus/. But if you haven't taken undergraduate level single/multivariable calculus and linear algebra, I would take them from MIT Opencourseware even if it's just to fight off the impostor syndrome a little bit.
Here's a nice, fairly detailed, summary - https://explained.ai/matrix-calculus/. But if you haven't taken undergraduate level single/multivariable calculus and linear algebra, I would take them from MIT Opencourseware even if it's just to fight off the impostor syndrome a little bit.
thank you! on a quick look, that seems straight to the point, for someone who had calculus and linear algebra in 2006.
a printable version of your link, in case someone want it as well:
https://arxiv.org/pdf/1802.01528.pdf
Frankly, I rarely see this complicated math in academic papers to be useful in practice.
Most of the stuff that is valuable in practice does not require much math.
Many people believe math actually hindered the development of AI.
Most of the stuff that is valuable in practice does not require much math.
Many people believe math actually hindered the development of AI.
The papers are interesting to people making tools - if you can see what mathematical assumptions are made you can see what limitations a method has and whether it can apply to cases you're interested in.
How much do you know already?
Well, not much. Data isn't the main product where I work for, so I end up just using python libraries for some mundane applications and small datasets (image classification and recommender systems). But I'm the "need to know from scratch" kinda person and I wonder if I should waste my time learning the math behind it or if its useless
What's the first derivative of
x * cos(x)
?wolfram alpha told me its: cos(x) - xsin(x)
lol kidding, i had calculus 15 years ago. so it sounds I should recap derivatives.
This book is a little old but you should learn whatever math it takes to understand it
https://www.amazon.com/Networks-Recognition-Advanced-Econome...
Another oldie-but-goodie is
https://www.amazon.com/Neural-Networks-Lecture-Computer-Scie...
which tells the secret of when to stop when you're doing "early stopping", something I've seen many modern deep learners fail to get right.
Off the top of my head I would say the fundamental math about deep networks is not really new. Most of the work in that field is pretty ad-hoc and not a lot is proven; probably people that are proving things are using difficult graduate-level math but you don't need to go there.
https://www.amazon.com/Networks-Recognition-Advanced-Econome...
Another oldie-but-goodie is
https://www.amazon.com/Neural-Networks-Lecture-Computer-Scie...
which tells the secret of when to stop when you're doing "early stopping", something I've seen many modern deep learners fail to get right.
Off the top of my head I would say the fundamental math about deep networks is not really new. Most of the work in that field is pretty ad-hoc and not a lot is proven; probably people that are proving things are using difficult graduate-level math but you don't need to go there.
thank you Paul for the references, will def take a look mainly in the early stopping problem.
there is not much math, don't let the academic papers razzle dazzle you, they make things look complicated to get their grants
in practice, as long as you've studied basic calculus and understand how to find a minimum of a function via derivative you're good, there is your "gradient descent" in a nutshell: https://www.mathsisfun.com/calculus/maxima-minima.html
everything else is plug-and-play from existing libraries
you can ask any "data scientist" or "ML engineer" what they do all day, it's a whole lot of copy paste, and tweaking the data and parameters through trial and error until it fits
Edit: Ok , it would also help to understand dimensionality reduction via PCA/SVD at least once, it's available in any linear algebra book: https://en.wikipedia.org/wiki/Singular_value_decomposition , https://en.wikipedia.org/wiki/Principal_component_analysis that's probably the best and most "scientific" part of ML
in practice, as long as you've studied basic calculus and understand how to find a minimum of a function via derivative you're good, there is your "gradient descent" in a nutshell: https://www.mathsisfun.com/calculus/maxima-minima.html
everything else is plug-and-play from existing libraries
you can ask any "data scientist" or "ML engineer" what they do all day, it's a whole lot of copy paste, and tweaking the data and parameters through trial and error until it fits
Edit: Ok , it would also help to understand dimensionality reduction via PCA/SVD at least once, it's available in any linear algebra book: https://en.wikipedia.org/wiki/Singular_value_decomposition , https://en.wikipedia.org/wiki/Principal_component_analysis that's probably the best and most "scientific" part of ML
Thank you! appreciate your reply, honestly. i have an engineering degree but it has nothing to do with computers, so i kinda struggle sometimes. I basically want to get fluent with the metrics/performance when applying ML/NN, when to tweak what, how to improve some stuff, what algorithm works best for some of the problems, etc.
This, in a nutshell.