Ask HN: Best way to get started with AI?
92 comments
I'm in the same boat. For long time, I was interested in AI but at the same time intimidated by math. I'm relatively comfortable with discrete mathematics and classical algorithms and at the same time calculus and linear algebra is completely foreign to me. Also, I do not accept way to learn ML without good understanding of core principles behind it. So math is a must.
A few months ago, I stumbled upon very amazing YouTube Channel 3Blue1Brown which explains math in very accessible way and at the same time I got feeling that I finally started understanding core ideas behind linear algebra and calculus.
Just recently he published 4 videos about deep neural networks:
https://www.youtube.com/watch?v=aircAruvnKk
https://www.youtube.com/watch?v=IHZwWFHWa-w
https://www.youtube.com/watch?v=Ilg3gGewQ5U
https://www.youtube.com/watch?v=tIeHLnjs5U8
So my fear of ML was gone away and I'm very excited to explore whole new world for neural networks and other things like support vector machines etc
A few months ago, I stumbled upon very amazing YouTube Channel 3Blue1Brown which explains math in very accessible way and at the same time I got feeling that I finally started understanding core ideas behind linear algebra and calculus.
Just recently he published 4 videos about deep neural networks:
https://www.youtube.com/watch?v=aircAruvnKk
https://www.youtube.com/watch?v=IHZwWFHWa-w
https://www.youtube.com/watch?v=Ilg3gGewQ5U
https://www.youtube.com/watch?v=tIeHLnjs5U8
So my fear of ML was gone away and I'm very excited to explore whole new world for neural networks and other things like support vector machines etc
Worth noting that 3Blue1Brown also did a series on linear algebra which is eye-opening to say the least. Playlist at:
https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...
Even if you think you grok matrices, have a go at the first few videos of that playlist, if just for the visualization. It really helped me see what matrices (and operations on matrices) represent!
https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...
Even if you think you grok matrices, have a go at the first few videos of that playlist, if just for the visualization. It really helped me see what matrices (and operations on matrices) represent!
3Blue1Brown is a treasure. The production value is excellent, and he's great at taking seemingly uninteresting ideas and painting a beautiful picture to connect them in twenty minutes. I used to go through a video before falling asleep each night.
I just watched the first video. Thank for sharing.
I have also used Mathematical monk, who was simple and good in introducing basic concepts and tools related to ML. https://www.youtube.com/user/mathematicalmonk
I also recommend taking up computer graphics for honing your skills in linear algebra. Graphics are essentially applied linear algebra.
I came here to write a similar comment. Really make sure to watch the playlists in the correct order on the above YouTube channel.
Having watched the third one out of sequence, seeing the first two and then watching the third again helped me get a good understanding of the fundamentals. 3blue1brown as a narrator does a excellent job of allowing a rather tricky subject be more approachable, and inspired me to buy a course to allow a deeper dive into the math behind ML+NNs.
Nice to some one pointing the fundamentals. A good understanding about probabilist models is good too. After getting into too the basic math knowledge, I suggest this: https://classroom.udacity.com/courses/ud730
Regarding linear algebra, I highly recommend Klein's Coding the Matrix which uses Python to teach linear algebra.
I believe it was developed for Brown's linear algebra course for CS undergrads.
I believe it was developed for Brown's linear algebra course for CS undergrads.
Wow thanks for this resource!
I highly recommend Andrew Ng's Coursera courses for both Machine Learning and Deep Learning. Good for beginners, Math is taught along with the course, and gets you a solid foundation:
https://www.coursera.org/learn/machine-learning/
https://www.coursera.org/learn/neural-networks-deep-learning...
https://www.coursera.org/learn/machine-learning/
https://www.coursera.org/learn/neural-networks-deep-learning...
Andrew Ng's courses are excellent. Another pretty good Coursera course is Machine Learning Foundations from the University of Washington. It is very high level and novice friendly. While it covers non of the math and very little programming it does give a nice quick introduction to the most popular ML techniques out there and when to use them. It all depends on what level you are interested in starting. They also have follow up courses that go deeper into the different techniques.
+1 for this recommendation.
I would specifically recommend Machine Learning Foundations: A Case Study Approach - It is fantastic and helped me greatly start my ML journey last year.
Turi is awesome, I hope Apple is doing something great with it.
I would specifically recommend Machine Learning Foundations: A Case Study Approach - It is fantastic and helped me greatly start my ML journey last year.
Turi is awesome, I hope Apple is doing something great with it.
Thank you! Should I start with the Machine Learning one?
From my experience Andrew Ng wiped the floor with every other lecturer I've had. Both the ML and his new Deep Learning course.
If the lecturers aren't very interesting Coursera can be as hard as any other lectures. I gave up on the Scala functional programming and disappointingly have stalled with Geoffrey Hinton's Neural Networks courses.
But I really can't understate how good Andrew Ng is, he has a very relaxed manner and manages to make some very complex topics seem almost trivial.
The worst of the mathematics is derivatives and matrix multiplication. You can even avoid matrix multiplication mostly in the ML course, but in his Deep Learning course he takes you through the 300x performance benefit you get from using NumPy and matrix multiplication vs loops.
If the lecturers aren't very interesting Coursera can be as hard as any other lectures. I gave up on the Scala functional programming and disappointingly have stalled with Geoffrey Hinton's Neural Networks courses.
But I really can't understate how good Andrew Ng is, he has a very relaxed manner and manages to make some very complex topics seem almost trivial.
The worst of the mathematics is derivatives and matrix multiplication. You can even avoid matrix multiplication mostly in the ML course, but in his Deep Learning course he takes you through the 300x performance benefit you get from using NumPy and matrix multiplication vs loops.
At your level yes, I would recommend starting with the ML course. It is really beneficial to understanding how the mathematics work.
The two most important things to remember, since the courses are challenging: 1) don't be in a hurry, and 2) don't give up! Take the time to learn every detail presented, do the optional exercises, and dig deep.
The two most important things to remember, since the courses are challenging: 1) don't be in a hurry, and 2) don't give up! Take the time to learn every detail presented, do the optional exercises, and dig deep.
It's definitely challenging. The math and just seeing the complicated formulas really push me, but the reward is good too. I'm tired of pushing pixels and doing some meaty stuffy like ML is a nice change of pace.
I would recommend starting with deep learning first since that's what you are interested in and it covers all the ML principles you need to be familiar with. If you want to go deeper and get familiar with other ML techniques too you can easily follow the old course afterwards.
Personally I recommend Stanford CSI 231n
http://cs231n.stanford.edu/
Its specifically geared towards visual recognition, but it starts with the basics of machine learning and moves on to feed forward nets and covnets and covers RNNs and attention towards the end.
The assignments are a great set of jupyter notebooks that really get your hands on the material and you can find a number of peoples complete assignments on github just by searching.
The lectures are available online as well https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-z...
I've done hinton's and Ngs courses and as someone who already has a non-ai development background I found this to be the best introduction. Its really an extension of Andrej Karpathy's Neural Nets for Hackers (http://karpathy.github.io/neuralnets/)
Its specifically geared towards visual recognition, but it starts with the basics of machine learning and moves on to feed forward nets and covnets and covers RNNs and attention towards the end.
The assignments are a great set of jupyter notebooks that really get your hands on the material and you can find a number of peoples complete assignments on github just by searching.
The lectures are available online as well https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-z...
I've done hinton's and Ngs courses and as someone who already has a non-ai development background I found this to be the best introduction. Its really an extension of Andrej Karpathy's Neural Nets for Hackers (http://karpathy.github.io/neuralnets/)
Did you signup for the course, or just followed what's available on the page?
It doesn't seem like there is any starting date there.
It's really important not to skip the math. As a friend once said to me, doing deep learning without understanding the math is like gambling. It's fine to initially take a more practical, project-based approach for the sake of staying motivated, and you'll retain things better if you have project goals in mind, but, the math is that important.
The good news is that compared to other technical fields, the math is also relatively shallow. Here are some good resources that you don't need more than calculus/linalg for (I've used all of them and they got me off the ground):
http://cs231n.stanford.edu/
http://neuralnetworksanddeeplearning.com/
http://course.fast.ai/
Once you feel confident, the Deep Learning book is more math-heavy, but it is really very good. The authors are more or less deep learning gods. It'll teach you a tremendous amount about how/why neural nets work and the principles used to discover new architectures, and gain a strong intuition for how to use neural nets as a tool. Read it slowly---unless you're already good at math, it takes a while to get through. Don't skip the first five chapters. Use Google and Wikipedia to pick up concepts you don't understand along the way instead of skipping over them (it will bite you later).
The good news is that compared to other technical fields, the math is also relatively shallow. Here are some good resources that you don't need more than calculus/linalg for (I've used all of them and they got me off the ground):
http://cs231n.stanford.edu/
http://neuralnetworksanddeeplearning.com/
http://course.fast.ai/
Once you feel confident, the Deep Learning book is more math-heavy, but it is really very good. The authors are more or less deep learning gods. It'll teach you a tremendous amount about how/why neural nets work and the principles used to discover new architectures, and gain a strong intuition for how to use neural nets as a tool. Read it slowly---unless you're already good at math, it takes a while to get through. Don't skip the first five chapters. Use Google and Wikipedia to pick up concepts you don't understand along the way instead of skipping over them (it will bite you later).
It somewhat depends on if you are looking to build AI to address business problems or if you are more interested in the type of AI work you see companies like Google discussing.
I can speak to what "AI" means for most businesses outside Top Tech which more frequently work with tabular, relational, or log data rather than image and text. For these companies, this is what you need to learn how to do
At my company, we've noticed a lot of programmers are intimated by the feature engineering step in particular, so we tried to make it easier by creating an open source library called Featuretools [0].
[0] https://github.com/featuretools/featuretools
I can speak to what "AI" means for most businesses outside Top Tech which more frequently work with tabular, relational, or log data rather than image and text. For these companies, this is what you need to learn how to do
1. Define a prediction problem and extract labels
2. Organize and clean the data for prediction
3. Perform feature engineering by applying domain expertise
4. Apply an off-the-shelf open source machine learning algorithm like a random forest
At my company, we've noticed a lot of programmers are intimated by the feature engineering step in particular, so we tried to make it easier by creating an open source library called Featuretools [0].
[0] https://github.com/featuretools/featuretools
Thanks for the featuretools project, it sounds really useful. Is it Python 2 only?
the latest release is for python 2.7 only. we have a pull request [0] up right now for python 3 that passes all the automated testing.
[0] https://github.com/Featuretools/featuretools/pull/16
[0] https://github.com/Featuretools/featuretools/pull/16
Great resources in the replies. If you want an environment to run code in w/o much setup, try our free service:
https://notebooks.azure.com
it has Py2, Py3, R, F#, anaconda, TF, CNTK, etc. pre-installed.
There are some ML tutorials on it already + you can use the "load from github" feature to load, run, edit, ... many of the great tutorials already on github.
Other similar environments include colab by google and cocalc.
#Disclaimer: Microsoft
https://notebooks.azure.com
it has Py2, Py3, R, F#, anaconda, TF, CNTK, etc. pre-installed.
There are some ML tutorials on it already + you can use the "load from github" feature to load, run, edit, ... many of the great tutorials already on github.
Other similar environments include colab by google and cocalc.
#Disclaimer: Microsoft
Interesting, didn't know about this, thanks for sharing
I'm involved with a startup that's specifically tackling this very problem -- how do you learn the theory & application of machine learning quickly (especially if you already know programming well). We teach using diagrams and interactive coding exercises in the browser: www.dataquest.io
If you already know Python, you could dive straight into machine learning (https://www.dataquest.io/course/machine-learning-fundamental...) and work your way upto calc / lin al, linear regression, decision trees, neural nets, etc.
If you want to get a taste without signing up, you can check out our blog posts that preview the course (like this one: https://www.dataquest.io/blog/machine-learning-tutorial/)
Happy to answer any questions over DM or email (srini@ourdomain).
If you already know Python, you could dive straight into machine learning (https://www.dataquest.io/course/machine-learning-fundamental...) and work your way upto calc / lin al, linear regression, decision trees, neural nets, etc.
If you want to get a taste without signing up, you can check out our blog posts that preview the course (like this one: https://www.dataquest.io/blog/machine-learning-tutorial/)
Happy to answer any questions over DM or email (srini@ourdomain).
1. Start with the fast.ai courses. It's an applied deep learning course using state of the art techniques.
2. For classical machine learning (regression, etc...), Andrew Ng's course on Coursera is widely considered "the basics"
3. As you progress, check out CS231 and CS224 from Stanford for state of the art image processing and natural language processing techniques. The lecture videos are on YouTube and the course assignments are available online. The third course I recommend is Geoffrey Hinton's neural networks course on Coursera (he is one of the most important researchers in the field).
4. If you're an application engineer, focus on using existing tooling to build cool projects. Keras and scikit-learn are great out of the box tools.
5. If you are more research oriented, you can start reading papers. In Silicon Valley, there's a meet up group that reads papers every Monday and tries to implement the algorithms in the paper. It takes a while to get to this level, but try not to get overwhelmed. Experts spend 7 years studying this stuff full time to get a PhD.
6. You really don't need much math to get started with ML. A high school understanding of calculus and some basic understanding of numerical optimization are the two main concepts you need to know. If you want to get into the research, there'll come a time when you will need more advanced math, but in my experience you can pick that up as you go along if you are curious.
Maybe you could start an AI study group online? The Silicon Valley study group was great, but I was just visiting.
Maybe you could start an AI study group online? The Silicon Valley study group was great, but I was just visiting.
If TensorFlow is what you're interested in I personally found "Hands-on Machine Learning with SciKit-Learn and TensorFlow by Aurélien Géron" to be the best introduction after introducing myself to the subject with Siraj's YouTube videos
https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-T...
https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-T...
AI != ML
For AI, I would take the Udacity AI courses.
For ML, I would take the Udacity ML courses.
I take a lot of different online courses, I have no affiliation with Udacity, but their courses are just too good.
I studied AI (focused on ML) in a decent grad school (and I like to think I had the best teachers there), and I think the quality of the courses is comparable.
For AI, I would take the Udacity AI courses.
For ML, I would take the Udacity ML courses.
I take a lot of different online courses, I have no affiliation with Udacity, but their courses are just too good.
I studied AI (focused on ML) in a decent grad school (and I like to think I had the best teachers there), and I think the quality of the courses is comparable.
Eh, I think most people use them somewhat synonymously nowadays. ML is a subset of AI, which has become buzzwordy enough to lost most meaning IMO.
Isn't AI just applied ML?
Or is it an operant/classical conditioning sort of thing, where AI is specifically about training programs to act rather than to perceive/categorize things?
I suppose you can have AI that incorporates no ML (like most video game AI), but I'd imagine that will become vanishingly rare in the future.
Or is it an operant/classical conditioning sort of thing, where AI is specifically about training programs to act rather than to perceive/categorize things?
I suppose you can have AI that incorporates no ML (like most video game AI), but I'd imagine that will become vanishingly rare in the future.
In brief, AI uses existing knowledge and/or heuristics (to solve problems that lack a closed-form solution), while ML acquires knowledge and heuristics toward the same end, with the added goal of improving performance as it learns and adapting to changing conditions.
Traditionally, AI has been divided into distinct subfields (e.g. search, planning, natural language and speech processing, game playing, computer vision, robotics, knowledge representation, expert systems, logic, and ML). Today, ML is employed in all AI subfields, but until recently, most subject matter in each AI subfield had been unrelated to ML. In the past decade especially, that's changed as deep learning and probabilistic methods have gained mindshare and now are largely unavoidable when tackling AI-related problems.
In general, AI's subfields have focused on identifying fundamental obstacles and important features in their own problem domain and developing appropriate techniques that operate on those features when solving problems (like using object recognition and localization to solve vision problems like autonomous driving). I suspect AI's past emphasis on feature engineering has faded as NN-based ML has risen.
Traditionally, AI has been divided into distinct subfields (e.g. search, planning, natural language and speech processing, game playing, computer vision, robotics, knowledge representation, expert systems, logic, and ML). Today, ML is employed in all AI subfields, but until recently, most subject matter in each AI subfield had been unrelated to ML. In the past decade especially, that's changed as deep learning and probabilistic methods have gained mindshare and now are largely unavoidable when tackling AI-related problems.
In general, AI's subfields have focused on identifying fundamental obstacles and important features in their own problem domain and developing appropriate techniques that operate on those features when solving problems (like using object recognition and localization to solve vision problems like autonomous driving). I suspect AI's past emphasis on feature engineering has faded as NN-based ML has risen.
It would seem that many people have divergent definitions but I’ve always learned that AI was Artificial intelligence broadly construed (goal seeking, planning, rule based expertise, logic etc) while machine learning is the specific subset that is (roughly speaking) statistical (neural nets, regressions, svm and the like).
Don’t have strong feelings about these definitions.
Don’t have strong feelings about these definitions.
In my opinion the best way to get started is first study statistical inference and modelling, in particular linear regression and the method of maximum likelihood. This will give you a critical eye later on for discerning when it's a good idea to actually use ML and when it's not (an important skill that apparently is in very short supply these days ;).
Introduction to Statistical Learning with Applications in R
http://www-bcf.usc.edu/~gareth/ISL/
Statistical Rethinking by Richard McElreath gives a good introduction to Bayesian approaches to statistical analysis https://www.youtube.com/channel/UCNJK6_DZvcMqNSzQdEkzvzA
Statistical Rethinking by Richard McElreath gives a good introduction to Bayesian approaches to statistical analysis https://www.youtube.com/channel/UCNJK6_DZvcMqNSzQdEkzvzA
My friend and I wrote this guide called ML4Humans.
https://medium.com/machine-learning-for-humans/why-machine-l...
People like you are our primary audience :) it should take you exactly where you want to start and take you a good chunk of the way to where you wanna get.
Please check it out
People like you are our primary audience :) it should take you exactly where you want to start and take you a good chunk of the way to where you wanna get.
Please check it out
Siraj Raval does a great job of explaining AI topics with fun, fresh, and easy to understand topics and examples:
https://www.youtube.com/channel/UCWN3xxRkmTPmbKwht9FuE5A
Here's a recent video where he talks about how to create new Pokemon with Generative Adversarial Networks (https://en.wikipedia.org/wiki/Generative_adversarial_network). Nice contrast from the usual MNIST dataset, especially if you want to be inspired to think about novel ways to apply this stuff:
https://www.youtube.com/watch?v=yz6dNf7X7SA
https://www.youtube.com/channel/UCWN3xxRkmTPmbKwht9FuE5A
Here's a recent video where he talks about how to create new Pokemon with Generative Adversarial Networks (https://en.wikipedia.org/wiki/Generative_adversarial_network). Nice contrast from the usual MNIST dataset, especially if you want to be inspired to think about novel ways to apply this stuff:
https://www.youtube.com/watch?v=yz6dNf7X7SA
Intro to AI
https://www.udacity.com/course/intro-to-artificial-intellige...
Machine Learning
https://www.coursera.org/learn/machine-learning
The Pacman programming exercises in python
http://ai.berkeley.edu/project_overview.html
And the Kaggle Titanic Survivability dataset
https://www.kaggle.com/c/titanic
But if you desire an even gentler intro. Try Daniel Shiffman's Nature of Code in P5
http://natureofcode.com/
best of luck ;)
https://www.udacity.com/course/intro-to-artificial-intellige...
Machine Learning
https://www.coursera.org/learn/machine-learning
The Pacman programming exercises in python
http://ai.berkeley.edu/project_overview.html
And the Kaggle Titanic Survivability dataset
https://www.kaggle.com/c/titanic
But if you desire an even gentler intro. Try Daniel Shiffman's Nature of Code in P5
http://natureofcode.com/
best of luck ;)
fast.ai - it's free, and there's a low level of assumed knowledge from the outset.
their 'deep learning in excel' [0] is a great simplification of the fundamentals.
[0]: https://www.youtube.com/watch?v=qnoLMkosHuE
[1]: http://www.fast.ai/2016/10/08/teaching-philosophy/
[0]: https://www.youtube.com/watch?v=qnoLMkosHuE
[1]: http://www.fast.ai/2016/10/08/teaching-philosophy/
This.
I have two engineering degrees and studied neural networks in college. He cuts down about 3 months worth of math to a couple of lines of code in such a way that makes it make sense AND is productive. It may just be that his teaching style matches my learning style really well, but I'm enjoying going through it.
I have two engineering degrees and studied neural networks in college. He cuts down about 3 months worth of math to a couple of lines of code in such a way that makes it make sense AND is productive. It may just be that his teaching style matches my learning style really well, but I'm enjoying going through it.
I'd second this. I'm not a "math person" so to speak, and they provide a great education that doesn't dip too deep into the math.
I am sitting in the same boat. Being a web developer for a couple of years, I wanted to try out a different domain. So I started to take Andrew Ng's course on Coursera [0]. Highly recommended. I supplement the course with audio and text by listening to the Machine Learning Guide Podcast [1] and by reading The Master Algorithm [2].
In addition, I started to apply my learnings in JavaScript [3]. Even though it's not the best language for ML, it makes it simpler to learn only one new thing and stick to known technologies for the rest. I have lined up ~7 articles about ML in JavaScript, so if you are interested, you can keep an eye on it :)
- [0] https://www.coursera.org/learn/machine-learning/
- [1] http://ocdevel.com/podcasts/machine-learning
- [2] https://www.goodreads.com/book/show/24612233-the-master-algo...
- [3] https://www.robinwieruch.de/linear-regression-gradient-desce...
In addition, I started to apply my learnings in JavaScript [3]. Even though it's not the best language for ML, it makes it simpler to learn only one new thing and stick to known technologies for the rest. I have lined up ~7 articles about ML in JavaScript, so if you are interested, you can keep an eye on it :)
- [0] https://www.coursera.org/learn/machine-learning/
- [1] http://ocdevel.com/podcasts/machine-learning
- [2] https://www.goodreads.com/book/show/24612233-the-master-algo...
- [3] https://www.robinwieruch.de/linear-regression-gradient-desce...
I got started being into algorithms and then making games with ai opponents like http://bitmultiplayerchess.com, http://wordsmashing.com
I took andrew ng’s coursera machine learning course which i would highly recommend, also his deep learning course is worth it too :)
His course inspired me to create a cryptocurrency trading bot which i spun into a business offering forecasting for altcoin markets: http://BitBank.nz - Crypto Market Predictions with Machine Learning
I managed to make much more successful forecasts by understanding the fundamentals taught in that course like under-fitting and over-fitting and how to visualize whats happening by plotting a learning curve ect.
The forecasting algorithm really just applies the fundamentals thoroughly in perhaps a novel way, e.g. some features we compute at the current time include the linear regression of trades over time weighted by their amount
So its definitely worth the investment i think :) try and apply the teaching to solve a real world problem which i think is the interesting part, although you’ll end up doing a lot of data engineering you’ll savor the AI/ML part even more and start to appreciate strategies for how you can improve your performance in your case and test them out.
Having a play around with the create your own deep neural net at playground.tensorflow.org is pretty helpful, try and conceptualize what youve been taught in the courses by playing around with that, e.g. add more layers/breadth to your network to watch it get more and more powerful and begin to overfit when you add noise to your data ect.
His course inspired me to create a cryptocurrency trading bot which i spun into a business offering forecasting for altcoin markets: http://BitBank.nz - Crypto Market Predictions with Machine Learning
I managed to make much more successful forecasts by understanding the fundamentals taught in that course like under-fitting and over-fitting and how to visualize whats happening by plotting a learning curve ect.
The forecasting algorithm really just applies the fundamentals thoroughly in perhaps a novel way, e.g. some features we compute at the current time include the linear regression of trades over time weighted by their amount
So its definitely worth the investment i think :) try and apply the teaching to solve a real world problem which i think is the interesting part, although you’ll end up doing a lot of data engineering you’ll savor the AI/ML part even more and start to appreciate strategies for how you can improve your performance in your case and test them out.
Having a play around with the create your own deep neural net at playground.tensorflow.org is pretty helpful, try and conceptualize what youve been taught in the courses by playing around with that, e.g. add more layers/breadth to your network to watch it get more and more powerful and begin to overfit when you add noise to your data ect.
Just hijacking this question for my benefit as well. I am a product manager in enterprise focused software. I want to transition to the world of AI. Is Udacity's $600 Deep Learning Nano degree worth it ?
I don't know about Deep Learning Nanodegree, but I took Machine Learning and AI Nanodegrees at Udacity and they are definitely worth it. I would not recommend them to total beginners in the field. You need to have at least some experience with data science and Python to be able to follow along. Do some free courses on Udacity, Coursera, EdX and other platforms, try to implement these algorithms with your own data and problems and then take the Nanodegree to fill the gaps.
Did you find a job after?
I was at the same point as you until I discovered the new Andrew Ng course on deep learning [1]
It's a good structured way to learn the core of ML while learning about Neural Networks and without having to become and linear algebra expert which for most people including like me was a deal breaker with other courses. The timing is great too as ML now is so much different than it was 2-3 years ago.
[1] https://www.coursera.org/specializations/deep-learning
It's a good structured way to learn the core of ML while learning about Neural Networks and without having to become and linear algebra expert which for most people including like me was a deal breaker with other courses. The timing is great too as ML now is so much different than it was 2-3 years ago.
[1] https://www.coursera.org/specializations/deep-learning
I think being effective in ML requires both theory, and practical knowledge you only get by doing.
Andrew Ng's ML course quickly provides a base in theory.
Ideally you couple that with some empirical work.
For that, I think sklearn is the best starting point (assuming you go down the python path). Modify some sample code and make a few simple models. Sklearn provides an excellent framework across all kinds of models (including deep learning if you use say keras.wrappers.scikit_learn), and can play well with pandas.
There are lots of practical concerns that come up that are not covered in intro ML courses.
Andrew Ng's ML course quickly provides a base in theory.
Ideally you couple that with some empirical work.
For that, I think sklearn is the best starting point (assuming you go down the python path). Modify some sample code and make a few simple models. Sklearn provides an excellent framework across all kinds of models (including deep learning if you use say keras.wrappers.scikit_learn), and can play well with pandas.
There are lots of practical concerns that come up that are not covered in intro ML courses.
As a follow up: I want to pursue a math degree study. What course titles and textbooks starting at the calculus level do you guys recommend? I want enough math chomps to then go onto a PhD in ML.
Calculus: Thomas, Weir & Hass - Thomas' Calculus
Linear Algebra (which is what you really need): Gilbert Strang - Linear Algebra and its applications
These two are all you need, with which you'll get a solid base. Then you're good to go on your own. These two combined are about 4 semesters worth of work. But if you really focus, I think you can get them done in a little less than 6 months.
If you want a 'just what I need' approach, Khan Academy.
Linear Algebra (which is what you really need): Gilbert Strang - Linear Algebra and its applications
These two are all you need, with which you'll get a solid base. Then you're good to go on your own. These two combined are about 4 semesters worth of work. But if you really focus, I think you can get them done in a little less than 6 months.
If you want a 'just what I need' approach, Khan Academy.
For self-study, you don't necessarily need to follow the usual progression of math classes that start at calculus. It's more important to get comfortable with linear algebra than calculus, especially the way a lot of intro calculus courses focus on calculating integrals and derivatives. Maybe it's not the best message, but the worst grades in my math degree were in the intro calculus classes.
I don't remember what intro linear algebra books I used, but my college uses this: https://www.math.ucdavis.edu/~linear/ (I took the class before this free textbook was developed).
I don't remember what intro linear algebra books I used, but my college uses this: https://www.math.ucdavis.edu/~linear/ (I took the class before this free textbook was developed).
Spivak calculus. There is nothing like it. You will learn how to think in math, not just calculus.
Whoa! I was just talking about this text book the other day! I think it's out of print these days, but it's a hidden gem!
This Reddit post by a struggling Phd student has some great advice on catching up on maths.
https://np.reddit.com/r/MachineLearning/comments/73n9pm/d_co...
https://np.reddit.com/r/MachineLearning/comments/73n9pm/d_co...
Possibly not what you're looking for (certainly not the cheapest option), but we (Lambda School - YC S17) just announced a live, remote class that trains engineers in AI & ML during weekday evenings for six months.
The next one starts in January, and is taught by an MIT grad that taught a similar course at MIT.
https://lambdaschool.com/artificial-intelligence
The next one starts in January, and is taught by an MIT grad that taught a similar course at MIT.
https://lambdaschool.com/artificial-intelligence
Would you do 20% for a salary of $100k or above?
What percentage would you do for $100k?
What percentage would you do for $100k?
It's still/always 17%, but it would cap out at $20,000 if above ~117k, so it would effectively be less than 17% on an annualized basis.
Or paying up-front/in monthly payments is $1041/month for 12 months.
Or paying up-front/in monthly payments is $1041/month for 12 months.
I've been going through this course: https://www.commonlounge.com/community/9dcdd386cc28446695305...
It's a bit more cursory and mostly just a collection of articles/papers, but it has the benefit of not being paced like a university course.
It's a bit more cursory and mostly just a collection of articles/papers, but it has the benefit of not being paced like a university course.
I like following article as I find it one of the easiest introduction to neural networks:
https://medium.com/technology-invention-and-more/how-to-buil...
https://medium.com/technology-invention-and-more/how-to-buil...
The AI for humans series is some reasonable, high level approach. http://www.heatonresearch.com/aifh/
After you've got a grasp of what these things are doing then you can move into the how. For that you will need some math background, with emphasis in calculus and probability.
After that, you can take a look at PRML. https://www.amazon.com/Pattern-Recognition-Learning-Informat...
Some people might prefer seeing things from another approach. http://pgm.stanford.edu/
Good luck.
After you've got a grasp of what these things are doing then you can move into the how. For that you will need some math background, with emphasis in calculus and probability.
After that, you can take a look at PRML. https://www.amazon.com/Pattern-Recognition-Learning-Informat...
Some people might prefer seeing things from another approach. http://pgm.stanford.edu/
Good luck.
For deep learning, my two favorite nominees are:
1. Hugo Larochelle's Deep Learning course available on YouTube
2. Depending on how much math you like, Nando de Freitas's Deep Learning course (also on YouTube) is also superb.
1. Hugo Larochelle's Deep Learning course available on YouTube
2. Depending on how much math you like, Nando de Freitas's Deep Learning course (also on YouTube) is also superb.
I liked the tutorials at Python Programming, sometimes the python details goes a bit fast and there are typos but over all it's the one that got me most understanding the practical parts.
https://pythonprogramming.net/machine-learning-tutorial-pyth...
https://pythonprogramming.net/machine-learning-tutorial-pyth...
I know you say you'd like to learn online, but I highly recommend picking up Duda and Hart's Pattern Classification to have a theoretical complement to the "hands on", programming type introductions. It's a very accessible intro to the topic, but also covers a lot of material in depth -- in particular, the topics you mention.
I'm currently working through "Deep Learning: A Practitioner's Approach" by Adam Gibson and Josh Patterson. Its a couple of years old but seems like a good book, and I'm certainly learning a lot. It doesn't consider some of the newer tooling, like TensorFlow, but the fundamentals plus a decent amount of theory are all covered.
What makes you say the book is 2 years old? Its publication date is Aug 1, 2017. It's based on Deeplearning4J and ND4J rather than TF, but that doesn't date it necessarily.
You're right, it is August 2017. I bought it from Amazon UK and they list the date as November 2015 [1] for some reason. I should have checked the book before posting. I've flagged the error with Amazon UK.
[1] https://www.amazon.co.uk/Deep-Learning-Practitioners-Adam-Gi...
[1] https://www.amazon.co.uk/Deep-Learning-Practitioners-Adam-Gi...
I can't speak to the rest of the content, but I found the introduction in the course accompanying the recently announced gluon library to be both comprehensive and comprehensible at the same time.
http://gluon.mxnet.io/
http://gluon.mxnet.io/
Whenever you see a new term you don't know about watch/read 3 videos/article on it
I find this helpful. All links in one place https://www.springboard.com/learning-paths/data-analysis/
My recommendation is the fast.ai course by Jeremy Howard. His explanations are amazing and the practical usefulness is immediate.
http://course.fast.ai/
http://course.fast.ai/
In my humble opinion, there's no better way to start than with the classic book - " Artificial Intelligence : A Modern Approach " by Russel Norvig.
Berkeley has an free videos/slides, combined with exams, projects, and homework.
Link:http: //ai.berkeley.edu/home.html
Link:http: //ai.berkeley.edu/home.html
[deleted]
this one starts with simple stuff MIT 6.034 Artificial Intelligence: https://www.youtube.com/watch?v=TjZBTDzGeGg&list=PLUl4u3cNGP...
If you’re genuinely a novice programmer/lesser background in linear algebra, AI should be the last thing on your mind. Any attempts at a shortcut will enhance the difficulty in learning AI, and being able to code things besides simple examples. (which is why I am annoyed by many of the ML MOOCs which are targeted toward novice programmers)
I disagree. There is nothing magic or hard about basic ML. You can do real work with only some basic linear algebra and programming skills. Sure you won't be doing novel deep learning on 100 terabyte datasets, but most problems aren't that anyway.
Yeah, and to be honest while it's always important to be able to understand the math behind what you're doing, you can easily get started with just understanding what these algorithms do and why, and then work to expand your knowledge of the math over time from there.
Why does this annoy you? Some of the MOOCs are amazing and they certainly don't shortcut. I had many hard-learned lessons and realizations when going through the MOOCs I have taken. They also help in "knowing what you don't know"...which is essential in breaking out of ignorance.
I've learned computer science in college, so I do have _some_ linear algebra background. I'm by no means an expert coder, but maybe 'intermediary' would have been a better description than 'novice'. :-)
Elitism isn't too helpful. Determination counts for more than skill.
I say it for the opposite reason of elitism (and I’ve spoken out many time against elitism in ML/AI).
As other posts note, there are many resources available for teaching the concepts. But they don’t teach the limits of AI, and the rise of MOOCs is setting novice programmers up for a shock when they encounter real world data that is not as nice as the Titanic dataset, and requires making smart decisions to handle the data, and handle it in a way that does not invalidate the results.
Many romanticize ML/AI as something that can solve any problem, which is a dangerous approach.
As other posts note, there are many resources available for teaching the concepts. But they don’t teach the limits of AI, and the rise of MOOCs is setting novice programmers up for a shock when they encounter real world data that is not as nice as the Titanic dataset, and requires making smart decisions to handle the data, and handle it in a way that does not invalidate the results.
Many romanticize ML/AI as something that can solve any problem, which is a dangerous approach.
ML is just stats with non-linear systems. No need for programming to do stats, and while linear algebra helps, it is not essential.
Here are the resources I found useful:
==========================================
Advices from Open AI, Facebook AI leaders
Courses You MUST Take:
Machine Learning by Andrew Ng (https://www.coursera.org/learn/machine-learning) /// Class notes: (http://holehouse.org/mlclass/index.html)
Yaser Abu-Mostafa’s Machine Learning course which focuses much more on theory than the Coursera class but it is still relevant for beginners.
(https://work.caltech.edu/telecourse.html)
Neural Networks and Deep Learning (Recommended by Google Brain Team) (http://neuralnetworksanddeeplearning.com/)
Probabilistic Graphical Models (https://www.coursera.org/learn/probabilistic-graphical-model...)
Computational Neuroscience (https://www.coursera.org/learn/computational-neuroscience)
Statistical Machine Learning (http://www.stat.cmu.edu/~larry/=sml/)
From Open AI CEO Greg Brockman on Quora
Deep Learning Book (http://www.deeplearningbook.org/) ( Also Recommended by Google Brain Team )
It contains essentially all the concepts and intuition needed for deep learning engineering (except reinforcement learning). by Greg
2. If you’d like to take courses: Linear Algebra — Stephen Boyd’s EE263 (Stanford) (http://ee263.stanford.edu/) or Linear Algebra (MIT)
(http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebr...)
Neural Networks for Machine Learning — Geoff Hinton (Coursera) https://www.coursera.org/learn/neural-networks
Neural Nets — Andrej Karpathy’s CS231N (Stanford)
http://cs231n.stanford.edu/
Advanced Robotics (the MDP / optimal control lectures) — Pieter Abbeel’s CS287 (Berkeley)
https://people.eecs.berkeley.edu/~pabbeel/cs287-fa11/
Deep RL — John Schulman’s CS294–112 (Berkeley) http://rll.berkeley.edu/deeprlcourse/
Courses You MUST Take:
Machine Learning by Andrew Ng (https://www.coursera.org/learn/machine-learning) /// Class notes: (http://holehouse.org/mlclass/index.html)
Yaser Abu-Mostafa’s Machine Learning course which focuses much more on theory than the Coursera class but it is still relevant for beginners.
(https://work.caltech.edu/telecourse.html)
Neural Networks and Deep Learning (Recommended by Google Brain Team) (http://neuralnetworksanddeeplearning.com/)
Probabilistic Graphical Models (https://www.coursera.org/learn/probabilistic-graphical-model...)
Computational Neuroscience (https://www.coursera.org/learn/computational-neuroscience)
Statistical Machine Learning (http://www.stat.cmu.edu/~larry/=sml/)
From Open AI CEO Greg Brockman on Quora
Deep Learning Book (http://www.deeplearningbook.org/) ( Also Recommended by Google Brain Team )
It contains essentially all the concepts and intuition needed for deep learning engineering (except reinforcement learning). by Greg
2. If you’d like to take courses: Linear Algebra — Stephen Boyd’s EE263 (Stanford) (http://ee263.stanford.edu/) or Linear Algebra (MIT)
(http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebr...)
Neural Networks for Machine Learning — Geoff Hinton (Coursera) https://www.coursera.org/learn/neural-networks
Neural Nets — Andrej Karpathy’s CS231N (Stanford)
http://cs231n.stanford.edu/
Advanced Robotics (the MDP / optimal control lectures) — Pieter Abbeel’s CS287 (Berkeley)
https://people.eecs.berkeley.edu/~pabbeel/cs287-fa11/
Deep RL — John Schulman’s CS294–112 (Berkeley) http://rll.berkeley.edu/deeprlcourse/
This list is solid, and could keep you busy for a few years.
I took a grad ML course based on this book: https://www.amazon.com/dp/B0759M2D9H
It teaches you the foundational theory behind ML, and shows how the fancier stuff is built on it. Good to know the foundations, so you can branch outside of predefined ML techniques.
It teaches you the foundational theory behind ML, and shows how the fancier stuff is built on it. Good to know the foundations, so you can branch outside of predefined ML techniques.
Learn about human intelligence
I'm a intermediate-level programmer, and would like to dip my toes in AI, starting with the simple stuff (linear regression, etc) and progressing to neural networks and the like. What's the best online way to get started?
Thanks!