Practical Deep Learning for Coders(course.fast.ai)
course.fast.ai
Practical Deep Learning for Coders
http://course.fast.ai/
38 comments
I took both of these courses and recommend. Practical is the key word in their description that differentiates them from other courses. The focus is on getting you up and running ASAP, then explaining the details and variation, then finally introducing you to learning how to translate the newest papers into usable code. You're not going to get a PhD from the course, but it will get you started on real-life projects.
You might get an MSc, however :P Half of my master's thesis was based on practical understanding and skills I learned from the first Fast.ai course. It's also inspired me to start a PhD in a more quantitative field to persue probabilistic models in medicine.
(Here's that's master's thesis if you're interested :) Warning, big PDF ahead: https://www.cs.mcgill.ca/~jszymb/thesis/260528685_Szymborski...)
(Here's that's master's thesis if you're interested :) Warning, big PDF ahead: https://www.cs.mcgill.ca/~jszymb/thesis/260528685_Szymborski...)
Same here! Still busy with my MSc thesis though because I'm too busy applying some of the other stuff I learned from Fast.ai on other projects ':)
How long did it take you to complete each course?
Do you have a recommended 'schedule'?
I have tried twice, but have not finished.
Do you have a recommended 'schedule'?
I have tried twice, but have not finished.
Try a schedule that keeps you on ~ two hours a day, focusing on at least one of the videos in the 14-video series. So: 14 weeks, paying attention to one video a week and whatever peripheral studies you choose to follow up on based on the ideas Jeremy pushes in his lectures.
Edit: also, find datasets online that fit in your interests. There's no better motivation than to solve problems in a domain that you're already familiar with. I grew up on a farm and built a classifier to detect diseases as part of my fast.ai course-work: https://t.me/shambadoctorbot
Edit: also, find datasets online that fit in your interests. There's no better motivation than to solve problems in a domain that you're already familiar with. I grew up on a farm and built a classifier to detect diseases as part of my fast.ai course-work: https://t.me/shambadoctorbot
Ok, from my understanding deep learning is actuially a fairly complicated subject, a bit more than learning the JavaScript framework of the month.
So is there any point in treating it as such? Just another skill we should all learn superficially, so that we have a lot of very shallow knowledge?
So is there any point in treating it as such? Just another skill we should all learn superficially, so that we have a lot of very shallow knowledge?
It depends on your goal. If you want to make advancements in the field and find innovations that no one else has done (e.g. GANs and Reinforcement Learning), then a superstrong mathematical background is helpful. But if you just want to find the best output given a set of inputs and tune things for business needs, modern tooling like sklearn and TensorFlow/Keras is more than sufficient, and following a set of modern data transformation and model heuristics can get you 80% of the way there.
Medium thought-pieces/YouTubers conflate both perspectives, which has become a problem.
Medium thought-pieces/YouTubers conflate both perspectives, which has become a problem.
cough siraj raval cough
The advantage of learning lots of “deep” skills superficially is that it gives you the perspective to have a better idea of which ones you should dive more deeply into to address particular real-world problems. (It may even give you the tools to apply some of them to the real world problems they are most simply applicable to without further study, as a bonus.)
I've taken the first part of this (not the newest version using Pytorch, tho I'd like to retake someday soon), and found this class to be extremely useful. I do agree that in order to be optimally efficient you'd want to have a background understanding of statistics/linear algebra; however, I love this class because it gets you up and running very quickly and excited about the content.
It's also my understanding that the ML stack is making it more and more accessible to actually apply these algorithms. Sure, after you take this class you probably won't go out and invent new algorithmic breakthroughs, but many people in this class do participate in in Kaggle competitions solving real world problems (some of the assignments are submissions) and do very well, even winning occasionally.
The point of the class is to generate interest and to get people useful quickly (make neural nets uncool again), it'd be much harder to get the field to blossom if you required people to start with years of stats/math beforehand.
It's also my understanding that the ML stack is making it more and more accessible to actually apply these algorithms. Sure, after you take this class you probably won't go out and invent new algorithmic breakthroughs, but many people in this class do participate in in Kaggle competitions solving real world problems (some of the assignments are submissions) and do very well, even winning occasionally.
The point of the class is to generate interest and to get people useful quickly (make neural nets uncool again), it'd be much harder to get the field to blossom if you required people to start with years of stats/math beforehand.
I think that just like most people don't need to know about the extremely complex internals of databases or networks, we should get deep-learning systems to a point where users can treat them the same way.
(Of course there's always value in knowing the details -- it helps to strive towards lowering the barrier to entry into complex systems.)
(Of course there's always value in knowing the details -- it helps to strive towards lowering the barrier to entry into complex systems.)
I wish people would learn to use relational databases properly. There is a lot of crap out in the wild.
They promise to teach you deep learning, not to master it.
For many people, this is a good primer into basic concepts and how deep learning works from a high level. If they are still interested then they can work towards mastery. But you have to start somewhere.
For many people, this is a good primer into basic concepts and how deep learning works from a high level. If they are still interested then they can work towards mastery. But you have to start somewhere.
This course is certainly not superficial. I have taken it as an international fellow. I suggest you do the first two lectures to get a feel for it.
> actually a fairly complicated subject
This is heavily dependent on your background. Deep learning is essentially the simplest thing that could possibly work (parameterized nonlinear function + SGD).
If you have experience with calculus roughly equivalent to an undergrad Calc 3, which is required for most fields of engineering, the core ideas of deep learning are very accessible to you without much effort.
After that, getting it to work is mostly an empirical and experimental endeavor.
(This refers to deep learning as used in practice, not some of the current research topics).
This is heavily dependent on your background. Deep learning is essentially the simplest thing that could possibly work (parameterized nonlinear function + SGD).
If you have experience with calculus roughly equivalent to an undergrad Calc 3, which is required for most fields of engineering, the core ideas of deep learning are very accessible to you without much effort.
After that, getting it to work is mostly an empirical and experimental endeavor.
(This refers to deep learning as used in practice, not some of the current research topics).
Sometimes, deep learning is too simple to actually work. (complex games with hidden state, exploration, generating art, decision making etc.)
Those use cases are in constant flux and research. (Though teams like DeepMind are making some serious inroads.)
Other problem is that DL is still highly sample inefficient - it needs many more examples than you'd want or sometimes even have to bootstrap.
Those use cases are in constant flux and research. (Though teams like DeepMind are making some serious inroads.)
Other problem is that DL is still highly sample inefficient - it needs many more examples than you'd want or sometimes even have to bootstrap.
You still need a business or research case and then a dataset as any other skill out there, don’t you?
Some of the best instruction I've ever received (and I have too many degrees). These courses are intensely practical. Making sure people understand the theoretical underpinnings comes second to making sure people can actually use DL/ML software for their own purposes.
The site could use a reorganization, though. Jeremy and Rachel have created a number of courses, but you have to hunt through their YouTube and fast.ai forum comments to find all of them. So much good content should not be buried!
The site could use a reorganization, though. Jeremy and Rachel have created a number of courses, but you have to hunt through their YouTube and fast.ai forum comments to find all of them. So much good content should not be buried!
If you're looking for a group to take this course with (Jeremy recommends it [1]), we just started a Fall session of the TWiML Online Meetup's Fast.ai study group. More info here: http://twimlai.com/fastai
[1] https://twitter.com/jeremyphoward/status/996445183456690176
[1] https://twitter.com/jeremyphoward/status/996445183456690176
What are the math prerequisites? Couldn't find anything in about page [0], all it says is:
> We assume that everyone taking this course has at least one year of coding experience.
so, no math background required? Would really appreciate an input from someone who has done the course
[0] - http://course.fast.ai/about.html
> We assume that everyone taking this course has at least one year of coding experience.
so, no math background required? Would really appreciate an input from someone who has done the course
[0] - http://course.fast.ai/about.html
There is a Computational Linear Algebra class taught by one of the founders of fast.ai, but I don't think it's a prerequisite necessarily [0]. I've been meaning to take it - I can't comment on its quality.
[0] https://github.com/fastai/numerical-linear-algebra/blob/mast...
[0] https://github.com/fastai/numerical-linear-algebra/blob/mast...
I forget where but it used to say roughly a high school level of math understanding. You don't have to do a lot of math in the program but it can be helpful for understanding the concepts. I am very poor at math ( never did much math past high school and that ended in 2009 ), and was able to work through the course, though the course in general is not easy.
I really recommend 3Blue1Brown youtube series on linear algebra if you are struggling with some of the math concepts. They have great explanations and great visualizations.
I really recommend 3Blue1Brown youtube series on linear algebra if you are struggling with some of the math concepts. They have great explanations and great visualizations.
Maybe just start watching it if you are interested in learning, instead of trying to put yourself in a box that you might not have the math skills needed for it.
It is free, you have nothing to lose. If the topic interests you than I am sure you will figure it out. You can always google a math concept that confuses you. There isn't a test and no one will even know.
It is free, you have nothing to lose. If the topic interests you than I am sure you will figure it out. You can always google a math concept that confuses you. There isn't a test and no one will even know.
> It is free
Just as a warning for anyone price-sensitive interested in taking this course, its cost is non-zero due to cloud costs. It is free to audit though! I'm not trying to dissuade anyone; it is just good to know going into it.
Just as a warning for anyone price-sensitive interested in taking this course, its cost is non-zero due to cloud costs. It is free to audit though! I'm not trying to dissuade anyone; it is just good to know going into it.
I did manage to setup the notebook and cuda on my local machine. I have an older GTX card.
That is true for any deep learning course. You need a GPU to run SOTA algorithms.
We can use Google Colab instead, right? They're free.
> we teach "top down" rather than "bottom up". For instance, you'll learn how to use deep learning to solve your problems in week 1, but will only start to learn why it works in week 2
That sounds more like bottom up to me (practice before theory).
That sounds more like bottom up to me (practice before theory).
I think they mean it in the sense that:
Theoretical foundations --> practical problems = Bottom Up,
practice without theory --> building up the theoretical legs to stand on == Top Down
Theoretical foundations --> practical problems = Bottom Up,
practice without theory --> building up the theoretical legs to stand on == Top Down
Has anyone here actually built an ml product? A product people use?
I have built computer vision products at my startup. I was familiar with some machine learning before, but only really learned to apply it on the job. Fast forward 7 years and the initial product and others that followed have been very successful. Machine learning is only a part of the whole product though.
That's setting the goalposts a bit narrow in terms of practical applications of ML. You can apply deep learning to solve problems aren't public facing, such as analytics forecasting.
Additionally, a lot of public ML products don't use, and don't need to use, deep learning (e.g. NLP applications).
Additionally, a lot of public ML products don't use, and don't need to use, deep learning (e.g. NLP applications).
Analytics forecasting doesn't really seem like a good candidate for deep learning. There are plenty of established methods for forecasting that are simpler, more robust and generally more effective in terms of effort to reward.
Google Photos. It has great search over collections of pictures. You can search for pictures of dogs, sunsets, trees, even pictures of store receipts (which is super convenient for filing expense reports).
iOS photos does this natively in search, too.
How do you mean? As a side project? I've seen and built several ML products that are used widely inside Amazon atleast.
I don’t understand the big deal about state of the art result. If you know the algorithm (from paper) and the high level tool is available (pytorch) then it is not difficult. If you find it difficult then you are not a good enough software developer and you are not able to produce anything substantial. With fast ai you would mostly end up burning energy in gpu and waste your time. I would better learn c++ then deep learning.