Dopamine framework: Fast prototyping of reinforcement learning algorithms (2018)(ai.googleblog.com)
ai.googleblog.com
Dopamine framework: Fast prototyping of reinforcement learning algorithms (2018)
https://ai.googleblog.com/2018/08/introducing-new-framework-for-flexible.html
15 comments
CS 294 is a great deep reinforcement learning course that I highly recommend [1]. Sergey Levine is god tier RL pretty much.
[1] http://rail.eecs.berkeley.edu/deeprlcourse/
[1] http://rail.eecs.berkeley.edu/deeprlcourse/
I came across this a few days ago:
For Reinforcement Learning specifically, the standard text is Reinforcement Learning: An Introduction[1]. Dave's UCL Course on RL[2] is great too (playlist of all lectures)[3].
Source: Julian Schrittwieser works on Deepmind at Google http://www.furidamu.org/
[1]http://incompleteideas.net/book/the-book-2nd.html
[2]http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html
[3]https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PL7-jPKtc4r...
Source: Julian Schrittwieser works on Deepmind at Google http://www.furidamu.org/
[1]http://incompleteideas.net/book/the-book-2nd.html
[2]http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html
[3]https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PL7-jPKtc4r...
Beyond those, my favorite resources are:
- "An Introduction to Deep Reinforcement Learning" by Vincent François-Lavet et al (https://arxiv.org/pdf/1811.12560.pdf)
- "A (Long) Peek into Reinforcement Learning" by Lilian Weng (https://lilianweng.github.io/lil-log/2018/02/19/a-long-peek-...)
- "Deep Reinforcement Learning: Pong from Pixels" from Andrej Karpathy (https://karpathy.github.io/2016/05/31/rl/)
Those are the basics. Some more resources listed on this post: https://news.ycombinator.com/item?id=18219620
- "An Introduction to Deep Reinforcement Learning" by Vincent François-Lavet et al (https://arxiv.org/pdf/1811.12560.pdf)
- "A (Long) Peek into Reinforcement Learning" by Lilian Weng (https://lilianweng.github.io/lil-log/2018/02/19/a-long-peek-...)
- "Deep Reinforcement Learning: Pong from Pixels" from Andrej Karpathy (https://karpathy.github.io/2016/05/31/rl/)
Those are the basics. Some more resources listed on this post: https://news.ycombinator.com/item?id=18219620
Thanks for this list. I’d like to get up to speed on RL and see how we can apply it for path planning and control of our gliding parachute UAVs.
Ooh! I want one of those for high altitude balloon payload recovery. Got any small ones or hobby grade projects doing this that you’re aware of?
No hobby ones I know of, although I haven't looked into the hobby space too much. All I can say about smaller systems is give our BD guys a call... =D
thanks!!
If you are not an expert I can recomment David Silver's lectures on the topic. There is a playlist on youtube (https://www.youtube.com/playlist?list=PLzuuYNsE1EZAXYR4FJ75j...) and from what I have seen it is quite well explained.
This goes over the theory behind RL mostly, so if you are looking for practical implementations you might want to find other resources.
This goes over the theory behind RL mostly, so if you are looking for practical implementations you might want to find other resources.
thanks, I will look at Silver’s lectures. I have not finished reading Sutton and Barto’s RL book yet, so I am still a beginner.
RL is really really brittle, i.e. one simple bug that wouldn't matter in GAN/LSTM (such as e.g. adding one more or less number than needed) would cause your RL algorithm not to converge and you'd spend days-weeks debugging it - it requires extreme discipline verifying every single step of your algorithm. Often you don't know if your model didn't converge because it is not strong enough or if there was some tiny little error/wrong assumption leading to divergence.
Yes, some types of NNs are resilient to errors in implementation.
I hired David Rumelhart as a consultant in the 1980s (when he was still at UCSD) when I was writing SAIC’s neural network product for Windows. He warned me that he was used to seeing projects with big implementation errors that still worked very well.
I do find GANs brittle. I use them for generating numeric/categorical spreadsheet data, not for images.
I hired David Rumelhart as a consultant in the 1980s (when he was still at UCSD) when I was writing SAIC’s neural network product for Windows. He warned me that he was used to seeing projects with big implementation errors that still worked very well.
I do find GANs brittle. I use them for generating numeric/categorical spreadsheet data, not for images.
> I do find GANs brittle. I use them for generating numeric/categorical spreadsheet data, not for images
I’m interested to hear more about this. Care to elaborate?
I’m interested to hear more about this. Care to elaborate?
> He warned me that he was used to seeing projects with big implementation errors that still worked very well.
For any sufficiently complex function, if you tune the constants enough, the wrong function will sometimes approximate the results the correct one would give if it were set up with the correct constants.
For any sufficiently complex function, if you tune the constants enough, the wrong function will sometimes approximate the results the correct one would give if it were set up with the correct constants.
May I ask out of curiosity what commercial application are you using GANs for if any?
I especially like that this framework is set up for experimenting with four popular models and the means to measure how well your models are doing.