Free “Deep Learning” Textbook by Goodfellow and Bengio Now Finished(facebook.com)
facebook.com
Free “Deep Learning” Textbook by Goodfellow and Bengio Now Finished
https://www.facebook.com/ia3n.goodfellow/posts/10102223910143043
133 comments
If anything, to me a lot of deep learning literature seems to lack the statistical insight and theory that's available to other subfields in machine learning (whether the Bayesian/graphical models camp, the statistical learning theory camp...)
If this book is trying to do more to bring statistical or probabilistic insights to bear on deep learning than I think that's a very good thing. It might make it less accessible to those coming from a pure computer science background, but potentially more so to those who like to think about machine learning from a probabilistic modelling perspective.
If they're using stats jargon in a gratuitous way that doesn't actually cast any light on the material then that's another thing, but from a quick skim I didn't see anything particularly bad on this front. Do you have any examples of the kind of jargon you're talking about?
To others reading, I just wanted to emphasise that statistics is really important in machine learning! Deep learning lets you get away with less of it than you might need elsewhere, but that doesn't mean one can treat it as an unnecessary inconvenience. It's a language you need to learn, especially if you want to try and get to the bottom of how and why aspects of deep learning work the way they do. As opposed to just an empirical "using GPU clusters to throw lots of clever shit at the wall and see what sticks" engineering field. Bengio seems very interested in these kinds of questions and I'm glad he's leading research in that direction, even if clear answers and intuition aren't always easy to come by at this point.
If this book is trying to do more to bring statistical or probabilistic insights to bear on deep learning than I think that's a very good thing. It might make it less accessible to those coming from a pure computer science background, but potentially more so to those who like to think about machine learning from a probabilistic modelling perspective.
If they're using stats jargon in a gratuitous way that doesn't actually cast any light on the material then that's another thing, but from a quick skim I didn't see anything particularly bad on this front. Do you have any examples of the kind of jargon you're talking about?
To others reading, I just wanted to emphasise that statistics is really important in machine learning! Deep learning lets you get away with less of it than you might need elsewhere, but that doesn't mean one can treat it as an unnecessary inconvenience. It's a language you need to learn, especially if you want to try and get to the bottom of how and why aspects of deep learning work the way they do. As opposed to just an empirical "using GPU clusters to throw lots of clever shit at the wall and see what sticks" engineering field. Bengio seems very interested in these kinds of questions and I'm glad he's leading research in that direction, even if clear answers and intuition aren't always easy to come by at this point.
Phew, I thought it was just me.
I found http://neuralnetworksanddeeplearning.com/ far more accessible.
I found http://neuralnetworksanddeeplearning.com/ far more accessible.
This is the first time I have ever heard anyone say that a text book is harder to understand and is less clear than research papers.
Journal papers are the pinnacle of unclear, convoluted context depend writing I know of.
This is an extremely damning critique.
Journal papers are the pinnacle of unclear, convoluted context depend writing I know of.
This is an extremely damning critique.
Except for conference papers, which are even worse. There's so much "secret sauce", only few are independently reproducible. Sadly, in some research areas they are the dominant way of publishing research output.
Do you know of any books that don't have the problems you speak of?
That's disappointing. I was really looking forward to reading the book.
You still should, I think. I mean, at least give it a try. Different people can have completely different experiences when reading the same material. Perhaps you will love it, perhaps you will agree with the OP. In any event, it's worth getting it straight from the horse's mouth.
I agree. Everyone should give it a try if they're genuinely interested in learning it. But if you find yourself struggling with it, know you're not the only one :)
Statistics [edit: and especially ML] is different from math. It is important to let go of one's preconceived notions and to try to absorb the new field as is. Its jargon exists for a reason --- the normal math jargon doesn't quite have the right concepts --- so it is worthwhile spending the effort to understand it.
Yeah, but I did my PhD in theoretical computer science and have published research papers in ML. So I shouldn't have such a hard time. (It could just be me, but maybe it's a weak signal)
What are you favorite books on the subject?
>And I am an advanced math geek.
For those that may not know, j2kun also writes an on-going expository machine learning series on his mathematics blog[1].
[1] http://jeremykun.com/2012/08/04/machine-learning-introductio...
For those that may not know, j2kun also writes an on-going expository machine learning series on his mathematics blog[1].
[1] http://jeremykun.com/2012/08/04/machine-learning-introductio...
>> it replaces very simple explanations of concepts with complicated paragraphs that I can't make sense of
It is good to see critical views but it will be even better if you could give concrete examples for statements like the above. Also, what other books do you recommend?
It is good to see critical views but it will be even better if you could give concrete examples for statements like the above. Also, what other books do you recommend?
> Also, what other books do you recommend?
This has been my frustration. I've been wanting to grok deep learning, but haven't found any source that can explain it in a way that doesn't overcomplicate simple things (which I can only tell it's doing when I already understand the topic). I also don't have the time or incentive to really dig in and do the math myself from scratch, since so much of it is wide open research directions.
I've also had this experience with much much simpler areas of statistics and statistical ML (cf. Markov chain monte carlo), so this is a sort of recurring theme for me. Considering how simple MCMC is now that I do understand it, it's difficult to dispel the nagging feeling that all the statistical ML literature is (likely unintentionally) obfuscated.
I could give concrete examples of excerpts and entire sections of the book that don't make sense to me, but I don't think it's all that productive because a lot of it boils down to organizational disagreements, cultural assumptions behind the math, and differences in priorities. Individually it seems like nitpicking, but they add up quickly to general muddled confusion. This is especially true when, for example, almost always the right answer to the question "Why does this particular technique work?" is, "We have no clue, but here is some anecdotal evidence and half-substantiated oversimplified theories." Instead these answers are passed as well-known fact.
This has been my frustration. I've been wanting to grok deep learning, but haven't found any source that can explain it in a way that doesn't overcomplicate simple things (which I can only tell it's doing when I already understand the topic). I also don't have the time or incentive to really dig in and do the math myself from scratch, since so much of it is wide open research directions.
I've also had this experience with much much simpler areas of statistics and statistical ML (cf. Markov chain monte carlo), so this is a sort of recurring theme for me. Considering how simple MCMC is now that I do understand it, it's difficult to dispel the nagging feeling that all the statistical ML literature is (likely unintentionally) obfuscated.
I could give concrete examples of excerpts and entire sections of the book that don't make sense to me, but I don't think it's all that productive because a lot of it boils down to organizational disagreements, cultural assumptions behind the math, and differences in priorities. Individually it seems like nitpicking, but they add up quickly to general muddled confusion. This is especially true when, for example, almost always the right answer to the question "Why does this particular technique work?" is, "We have no clue, but here is some anecdotal evidence and half-substantiated oversimplified theories." Instead these answers are passed as well-known fact.
Have you read the acclaimed "Elements of Statistical Learning"[1] or its lighter weight counterpart, "Intro to Statistical Learning"[2]?
I think both explain statistical ML clearly at just the right level of detail. But I'm not an advanced math person, so it's certainly possible that I've failed to notice any flaws in their approach.
[1] https://web.stanford.edu/~hastie/local.ftp/Springer/OLD/ESLI...
[2] http://www-bcf.usc.edu/~gareth/ISL/
I think both explain statistical ML clearly at just the right level of detail. But I'm not an advanced math person, so it's certainly possible that I've failed to notice any flaws in their approach.
[1] https://web.stanford.edu/~hastie/local.ftp/Springer/OLD/ESLI...
[2] http://www-bcf.usc.edu/~gareth/ISL/
I haven't looked too deeply at this particular book yet, but your overall sentiment is sounds very familiar.
Too many people are willing to accept (and defend!) sub-par explanations and overcome them through titanic mental grind. The scary part of it is that often this leaves you with a broken mental model that continues to require tons of effort in application. Meanwhile, much better explanations exist. They make learning easy and fun, while also giving you intuition that is easy to apply and even expand to other areas.
I am pretty good at spotting bad mental models, but it's really hard to prove that they are bad, until you find one clearly superior. Recently I stumbled upon MIT's linear algebra class at Open Courseware and was blown away by how much easier it was to follow than the stuff I had in college, while covering the same material in much greater detail.
Too many people are willing to accept (and defend!) sub-par explanations and overcome them through titanic mental grind. The scary part of it is that often this leaves you with a broken mental model that continues to require tons of effort in application. Meanwhile, much better explanations exist. They make learning easy and fun, while also giving you intuition that is easy to apply and even expand to other areas.
I am pretty good at spotting bad mental models, but it's really hard to prove that they are bad, until you find one clearly superior. Recently I stumbled upon MIT's linear algebra class at Open Courseware and was blown away by how much easier it was to follow than the stuff I had in college, while covering the same material in much greater detail.
The new O'Reilly book "Fundamentals of Deep Learning" by Nikhil Buduma (available on Safari for a while now) is good at the fundamentals- very clearly explained, nice diagrams. It it relatively close to the path of my Neural Networks classes (although those were 20 years ago).
It necessarily shorter on detail in terms of the tricks of implementation that have radically improved performance of these techniques over the past 5 years, and might serve as a good read before diving into this via Goodfellow, Bengio, and Courville.
It necessarily shorter on detail in terms of the tricks of implementation that have radically improved performance of these techniques over the past 5 years, and might serve as a good read before diving into this via Goodfellow, Bengio, and Courville.
I haven't seen this one before. Looks like it's still in development.
Yes, only the first three chapters are released.
It is valuable to cite examples because it will help others determine if they have the same organizational disagreements and cultural assumptions that would make the book burdensome to read.
On the other hand, if we find the cited examples clear due to e.g. different backgrounds, expectations, perspectives, etc, then that might be a good signal that we should look into the book.
On the other hand, if we find the cited examples clear due to e.g. different backgrounds, expectations, perspectives, etc, then that might be a good signal that we should look into the book.
You know, if you wanted to write a book, you could set up a Patreon with a per-chapter benefit. Perhaps write the introduction for free, setting out your goals for the book. I'd chip in.
Even if the material is already 'covered' elsewhere, simply re-explaining the same concepts would do a world of good for those who think in the same way as you.
Even if the material is already 'covered' elsewhere, simply re-explaining the same concepts would do a world of good for those who think in the same way as you.
[deleted]
Part of the problem in writing a deep learning book, is that very little that warrants being in a book, is actually known. The mainstream deep learning academic community welcomes theoretical work but articles on new techniques which beat SOTA are given much more attention than articles on setting up a theoretical structure of the models. Consequently, that line of research is also under-cited and under-developed. In some ways, 'deep learning' is in a different Kuhnian paradigm altogether. So, people who are used to learning about all the intricacies of classical ML models fail to appreciate deep learning, because the metrics with which you would judge a deep learning model are different from the ones you would use to judge a classical ML model. The same applies for the books in different Kuhnian paradigm.
>The mainstream deep learning academic community welcomes theoretical work but articles on new techniques which beat SOTA are given much more attention than articles on setting up a theoretical structure of the models.
The fact that beating an accuracy test on some arbitrary data set is seen as more important than understanding why and when your methods works (or do not work) is deeply disturbing.
The fact that beating an accuracy test on some arbitrary data set is seen as more important than understanding why and when your methods works (or do not work) is deeply disturbing.
It's not an arbitrary dataset. To be taken seriously you need to beat SOTA on several standard benchmark datasets where only reaching SOTA on one is already hard.
Or alternatively, reach comparable level of validation accuracy with significantly less compute or memory usage (computational complexity).
Or alternatively, reach comparable level of validation accuracy with only a subset of labeled samples + unlabeled samples (sample complexity).
Or alternatively, reach comparable level of validation accuracy with significantly less compute or memory usage (computational complexity).
Or alternatively, reach comparable level of validation accuracy with only a subset of labeled samples + unlabeled samples (sample complexity).
That still amounts to publishing when you can say, "We know it does work" as opposed to "We know why it works".
...which is very valuable. After all, we don't know when, if ever, a satisfactory explanation (by what criteria?) will derive. To defer all progress in the field until it should meet the standards of some other field would be a great loss, and might preclude any number of future further discoveries.
As far as I understand the concept of "science", an empirical observation (including "it works from an engineering perspective, kinda") is only ever an observation. Scientific progress, by definition, involves advancing our understanding of hows and whys.
I, too, spent some time reading this book, and arrived at the conclusion that it's terribly written.
From a mathematical perspective, the book does not define hardly anything. Take, for example, the important chapter on Deep Feedforward Networks. They hem and haw saying things like "quintessential deep learning models" but never actually explain _what a deep feedforward network is_. There are no definitions: they give what amounts to preamble of a definition, and then move right on to using the notion, but don't actually give the definition itself. In the deep feedforward network example, the closest thing to a definition is, "A feedforward network defines a mapping y=f(x;theta)". That doesn't really say anything though. If you're asked to define "dog", you can say, "A dog has four legs", but how does the reader know that a cat isn't a dog?
I find that for basic machine learning it's much better to just go through machine learning framework tutorials. For advanced learning, I'm not there yet so I can't speak from experience but I'll be shocked if this book fits well in any role except maybe last-resort reference material. If that.
From a mathematical perspective, the book does not define hardly anything. Take, for example, the important chapter on Deep Feedforward Networks. They hem and haw saying things like "quintessential deep learning models" but never actually explain _what a deep feedforward network is_. There are no definitions: they give what amounts to preamble of a definition, and then move right on to using the notion, but don't actually give the definition itself. In the deep feedforward network example, the closest thing to a definition is, "A feedforward network defines a mapping y=f(x;theta)". That doesn't really say anything though. If you're asked to define "dog", you can say, "A dog has four legs", but how does the reader know that a cat isn't a dog?
I find that for basic machine learning it's much better to just go through machine learning framework tutorials. For advanced learning, I'm not there yet so I can't speak from experience but I'll be shocked if this book fits well in any role except maybe last-resort reference material. If that.
First impressions:
1. It also covers "classical" artificial neural networks, i.e., things like backprop from before Hinton and others made breakthroughs for deep learning. This means you can start with this book even if you are new to ANNs. The later sections cover "real deep learning".
2. The language is great for beginners and users. You don't have to be an advanced math geek to follow everything. They seem to cover a fair amount of ground too, so its not dumbed down either.
3. I guess it covers most of the underlying theory and practical technicques but is implementation neutral. You should probably pick up a tutorial for your favorite implementation like Theano, TensorFlow, etc.
All in all, I like it a lot.
1. It also covers "classical" artificial neural networks, i.e., things like backprop from before Hinton and others made breakthroughs for deep learning. This means you can start with this book even if you are new to ANNs. The later sections cover "real deep learning".
2. The language is great for beginners and users. You don't have to be an advanced math geek to follow everything. They seem to cover a fair amount of ground too, so its not dumbed down either.
3. I guess it covers most of the underlying theory and practical technicques but is implementation neutral. You should probably pick up a tutorial for your favorite implementation like Theano, TensorFlow, etc.
All in all, I like it a lot.
The old backprop from the 80's / 90's is still in use and the primary way to train deep nets. We tend to call it SGD (on a composition of differentiable functions) nowadays but it's the same algorithm.
This looks interesting, can't wait to dig into it.
Another great great free online book on this topic: http://neuralnetworksanddeeplearning.com/
Another great great free online book on this topic: http://neuralnetworksanddeeplearning.com/
I'm working my way through that at the moment and so far it's pitched rather well (at least for my prior experience), describing things simply and concisely but still with enough background that it's easy enough to see where everything fits.
+1 for that recommendation, a really good resource.
For anyone interested, Goodfellow is answering questions about the book at: https://www.reddit.com/r/MachineLearning/comments/4domnk/the...
I don't claim to have a solution, but these models of book monetisation really seem doomed. What are the chances that I will buy this book just because they made it artificially harder for me to download it? Probably a net negative.
[deleted]
Athena (http://athenapdf.com/) does a phenomenal job at turning those HTML pages into convenient PDF files.
spent lot of time to convert each pages using this website and merged the pdfs into single pdf and then found that Athena has not done the conversion correctly. The diagrams not converted correctly.
Waste of time!!
Waste of time!!
Does this book mention attention models?
PDF version is: http://www.deeplearningbook.org/front_matter.pdf
Unfortunately, that's just the ToC and the bibliography. It looks like their "contract with MIT Press forbids distribution of too easily copied electronic formats of the book."
Does that mean there's a fifteen-step asciidoc/gitbook/LaTeX build that is freely distributed? Because I'm willing to spend a few minutes building a pdf for the sake of getting good content.
It's pretty easy to just run "Save as PDF" on each chapter individually, and then stitch them together with:
pdftk $(ls -tr *.pdf) cat output DeepLearningBook.pdf
In case anyone else has trouble with the "Save as PDF" part, I did it successfully using Firefox on OSX: from the print menu, choose "show details" and change all headers/footers to "--blank--" so there's no ugly URLs/dates in the corners, then press PDF -> "Save as PDF". (In Chrome, on the other hand, saving/printing as PDF chopped each page into quarters for me...) At least in the one chapter I've saved so far, all the math notation renders just like in the HTML version.
That would give you a PDF version of the web page, right? Wouldn't you still have problems with "some things like subscript expressions do not render correctly"?
Unfortunately, according to the site, the HTML pages themselves have problems, such as missing parens and incorrect math symbols.
Doubt it. We talked to MIT Press about publishing our book and they were extremely hostile to having any way for our readers to produce an un-DRM'd PDF, which is how our book is currently distributed.
Actually, they were mostly just very hostile and unpleasant. This is among the many reasons why we're probably staying independent and printing the book ourselves - so that the digital version stays unencumbered among other things.
Actually, they were mostly just very hostile and unpleasant. This is among the many reasons why we're probably staying independent and printing the book ourselves - so that the digital version stays unencumbered among other things.
Ugh, MIT Press, the same organization that won't allow easy distribution of things like the Wizard Book.
https://mega.nz/#!vZVjlazb!rWcCCC7iv1RI9ssjdUFj7iTpphUCIFQY3...
Preview: http://i.imgur.com/5OXv28R.png
Call me evil but I did it:p
Preview: http://i.imgur.com/5OXv28R.png
Call me evil but I did it:p
[deleted]
Somebody please package the html into a pdf!
Hacky solution:
On a suitably equipped unix-like system one might...
edit: changed ".pdf" from a slightly longer approach to ".html" which actually exists in this workflow. Thanks @TheCabin!
edit(2): ... and gave a valid path to this listdir. Check before I post... check before I post...
edit(3): ... and removed the os.listdir line no longer needed in this approach. Gosh. Just ignore what I was saying and build your own approach. That'll probably be faster at this rate.
edit(4): Don't import os without using it.
mkdir dlbook;cd dlbook;wget --recursive --level=1 http://www.deeplearningbook.org/
cd www.deeplearningbook.org/contents
python
import pdfkit
pdfkit.from_file(
["TOC.html","acknowledgements.html","notation.html","intro.html","part_basics.html","linear_algebra.html","prob.html","numerical.html","ml.html","part_practical.html","mlp.html","regularization.html","optimization.html","convnets.html","rnn.html","guidelines.html","applications.html","part_research.html","linear_factors.html","autoencoders.html","representation.html","graphical_models.html","monte_carlo.html","partition.html","inference.html","generative_models.html","bib.html","index-.html"],
"Goodfellow-et-al-2016-Book.pdf")
edit: changed ".pdf" from a slightly longer approach to ".html" which actually exists in this workflow. Thanks @TheCabin!
edit(2): ... and gave a valid path to this listdir. Check before I post... check before I post...
edit(3): ... and removed the os.listdir line no longer needed in this approach. Gosh. Just ignore what I was saying and build your own approach. That'll probably be faster at this rate.
edit(4): Don't import os without using it.
Just found this gist: https://gist.github.com/luoyetx/a44eea84272123f608dcf737588c...
Avoids the awkward file system structure by using pdfkit.from_url, but creates one .pdf for each chapter. I tried using a list of urls, but pdfkit failed because my version of wkhtmltopdf did not accept multiple input files.
Edit: pdfkit.from_file also fails on my system when passing multiple files. If that works for you, multiple urls are probably fine, too.
Avoids the awkward file system structure by using pdfkit.from_url, but creates one .pdf for each chapter. I tried using a list of urls, but pdfkit failed because my version of wkhtmltopdf did not accept multiple input files.
Edit: pdfkit.from_file also fails on my system when passing multiple files. If that works for you, multiple urls are probably fine, too.
Only returns a blank pdf page for me...
In my case too. I guess it is related to the wkhtmltopdf version and the woff fonts.
For what it's worth I got pretty decent conversion results of one file at a time with the Ubuntu repository version (0.9.9)
But that version doesn't work with collecting it into a single output file, no.
I think it was a mistake to try shaving off a couple of lines and a step at the price of a more convoluted install and a brittle process. Most people would probably be best off just converting one HTML file at a time to pdf (e.g. pdfkit.from_file(filename_in_html,filename_in_html[:-4]+".pdf") in some sort of iteration over the file names) and then concatenating the resulting PDF:s, for instance from command line by
I think it was a mistake to try shaving off a couple of lines and a step at the price of a more convoluted install and a brittle process. Most people would probably be best off just converting one HTML file at a time to pdf (e.g. pdfkit.from_file(filename_in_html,filename_in_html[:-4]+".pdf") in some sort of iteration over the file names) and then concatenating the resulting PDF:s, for instance from command line by
pdftk TOC.pdf acknowledgements.pdf notation.pdf intro.pdf part_basics.pdf linear_algebra.pdf prob.pdf numerical.pdf ml.pdf part_practical.pdf mlp.pdf regularization.pdf optimization.pdf convnets.pdf rnn.pdf guidelines.pdf applications.pdf part_research.pdf linear_factors.pdf autoencoders.pdf representation.pdf graphical_models.pdf monte_carlo.pdf partition.pdf inference.pdf generative_models.pdf bib.pdf index-.pdf cat output Goodfellow-et-al-2016-Book.pdf
"Can I get a PDF of this book? No, our contract with MIT Press forbids distribution of too easily copied electronic formats of the book."
Look, I'm not planning on printing the whole thing, binding it and sticking it on my shelf. I want the pdf because then I can use it when I'm offline, across devices and search it easily.
If I want this book in hard copy, then I will purchase it - I've done this regularly with free digital books - but when it is offered free digitally then in my opinion prohibiting to only certain file formats is futile (as evidenced here), and such constraints are ineffective attempts to encourage people to buy the hard copy through inconvenience.
And I must add that this is no slight to the authors, whom have my greatest appreciation for compiling their vast knowledge into a book and offering it for free. These guys are legends.
If I want this book in hard copy, then I will purchase it - I've done this regularly with free digital books - but when it is offered free digitally then in my opinion prohibiting to only certain file formats is futile (as evidenced here), and such constraints are ineffective attempts to encourage people to buy the hard copy through inconvenience.
And I must add that this is no slight to the authors, whom have my greatest appreciation for compiling their vast knowledge into a book and offering it for free. These guys are legends.
@dandermotj I understand that, just included the reason why you cannot get it. The online book really sucks. I turned the styling off.
This type of thing is what gets people thinking about a way around the no-pdf solution.
What does the contract say about 3rd parties wrapping it in PDF?
I was able to print to PDF in Chrome and combined the PDFs in Acrobat.
I guess this could be automated. For instance, you could download all html files using a plugin like "Download Them All" with a renaming mask like "inum-nameinum.ext" and then try:
find . -iname '*.html' -exec wkhtmltopdf {} {}.pdf \;
There are also tools to convert the resulting files to a single pdf. The only problem I got is, that the woff fonts are not rendered by "wkhtmltopdf" :-/ Ideas?
find . -iname '*.html' -exec wkhtmltopdf {} {}.pdf \;
There are also tools to convert the resulting files to a single pdf. The only problem I got is, that the woff fonts are not rendered by "wkhtmltopdf" :-/ Ideas?
in google chrome you can do a control + p ( print ) and save as pdf
Saving as PDF was not working on OSX Chrome. However, using Safari with "show no footers" option, generated a perfect PDF.
Literally, the first link is a link to a pdf version (granted it's probably not quite intuitive).
For the lazy: http://www.deeplearningbook.org/front_matter.pdf
Personally, I was looking for an ePub version but no biggie.
For the lazy: http://www.deeplearningbook.org/front_matter.pdf
Personally, I was looking for an ePub version but no biggie.
If there are restrictions around distribution formats, it is misleading to call it "free".
The FSF does not have a monopoly over the word "free". For a lot of people, "free" means as in "free beer", which this book is, in the online format.
If only we could convince publishers to publish GFDL or Creative Commons licenses the way we've convinced some software companies to commercialize open source software...
A bit unrelated, but can anybody tell me what typeface is used for the body text in the PDF?
The font is "Computer Modern," the default LaTeX font. Nothing screams "I'm an academic engineering textbook!" quite like Computer Modern.
Concrete Roman could compete: http://www.tex.ac.uk/FAQ-concrete.html
It is amazing that Knuth still had time left for some mathematics, between designing fonts :)
(Obligatory: http://www.xkcd.com/974/ -- but Knuth solved the arbitrary condiments problem, and others besides.)
The Concrete Mathematics book that resulted was a masterpiece in my opinion. I learned so much about how to computationally do discrete math from that book. And it's a very elegant package.
The Concrete Mathematics book that resulted was a masterpiece in my opinion. I learned so much about how to computationally do discrete math from that book. And it's a very elegant package.
It's amazing to me that he designed a program (METAFONT) to design fonts for him, and that the various outputs of this program actually are pretty good fonts.
Didn't Hermann Zapf design the math font to go with CCR?
[Answer] Yep, Euler: https://en.wikipedia.org/wiki/AMS_Euler
[Answer] Yep, Euler: https://en.wikipedia.org/wiki/AMS_Euler
Here's a handy service for that: https://www.myfonts.com/WhatTheFont/
Just submit a screenshot image of some glyphs of the font at a reasonable size, and let the service figure out the font!
Just submit a screenshot image of some glyphs of the font at a reasonable size, and let the service figure out the font!
Pretty handy service. Any idea what's the mechanism behind the recognition software?
The default TeX font: https://en.wikipedia.org/wiki/Computer_Modern
Does anybody know how to make the book actually readable? http://i.imgur.com/C4rhclk.png
You can print to a pdf on Chrome. That comes out nicely for me.
Thanks!
Related to printing - this made me chuckle:
> Printing seems to work best printing directly from the browser, using Chrome. Other browsers do not work as well. In particular, the Edge browser displays the "does not equal" sign as the "equals" sign in some cases.
Of all the printing bugs for a maths/logic heavy text! Can just visualise the head banging against the desk upon discovering this one, having struggled with understanding something for hours - well ok, either the universe is broken or... oohhhhhhh :-D
Related to printing - this made me chuckle:
> Printing seems to work best printing directly from the browser, using Chrome. Other browsers do not work as well. In particular, the Edge browser displays the "does not equal" sign as the "equals" sign in some cases.
Of all the printing bugs for a maths/logic heavy text! Can just visualise the head banging against the desk upon discovering this one, having struggled with understanding something for hours - well ok, either the universe is broken or... oohhhhhhh :-D
That worked. Thanks
[deleted]
Looks fine here: http://imgur.com/0pQ3a53.png
Which browser are you using? It is perfectly readable in Safari and Firefox. Printing to PDF from Firefox on OSX yields excellent results, while printing to PDF from Safari didn't work well for me.
When i turned off scripts via NoScript it turned to readable again
The text looks readable but the formulas are missing pieces since the TeX fonts arent loaded. Printing to PDF worked for me
Chrome on Mac OS X works fine for me.
Thanks for this. I'm currently re-learning statics/probabilities and linear algebra so your book will be useful in a few months down the line ;)
Khan Academy has excellent material covering Probability and Statistics (https://www.khanacademy.org/math/probability) and Linear Algebra (https://www.khanacademy.org/math/linear-algebra).
This is exactly where I'm learning from as well.
Would you mind sharing any of the resources you are using for re-learning? I've been meaning to do the same.
For probability/statistics you could also use the MIT Course https://www.edx.org/course/introduction-probability-science-...
Same course if you prefer the classroom lectures http://ocw.mit.edu/courses/electrical-engineering-and-comput...
Or if you want more rigor you can go through these notes that cover the same material but in a more formal way (via sigma algebras and measure theory) http://ocw.mit.edu/courses/electrical-engineering-and-comput...
Same course if you prefer the classroom lectures http://ocw.mit.edu/courses/electrical-engineering-and-comput...
Or if you want more rigor you can go through these notes that cover the same material but in a more formal way (via sigma algebras and measure theory) http://ocw.mit.edu/courses/electrical-engineering-and-comput...
Just saying, but if you want to hop onto the ML bandwagon (for instance), then don't bother going over linear algebra or probabilities first, and instead just learn what you need as you go. For example, the first sections of this book are already devoted to getting you on the right track, and it's somewhat standard to do so. And besides, there's no need in learning what are rotation matrices if you won't use them.
As a counterpoint, if parent is interested in taking ML further, a solid foundation in linear algebra will be huge when more advanced signal processing applications come up.
A couple of friends recommended these:- (Not sure if they are relevant though for deep learning specifically)
1) http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-... 2) https://www.khanacademy.org/math/linear-algebra/vectors_and_...
If anyone knows anything else (relevant to deep learning) could you please share :)
1) http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-... 2) https://www.khanacademy.org/math/linear-algebra/vectors_and_...
If anyone knows anything else (relevant to deep learning) could you please share :)
Introduction To Statistical Learning:
http://www-bcf.usc.edu/~gareth/ISL/
Is an excellent statistical learning reference.
http://www-bcf.usc.edu/~gareth/ISL/
Is an excellent statistical learning reference.
I've been going through this series of video lectures on Youtube:
https://www.youtube.com/playlist?list=PL5102DFDC6790F3D0
for a basic "Stats 101" course.
There's also this archived Coursera course. There aren't any active sections to sign up for, but the videos are still available:
https://class.coursera.org/introstats-001
https://www.youtube.com/playlist?list=PL5102DFDC6790F3D0
for a basic "Stats 101" course.
There's also this archived Coursera course. There aren't any active sections to sign up for, but the videos are still available:
https://class.coursera.org/introstats-001
I'm mostly using Khan Academy at the moment. But I see several people already posted alternatives which is nice to have ;)
* Foundations of Machine Learning
* All of Statistics
* Doing Bayesian Data Analysis
Also the ML specialization on Coursera
* All of Statistics
* Doing Bayesian Data Analysis
Also the ML specialization on Coursera
There isn't much hard probability/statistics in deep learning since most of the stuff is empirical(IE: these neural network architectures work b/c we tried it and it works!). I'd say that deep learning is very accessible to the novice practictioner if you really want to dive into it; it doesn't require the same sort of mathematical background that something like signal processing might.
If you want to learn probability and statistics I recommend http://probabilitycourse.com
For linear algebra I heard Paul Dawkins' document is great, it's not on his site anymore but you can find it online. I've read calculus 1 and 1/3 of calculus 2 and it's good material.
For linear algebra I heard Paul Dawkins' document is great, it's not on his site anymore but you can find it online. I've read calculus 1 and 1/3 of calculus 2 and it's good material.
This book looks amazing. Thanks for this.
I am a big fan of Linear Algebra Done Right, if you are looking for an actual, dead-tree book with good explanations.
They better release a PDF in the future.
Don't quite get the complaints about it not being available in PDF. "We'll publish your book, and you can give it away for free as long as you make people click through to each chapter" is a much, much better deal than I would expect from a big publisher.
Pretty much ... the only edge case I feel for is having an offline copy in places with low or expensive internet access. Like parts of the developing world. Clicking through isn't such a big deal.
In the old days IE had a feature for saving pages for offline reading that would ask you how deep to recursively traverse. It was considered hilarious to set it for 99 on the school dialup connection.
Give an inch, take a mile.
Remove Facebook link
[deleted]
The HTML format is quite peculiar.
It kinda looks like someone ran the original PDF through PDF.js and saved the rendered output to a HTML file.
It kinda looks like someone ran the original PDF through PDF.js and saved the rendered output to a HTML file.
The HTML source contains this:
<!-- Created by pdf2htmlEX (https://github.com/coolwanglu/pdf2htmlex) -->
<!-- Created by pdf2htmlEX (https://github.com/coolwanglu/pdf2htmlex) -->
"This format is a sort of weak DRM required by our contract
with MIT Press. It's intended to discourage unauthorized
copying/editing of the book. Unfortunately, the conversion
from PDF to HTML is not perfect, and some things like
subscript expressions do not render correctly. If you have a
suggestion for a better way of making the book available to
a wide audience while preventing unauthorized copies, please
let us know."they just need to use the --zoom option in pdf2htmlEX when generating the html. 1.5 zoom should be good.
Can any practitioners / experts out there comment on the range of topics? For example, I understand the book to be introductory, and so the scope is likely somewhat limited. But how close does it get you to the ANNs currently in use, at least conceptually if not in complete detail? Thanks!
Non-expert DL practitioner here. The coverage is very good - if I were running some kind of "Deep Learning onboarding class" for a university or tech company, this is what I would present.
My favorite aspect of this book is that it provides a graphical models interpretation of DL methods, which is the most powerful perspective we have right now to reason about model design (instead of some large black box function that we train end-to-end without knowing what's in between).
It also explains some fairly recent models and techniques well (VAE, DCGAN, regularization) that form the basis of more complex architectures. If you understand the models here, you should be able to understand the design choices made in more complex architectures.
Thanks to Goodfellow, Bengio, and Courville for this excellent work.
My favorite aspect of this book is that it provides a graphical models interpretation of DL methods, which is the most powerful perspective we have right now to reason about model design (instead of some large black box function that we train end-to-end without knowing what's in between).
It also explains some fairly recent models and techniques well (VAE, DCGAN, regularization) that form the basis of more complex architectures. If you understand the models here, you should be able to understand the design choices made in more complex architectures.
Thanks to Goodfellow, Bengio, and Courville for this excellent work.
Great, that's very helpful to know. Thanks!
This looks great, any other recommendations for enjoyable reads on ml/stat learning?
ESL ISLR doing Bayesian data analysis w/ jags/Stan bda3 - gelman prob graphical models Convex analysis - Boyd adv data analysis from elem pov - shalizi
Trying to build out my library. I have a background in prob/stats/analysis and measure theory/linear algebra and also knowledge of algorithms and data structures at the advanced undergrad level, So I'm not too concerned about technical depth just want to enjoy a good technical expository and gain intuition.
ESL ISLR doing Bayesian data analysis w/ jags/Stan bda3 - gelman prob graphical models Convex analysis - Boyd adv data analysis from elem pov - shalizi
Trying to build out my library. I have a background in prob/stats/analysis and measure theory/linear algebra and also knowledge of algorithms and data structures at the advanced undergrad level, So I'm not too concerned about technical depth just want to enjoy a good technical expository and gain intuition.
My main issue is that the book tells you all about the different parameter tweaks, but passes little concrete wisdom to the reader. It doesn't distinguish between modeling assumptions, and it replaces very simple explanations of concepts with complicated paragraphs that I can't make sense of.
I think it boils down to something that I have been feeling and hearing a lot in the past few years: the statistical jargon is so overwhelming that the authors can't explain things clearly. I can point to many examples in this book that I feel are unnecessary stumbling blocks, but the fact is that I'll spend an hour or two discussing parts of this book with a room full of smart machine learning researchers, and at the end we'll all agree we don't understand the material better than we did at the start.
On the other hand, I'll read research papers that don't force the statistical perspective down the reader's throat (e.g. http://arxiv.org/abs/1602.04485v1) and find them very easy to understand by comparison.
It might be a cultural difference, but I've heard this complaint enough from experts who straddle both sides of the computational/statistical machine learning divide that I don't think it's just me.