Can anyone recommend good books, articles or essays that introduce machine learning?
I intend to find out for myself but I'm curious to know how steep a learning curve you've found it to be.
24 comments
If you want a fairly easy read without too many equations, try:
Data Mining: Practical Machine Learning Tools and Techniques (Second Edition) http://www.cs.waikato.ac.nz/~ml/weka/book.html
Which goes nicely with the Weka open source ML toolkit http://www.cs.waikato.ac.nz/ml/weka/
(although it is a good read without the toolkit)
If you want a bit more math, I really like the recent (Oct 2007) book:
Pattern Recognition and Machine Learning by Christopher M. Bishop http://www.amazon.com/Pattern-Recognition-Learning-Informati...
It is nicely self contained, going through all the stats you'll need.
Data Mining: Practical Machine Learning Tools and Techniques (Second Edition) http://www.cs.waikato.ac.nz/~ml/weka/book.html
Which goes nicely with the Weka open source ML toolkit http://www.cs.waikato.ac.nz/ml/weka/
(although it is a good read without the toolkit)
If you want a bit more math, I really like the recent (Oct 2007) book:
Pattern Recognition and Machine Learning by Christopher M. Bishop http://www.amazon.com/Pattern-Recognition-Learning-Informati...
It is nicely self contained, going through all the stats you'll need.
I didn't particularly like the WEKA book, but Bishop's book is excellent.
If you're interested in introductory data mining stuff, I would recommend Tan's Introduction to Data Mining: http://www-users.cs.umn.edu/~kumar/dmbook/index.php
If you're interested in introductory data mining stuff, I would recommend Tan's Introduction to Data Mining: http://www-users.cs.umn.edu/~kumar/dmbook/index.php
Machine Learning by Tom Mitchell for a serious academic book.
Programming Collective Intelligence by Toby Segaran for a practical approach in Python.
Programming Collective Intelligence by Toby Segaran for a practical approach in Python.
I disagree wrt Mitchell; too dry and out-of-date. In its place I'd recommend the Norvig book which covers everything Mitchell does (IIRC) except 'Probably Approximately Correct' (PAC) learning. And, it's more up-to-date (Support Vector Machines) and is much more readable. Also, I haven't read it except a flip-through at a bookstore, but Christopher Bishop's one I'd look at (perhaps the Amazon comments).
I've seen the Norvig book recommended by others on this site for ML, but I don't understand why: the Norvig book is an AI book, not an ML book. It only has a few chapters on learning, and IIRC much of that is on reinforcement learning. Obviously the Norvig book is very well-written and is good background for learning about ML, but I don't think it is a sufficient ML book as such.
As far as ML goes, I found the "Programming Collective Intelligence" book to very readable and practical, but very light on the theoretical foundations (which is intentional, of course). I've got a copy of the Witten ML book ("Data Mining: Practical Machine Learning Tools and Techniques"), but to be honest I haven't gotten much from it yet, either: it doesn't seem to discuss SVMs in any detail, nor random forests or neural networks. But I haven't really dug into it yet.
As far as ML goes, I found the "Programming Collective Intelligence" book to very readable and practical, but very light on the theoretical foundations (which is intentional, of course). I've got a copy of the Witten ML book ("Data Mining: Practical Machine Learning Tools and Techniques"), but to be honest I haven't gotten much from it yet, either: it doesn't seem to discuss SVMs in any detail, nor random forests or neural networks. But I haven't really dug into it yet.
This is kind of moot, truthfully I'd go, now (or look first at, but they're both getting praise so...) for Chris Bishop or Ethem Alpaydin's new books. But between Norvig and Mitchell, Norvig has 138 pages on learning vs Mitchell's ~390, but, Mitchell's is from a different era, Norvig is easier to read - larger pages, more diagrams, better writing, and you know where you are better - and fresher material. To each his own. But like I say, I'd probably go with one of the even newer ones. For a while Mitchell was all there was, then Norvig came along, and now there're a few to choose from.
If you can read and understand Mitchell's book, you will have a very good foundation for understanding modern ML techniques. The poster was looking for references to introduce him to the field.
I agree on the Mitchell book.
If you are looking for a light introduction to naive Bayesian text classification then there's always the piece I wrote for DDJ: http://www.ddj.com/development-tools/184406064
John.
If you are looking for a light introduction to naive Bayesian text classification then there's always the piece I wrote for DDJ: http://www.ddj.com/development-tools/184406064
John.
[deleted]
The "Programming Collective Intelligence" book is pretty good in terms of a practical introduction to a variety of techniques.
Artificial Intelligence A Modern Approach SE, by Stuart Russell and Peter Norvig.
I am a big fan of Peter Norvig in particular, he has a ton of great essays and code available online: http://norvig.com/
I am a big fan of Peter Norvig in particular, he has a ton of great essays and code available online: http://norvig.com/
Information Theory, Inference and Learning Algorithms by David MacKay is excellent, and freely available online.
http://www.inference.phy.cam.ac.uk/mackay/itila/book.html
http://www.inference.phy.cam.ac.uk/mackay/itila/book.html
I'll second this recommendation - I bought the printed copy, and I'm constantly going to it for reference. The fact that it's available for free is just an added bonus.
I would also suggest Elements of Statistical Learning: http://www-stat.stanford.edu/~tibs/ElemStatLearn/
As well as Duda, Hart, and Stork's Pattern Classification: http://rii.ricoh.com/~stork/DHS.html
I would also suggest Elements of Statistical Learning: http://www-stat.stanford.edu/~tibs/ElemStatLearn/
As well as Duda, Hart, and Stork's Pattern Classification: http://rii.ricoh.com/~stork/DHS.html
This is a great book, but it's definitely not introductory. I think Segaran's "Programming Collective Intelligence", mentioned above, is the best first bet, primarily because it's fun. And the code is good.
books about machine learning applications/tools:
- "Programming Collective Intelligence"
- http://www.oreilly.com/catalog/9780596529321/
- "Data Mining: Practical Machine Learning Tools and Technique"
- http://www.cs.waikato.ac.nz/~ml/weka/book.html
books about machine learning background/theory: - "machine learning" - http://www.cs.cmu.edu/~tom/mlbook.html - "learning and soft computing" - http://cognet.mit.edu/library/books/view?isbn=0262112558
For a simple-to-use (Python-based) machine learning tool/API check out Orange: http://magix.fri.uni-lj.si/orange/
books about machine learning background/theory: - "machine learning" - http://www.cs.cmu.edu/~tom/mlbook.html - "learning and soft computing" - http://cognet.mit.edu/library/books/view?isbn=0262112558
For a simple-to-use (Python-based) machine learning tool/API check out Orange: http://magix.fri.uni-lj.si/orange/
Check out the lecture notes for CS229, Stanford's class on Machine Learning. The class has no textbook, so the notes are fairly comprehensive. (Though the math notation can get a bit intense...)
http://www.stanford.edu/class/cs229/materials.html
http://www.stanford.edu/class/cs229/materials.html
Also, the notes assume a reasonable knowledge of probability theory and linear algebra. If you're unsure, you might do well to review those topics before approaching machine learning (or at least keep good references handy). (Edit: the link above has Section Notes that review probability theory, linear algebra, and convex optimization. You might find those useful.)
And it's not an easy course. Don't be discouraged if the problem sets seem impossible. (They very nearly are.)
And it's not an easy course. Don't be discouraged if the problem sets seem impossible. (They very nearly are.)
Sutton and Barto's book on reinforcement learning is available online at http://www.cs.ualberta.ca/~sutton/book/the-book.html, and is probably the definitive work on the subject.
I wish they'd make a new edition; but, probably, the field is moving too fast.
A nice read early on would be "Elephants don't play chess" but Rodney Brooks.
http://people.csail.mit.edu/brooks/papers/elephants.pdf
http://people.csail.mit.edu/brooks/papers/elephants.pdf
I stopped reading on page 3, because it seems to me that the "physical grounding hypothesis" is complete bullshit. Where is the "physical grounding" aspect if I want to build a search engine for the web? It's all just information.
Not saying that getting physical can not yield interesting results, but to state that it is the only possible way to program intelligence seems just wrong.
Not saying that getting physical can not yield interesting results, but to state that it is the only possible way to program intelligence seems just wrong.
The machine learning blog at http://hunch.net/ is a good resource.
ML is a very interesting field.
Great resources can be found on:
http://www.aaai.org/AITopics/html/machine.html
http://aima.cs.berkeley.edu/ai.html
Start by reading and completely understanding a few of the simpler machine learning algorithm (Backpropagation neural networks for example), how&why they work. and continue from there.
Start by reading and completely understanding a few of the simpler machine learning algorithm (Backpropagation neural networks for example), how&why they work. and continue from there.
Mitchell, Norvig, and "Probabilistic Robotics" by Thrun.