The Little Learner: A Straight Line to Deep Learning(mitpress.mit.edu)
mitpress.mit.edu
The Little Learner: A Straight Line to Deep Learning
https://mitpress.mit.edu/9780262546379/the-little-learner/
101 comments
[deleted]
The initial preview is a little disappointing because it feels like it does a great job displaying the pedagogical style but not enough to demonstrate the pedagogical value (i.e. whether this resource actually does a good job teaching a complex topic).
Given the wealth of information and the problem of appraising it all, I don't think it helps this book that it costs $55 and requires the reader to learn an esoteric language (for the field), when there are so many educational books and lecture series by experts in the field that you can find freely available online.
Given the wealth of information and the problem of appraising it all, I don't think it helps this book that it costs $55 and requires the reader to learn an esoteric language (for the field), when there are so many educational books and lecture series by experts in the field that you can find freely available online.
> The initial preview is a little disappointing because it feels like it does a great job displaying the pedagogical style but not enough to demonstrate the pedagogical value (i.e. whether this resource actually does a good job teaching a complex topic).
This is exactly the concern I had when I saw what it was. I'll definitely keep my eye on it and read the reviews.
This is exactly the concern I had when I saw what it was. I'll definitely keep my eye on it and read the reviews.
The great thing about Scheme is that you can learn it in less than 10 minutes and focus on the ideas.
Sure - but I think that's the least important part of my comment. The preview is basically just an intro to Scheme. I would rather be able to see how well the book covers interesting ideas about deep learning. While the jury is still out, my personal skepticism is that this is a really beautiful passion project that looks great on bookshelves, but that any serious learner would be better served dedicating their time to working through the existing, more traditional, and very high-quality resources available online.
This is what the book has to say (part of a foreword by Peter Norvig):
> Maybe, maybe not. But even if you use a machine learning toolkit like TensorFlow or PyTorch, what you will take away from this book is an appreciation for how the fundamentals work.
> Maybe, maybe not. But even if you use a machine learning toolkit like TensorFlow or PyTorch, what you will take away from this book is an appreciation for how the fundamentals work.
I think all serious researchers have implemented core Deep Learning algorithms from scratch. I know I have.
There are two books that do exactly this:
1. Deep Learning from Scratch
2. Data Science from Scratch
In these books, you implement each part of the ML/DL pipeline, from scratch, in Python.
There is also a GitHub project call Minitorch that teaches you the inner workings of a framework like PyTorch.
And then there are several other good resources for exactly this.
What he claims to have as a content is neithet new nor unique.
There are two books that do exactly this:
1. Deep Learning from Scratch
2. Data Science from Scratch
In these books, you implement each part of the ML/DL pipeline, from scratch, in Python.
There is also a GitHub project call Minitorch that teaches you the inner workings of a framework like PyTorch.
And then there are several other good resources for exactly this.
What he claims to have as a content is neithet new nor unique.
How tied are those two books to python? If very much, are there books covering the same content with a different programming language?
They are aren’t that tied to Python.
Even if you think so, Python is really an easy language, and you can easily port the code to something else.
If you already have the basic ideas about the parts of a Neural Network pipeline, you can just search google "implement part-X in Y language", and you will get well written articles/tutorials.
Many learners/practitioners of Deep Learning, when they have the big enough picture, write an NN training loop in their favorite language(s) and post it online. I remember seeing a good enough "Neural Network in APL" playlist in YT. It implements every piece in APL and gains like 90%+ accuracy in MNIST.
I also remember seeing articles in Lisp (of course!), C, Elixir, and Clojure.
I am writing one in J-lang in my free time.
Even if you think so, Python is really an easy language, and you can easily port the code to something else.
If you already have the basic ideas about the parts of a Neural Network pipeline, you can just search google "implement part-X in Y language", and you will get well written articles/tutorials.
Many learners/practitioners of Deep Learning, when they have the big enough picture, write an NN training loop in their favorite language(s) and post it online. I remember seeing a good enough "Neural Network in APL" playlist in YT. It implements every piece in APL and gains like 90%+ accuracy in MNIST.
I also remember seeing articles in Lisp (of course!), C, Elixir, and Clojure.
I am writing one in J-lang in my free time.
they mostly use numpy (matrix maths library).
So if you use a library for matrix multiplication, inverse, transpose, ... with a nice syntax, you're good to go.
So if you use a library for matrix multiplication, inverse, transpose, ... with a nice syntax, you're good to go.
What does "from scratch" really mean? You don't reimplement Python itself, or invent a new GPU hardware, a new CUDA including compiler, etc. You don't reimplement the OS. Where do you draw the line?
Do you reimplement matmul or other basics?
Do you reimplement auto-diff?
Maybe PyTorch or TensorFlow using auto-diff is a good "from scratch" basepoint, without using predefined optimizers, or modules/layers, or anything. Just using the low-level math functions, and then auto-diff.
Do you reimplement matmul or other basics?
Do you reimplement auto-diff?
Maybe PyTorch or TensorFlow using auto-diff is a good "from scratch" basepoint, without using predefined optimizers, or modules/layers, or anything. Just using the low-level math functions, and then auto-diff.
I don't understand why you are arguing with me.
Yes, in those books, you do implement matmul, auto-diff, etc.
Yes, in those books, you do implement matmul, auto-diff, etc.
I don't understand. I don't argue with you? I also don't speak about these books. I just made a generic comment, to start a discussion.
I just wanted to point out that "from scratch" is not really well defined. There is always some arbitrary line. I just found it interesting to discuss and think about where to draw this line exactly. Obviously it's never really from scratch, i.e. you don't reinvent the hardware level, for example. Or you don't start with teaching quantum physics. So you start from somewhere.
And I was wondering whether auto-diff is maybe something which could also already be the starting point, or also matmul. Reimplementing an efficient matmul on CUDA is not easy, and might distract from the main deep learning content. But it depends also where you want to have the focus on.
I just wanted to point out that "from scratch" is not really well defined. There is always some arbitrary line. I just found it interesting to discuss and think about where to draw this line exactly. Obviously it's never really from scratch, i.e. you don't reinvent the hardware level, for example. Or you don't start with teaching quantum physics. So you start from somewhere.
And I was wondering whether auto-diff is maybe something which could also already be the starting point, or also matmul. Reimplementing an efficient matmul on CUDA is not easy, and might distract from the main deep learning content. But it depends also where you want to have the focus on.
Now it makes more sense.
Maybe I misread your comment.
Thanks for rewording.
Where do people draw the line? Where they want to.
Some people choose to go closer to the metal than the others. It's just people's choice.
Some just write stuff in Python, some write CUDA kernel for themselves (some among them had to), a friend even wrote his own compiler and programming language for Deep Learning.
So it depends on your choice. And how much deep you want to go also depends on what you want to do- i.e. you choice of career, direction of research, etc.
Maybe I misread your comment.
Thanks for rewording.
Where do people draw the line? Where they want to.
Some people choose to go closer to the metal than the others. It's just people's choice.
Some just write stuff in Python, some write CUDA kernel for themselves (some among them had to), a friend even wrote his own compiler and programming language for Deep Learning.
So it depends on your choice. And how much deep you want to go also depends on what you want to do- i.e. you choice of career, direction of research, etc.
More than happy to leave the traditional, high quality resources available online to the serious learners
I would think that the main audience for this books would be people who enjoyed the other books in the authors' "Little" series, which almost all use Scheme, so it wouldn't be something new to learn.
Checking the library the book implements can help: https://github.com/themetaschemer/malt
If this is anything like the others in the series. I really like the style of the others; it is a really good way of learning for me. And the little prover&typer was when I already was well versed in the theory but they were really good to work through nonetheless.
So yep, must have. If you haven’t read the rest, they are really must haves imho.
So yep, must have. If you haven’t read the rest, they are really must haves imho.
No way. Does it use Scheme?
EDIT: Half Life 3 confirmed: "Presents key ideas of machine learning using a small, manageable subset of the Scheme language"
EDIT: Half Life 3 confirmed: "Presents key ideas of machine learning using a small, manageable subset of the Scheme language"
Some time ago I wrote this: https://notabug.org/ZelphirKaltstahl/guile-ml -- It contains a parallelized decision tree implementation in GNU Guile, a Scheme dialect. There is also https://github.com/lloda/guile-ffi-cblas which at least gives fast matrix multiplication. Example usage I have here: https://notabug.org/ZelphirKaltstahl/guile-math-examples/src...
I don't know what is in the book, but things are possible.
I don't know what is in the book, but things are possible.
For sure, Lecun wrote a DL framework in Lisp decades ago.
I didn't know about his past feat. Couldn't find the lisp framework you mention but there's this https://lush.sourceforge.net/faq.html
(trivia, he's also behind djvu format)
(trivia, he's also behind djvu format)
Lush was really great back then. He often criticizes Python openly.
Personally, I also feel a bit sad the ecosystem has become a Python monoculture.
Lua was much nicer. Let's hope this book sparks some interest in alternative languages for ML.
Personally, I also feel a bit sad the ecosystem has become a Python monoculture.
Lua was much nicer. Let's hope this book sparks some interest in alternative languages for ML.
I'm really surprised that Haskell didn't take off here. Strong types, plus lazy evaluation that seems perfect for orchestrating asynchronous GPU operations.
Love the beauty of Scheme, but lack of types (mypy Python / Java dev/ TS dev) infuses fear to me.
I think with a little of patience I can get by.
I think with a little of patience I can get by.
You should try typed Racket!
Will do. I have also found that 3 days ago
https://github.com/zdimension/hm-infer-scheme
fantastic for research
https://github.com/zdimension/hm-infer-scheme
fantastic for research
CL is decent in this regard: http://alhassy.com/TypedLisp
I don’t have much experience with CL, but I was always a bit put off by a mixture of different paradigms it has, ie it’s not being a pure or kinda pure functional language.
That's an advantage for me, tbh.
I meant regarding types, I think the equivalent to the other link would be Coalton: https://github.com/coalton-lang/coalton
I am looking at the GitHub repo for the Malt library developed in the book https://github.com/themetaschemer/malt. It looks like they use Racket (Scheme) vectors to implement tensors. I experimented with loading Keras simple models into Racket several years ago: the built in math library matrix support was fast enough for my needs so the book software may be both pedagogical and useful for smaller problems.
EDIT: I have not tried the OpenBLAS Racket bindings here (https://github.com/soegaard/sci) but perhaps the low level tensor and tensor ops book code could be optimized,
EDIT: I have not tried the OpenBLAS Racket bindings here (https://github.com/soegaard/sci) but perhaps the low level tensor and tensor ops book code could be optimized,
Yes. At least from the preview.
Very much so.
This looks interesting, though I'd probably do at LEAST little schemer finally before checking this one out (been meaning to pick up some of the Little x books but haven't, though I've been getting back to lisp through CL on things like PAIP).
Nothing little about a 450 page book, but still... I have to buy it.
I think the little learner is the reader rather than the book
i think the little part is the chapter sizes on purpose
I helped proofread this book, definitely check it out.
What do you think of this comment https://news.ycombinator.com/item?id=34812501
You are the best!
I went through some of the eBook providers but none seem to be offering a PDF version. Did anyone notice if a PDF version was for sale?
I bought "The Reasoned Schemer" as physical book and searched for a way to buy it as a PDF version, but could find it nowhere (also had no problem to pay twice for the book).
The Kindle version of it does not even have fixed table cell width for the layout of the conversation.
German book website buecher.de has "The Little Learner" to preorder as ePUB with Adobe DRM. I don't know, if you can buy digital books on this site outside of Germany. Also notice, that copy-protected digital books on this website are highly unusual; most of the books are with watermark instead, if at all.
I have no clue why a book with pedagogical background and custom layout, has copy-protection and no print-quality PDF version (but instead one with poor layout).
The Kindle version of it does not even have fixed table cell width for the layout of the conversation.
German book website buecher.de has "The Little Learner" to preorder as ePUB with Adobe DRM. I don't know, if you can buy digital books on this site outside of Germany. Also notice, that copy-protected digital books on this website are highly unusual; most of the books are with watermark instead, if at all.
I have no clue why a book with pedagogical background and custom layout, has copy-protection and no print-quality PDF version (but instead one with poor layout).
I’m also interested in this. Unfortunately, the other schemer/typer books appear to be available for kindle… so that’s likely the best we’ll get
I didn't find one. However, amazon has a kindle version, which might not be what you're looking for!
Just to piggy back on this. The typesetting seems off kindle and other ebook platforms. Anyone seeing this?
Wow, I would really like to get this book to complete my set of the Little Schemer, the Seasoned Schemer and the Reasoned Schemer. But 55$ is almost twice as expensive as the other books. Yes, it is bigger than those but still...
Have to think about this for a while :)
Have to think about this for a while :)
Rated on dollars per hour of brain upgrades, thinking and enjoyment, it doesn’t get better than a good book.
The little schemer is $40 now, reasoned schemer is $30, and the little typer is $38 (about twice as thick as the other two).
I had to think on the little schemer, but grabbed it because my ten year old said that the little typer was "interesting", and I thought he might learn something from it.
Probably should finish working through what I have before getting another, though.
I had to think on the little schemer, but grabbed it because my ten year old said that the little typer was "interesting", and I thought he might learn something from it.
Probably should finish working through what I have before getting another, though.
Your ten-year-old was interested in dependent types? You can turn to the back of the book and show him that truths begin with 𝚪 ⊢
There's also "The Little Typer" and "The Little Prover."
In the same series, there's also "A Little Java" by Matthias Felleisen, one of the creators of Racket (a popular Lisp).
https://mitpress.mit.edu/9780262561150/a-little-java-a-few-p...
https://mitpress.mit.edu/9780262561150/a-little-java-a-few-p...
Is it still good? I see it was published in 1997.
I like the style of these books, and might read this one - I'm just worried that Java has changed quite a bit since then. Is it still, say, 90% OK?
I like the style of these books, and might read this one - I'm just worried that Java has changed quite a bit since then. Is it still, say, 90% OK?
It's mostly a novelty today. Felleisen is a programming language theorist, and him writing a little Java tutorial sounds absurd.
The book itself takes you on a journey to reinvent the Visitor pattern, in a version of Java that predates generics. You can probably find a better explainer for the Visitor pattern in Java.
The book itself takes you on a journey to reinvent the Visitor pattern, in a version of Java that predates generics. You can probably find a better explainer for the Visitor pattern in Java.
Thanks. I see it for sale for £4 second hand, so maybe I should buy it for the novelty value alone. I presume I shouldn't try to refresh my java from it though.
You aren't helping him. :)
yeah, I just realized that too. screw it, I'll just grab all of them ;-)
Are there other books written in this format?
I know _why wrote his books as a conversation between two foxes.
I know _why wrote his books as a conversation between two foxes.
The Little Schemer (formerly LISPer) - Intro to Scheme, or perhaps an introduction to computing with Scheme. Guess it depends on perspective.
The Seasoned Schemer - More on Scheme.
The Reasoned Schemer - Minikanren in Scheme
The Little Prover - Proofs about programs
The Little Typer - Dependent types
The Little MLer - The ML language, not machine learning.
A Little Java - Java
I have the first 5 books, but never got around to finishing Little Prover or Typer (got married and moved that year, probably not the best time to try and start something like that). Definitely like the first three and what I read in those other two.
The Seasoned Schemer - More on Scheme.
The Reasoned Schemer - Minikanren in Scheme
The Little Prover - Proofs about programs
The Little Typer - Dependent types
The Little MLer - The ML language, not machine learning.
A Little Java - Java
I have the first 5 books, but never got around to finishing Little Prover or Typer (got married and moved that year, probably not the best time to try and start something like that). Definitely like the first three and what I read in those other two.
Smullyan, but those are decades old and not application focused.
Yes, a whole series. Look until the author for more.
Someone who has access to O'Reilly and other free source to learn ML/AI e.g. Fast.ai, is there any advantage of buying this?
'Little..' series books are famous for their pedagogical style: small bit of concepts, presented as a dialogue, making you think at each step, in a well thought out order. It will appeal to anyone who wants to build it out themselves, understand the 'big' ideas in the field. Skim through the second chapter to see how this will be done: they define line function in inverted way where slope and intercept are taken as parameters later. Feels weird but exactly the mental model shift we should be doing when thinking about ML.
Read the preface. A very high praise coming from Guy Steele Jr and Peter Norvig.
At the same time, a warning! Scheme based introductions don't appeal to everyone. Some people feel way out of comfort zone in coding with it (which is sad because it is much simpler). Also, the utilitarian appeal is low: it won't right away see a step change in your Pytorch knowledge or whatever. The appeal of these books is to think deeply about fundamental ideas by implementing them in simplest language without too much help.
In short, YMMV. But if you have a long term view it might help you a lot than sort of currently fashionable trends. (Though I must admit that fast.ai is not just a flavor of the season resources but much better!)
Read the preface. A very high praise coming from Guy Steele Jr and Peter Norvig.
At the same time, a warning! Scheme based introductions don't appeal to everyone. Some people feel way out of comfort zone in coding with it (which is sad because it is much simpler). Also, the utilitarian appeal is low: it won't right away see a step change in your Pytorch knowledge or whatever. The appeal of these books is to think deeply about fundamental ideas by implementing them in simplest language without too much help.
In short, YMMV. But if you have a long term view it might help you a lot than sort of currently fashionable trends. (Though I must admit that fast.ai is not just a flavor of the season resources but much better!)
So you mean that fast.ai maybe a better resource for people new to general ML?
Well, hard to say if it is a better resource. But it will appeal to those who are in a hurry (and yes, it is okay to be in a hurry). For example, fast.ai first chapter will have you build a dogs vs cats classifier. But how do they build it? By doing `from fastai.vision import *`. They justify it by claiming to use Teaching the whole game approach. If you are a somewhat experienced engineer who already knows Python, what is Jupyter then this approach will get you started quickly. For someone who feels did I spend my time/money but nothing exciting is happening yet, this is a good start. But for some it is crazy and makes them feel even more scared (what is happening with all this imports, how does it work).
My criticism with fast.ai, (I am part time educator), is that this approach is an information overload and poor sequencing. Their comparison with Teach whole game approach is flawed because a game of, say Football, is essentially simple. So you can say just start kicking around. But we don't teach chess this way. It is accepted that you have to spend some time upfront to learn the rules before you can play even simple game. Sure one need not learn castling or en-passant upfront. But you get the drift.
This book (looking at the preview chapters) is going to follow the lego blocks approach or bottoms up approach to build it. For me, this is correct way to teach supervised ML focussed on neural networks and deep learning. We have a problem of too many library plumbers in the ML field currently. People who can piece together library function calls without knowing why it is working. But this house of cards is not sustainable strategy to build AI based application over long term.
Long story short, the book will need patience but that patience will be worth it!
My criticism with fast.ai, (I am part time educator), is that this approach is an information overload and poor sequencing. Their comparison with Teach whole game approach is flawed because a game of, say Football, is essentially simple. So you can say just start kicking around. But we don't teach chess this way. It is accepted that you have to spend some time upfront to learn the rules before you can play even simple game. Sure one need not learn castling or en-passant upfront. But you get the drift.
This book (looking at the preview chapters) is going to follow the lego blocks approach or bottoms up approach to build it. For me, this is correct way to teach supervised ML focussed on neural networks and deep learning. We have a problem of too many library plumbers in the ML field currently. People who can piece together library function calls without knowing why it is working. But this house of cards is not sustainable strategy to build AI based application over long term.
Long story short, the book will need patience but that patience will be worth it!
Thank you for your reply.
I'm a Lisper (used Common Lisp, Racket/Scheme, and Clojure) and a math graduate student, and am interested in learning more about ML and DL so that I could potentially use them in research in the future, to come up with constructions and counter-examples, so I may have some time in my hand. I have always been a fan of Dan Friedman, so I'm definitely thinking about getting this book.
I found out that fast.ai seems to be of this approach that you mentioned. Since you are an educator, if I'm interested in learning more about ML and DL (as well as the math behind and implementation), which book(s) would you recommend me to study (I have some experience with Andrew Ng's coursera course many years ago)? I did some research here and people seem to recommend different books...
Thank you!
I'm a Lisper (used Common Lisp, Racket/Scheme, and Clojure) and a math graduate student, and am interested in learning more about ML and DL so that I could potentially use them in research in the future, to come up with constructions and counter-examples, so I may have some time in my hand. I have always been a fan of Dan Friedman, so I'm definitely thinking about getting this book.
I found out that fast.ai seems to be of this approach that you mentioned. Since you are an educator, if I'm interested in learning more about ML and DL (as well as the math behind and implementation), which book(s) would you recommend me to study (I have some experience with Andrew Ng's coursera course many years ago)? I did some research here and people seem to recommend different books...
Thank you!
Probably, but "two Python books" mentioned in this thread are even more suited to a general beginner.
Thank you. Maybe it's not modest, but I do not consider myself to be a "general beginner". I majored in math and CS, so I'm thinking about getting something more hard-core and serious...
Great. Have you tried just jumping in to Karpathy's video series on youtube? Maybe that plus reading some papers?
I second this. Given your background (lisper) maybe do the Little Learner book once it is out + Karpathy's video series. Follow it up by building a slightly complicated application in your favourite domain (text, images, videos, time series).
Also word of advise from my experience (I'm not an expert in DL either): Think of DL field as a game of lego blocks. The ideas in this book / Karpathy's videos are the basic lego blocks: parameterised linear functions, non-linearities, auto-grad, cross-entropy / KL divergence loss and gradient descent. Then there is entire body of more complex legos discovered simply by practice (alchemy!): transformer blocks, layer norm, max-pooling etc. It is impossible to understand how the second kind were obtained from first principles. The trick is to not beat yourself up about the advanced blocks too much but just play around, read up things in papers. Just focus on fundamental blocks.
Also word of advise from my experience (I'm not an expert in DL either): Think of DL field as a game of lego blocks. The ideas in this book / Karpathy's videos are the basic lego blocks: parameterised linear functions, non-linearities, auto-grad, cross-entropy / KL divergence loss and gradient descent. Then there is entire body of more complex legos discovered simply by practice (alchemy!): transformer blocks, layer norm, max-pooling etc. It is impossible to understand how the second kind were obtained from first principles. The trick is to not beat yourself up about the advanced blocks too much but just play around, read up things in papers. Just focus on fundamental blocks.
I hope this motivates an effort to add great bindings to TF/Torch from Racket/Scheme.
I have none of these "The Little" books in my bookshelf, and I would gladly purchase a bunch of them. Are there any such deals anywhere?
Very cool! I have such fond memories of The Little Schemer
Not yet available in India. Hope they launch soon!
At what age is this appropriate?
Why is this book so special, could someone enlighten me?
Books in the little series are all about "learning by doing", and teach using the Socratic method (question/answer). Each concept is introduced with small problems that build upon each other. You are forced to get your hands dirty from the beginning, passive learning is not really an option. For most people, I think this helps facilitate a much deeper understanding than just reading a text.
I always had trouble with recursive functions when I was new to programming, and many recommended working through "the little schemer" to solve that problem. It was a tough read for me, but the investment was well worth it and it did for me what it said on the tin. I didn't have nearly as much trouble with recursion after that book, but an unfortunate side effect was developing an affinity for lisps which I haven't yet shaken.
I always had trouble with recursive functions when I was new to programming, and many recommended working through "the little schemer" to solve that problem. It was a tough read for me, but the investment was well worth it and it did for me what it said on the tin. I didn't have nearly as much trouble with recursion after that book, but an unfortunate side effect was developing an affinity for lisps which I haven't yet shaken.
> "learning by doing"
Too little of that in this world nowadays. Too much ios and not enough arch linux.
Too little of that in this world nowadays. Too much ios and not enough arch linux.
ok cool! thanks for explaining. do you think a beginner in the world of AI and programming would benefit from this DL book as well?
Tough to answer since I've only read the two intro chapters currently available. I'm still in the early stages myself with with Deep Learning but have years of experience with programming and lisp, so I'm coming in with a background that might bias my views.
If you have never used lisp, you need to be patient with the notation and resist the urge to balk at what is unfamiliar. I remember grumbling quite a bit when I first went through TLS, but that phase was over pretty quickly and within days I had no trouble following the code.
Are there better ways to introduce deep learning and neural networks? Maybe.. but I like "little" books and there is no better way to learn something than by building it yourself. For that reason alone I'd recommend the book sight unseen (but with the knowledge of prior little books). I do think choosing what is likely an unfamiliar language for most may be somewhat of an impediment for an already challenging topic, but scheme is a simple language and allows the author to focus on the ML concepts.
I'm highly confident you'll learn a lot, if you put in the work. For DL as a separate topic, the best resources I've found are:
If you have never used lisp, you need to be patient with the notation and resist the urge to balk at what is unfamiliar. I remember grumbling quite a bit when I first went through TLS, but that phase was over pretty quickly and within days I had no trouble following the code.
Are there better ways to introduce deep learning and neural networks? Maybe.. but I like "little" books and there is no better way to learn something than by building it yourself. For that reason alone I'd recommend the book sight unseen (but with the knowledge of prior little books). I do think choosing what is likely an unfamiliar language for most may be somewhat of an impediment for an already challenging topic, but scheme is a simple language and allows the author to focus on the ML concepts.
I'm highly confident you'll learn a lot, if you put in the work. For DL as a separate topic, the best resources I've found are:
- 3 blue 1 brown Neural Net playlist
- Karpathy's "The spelled-out intro to neural networks and backpropagation"I've read the Little Schemer and based on that, it has a unique style of teaching. Every new concept begins as a fun little question and through a series of more questions expands to a whole concept.
It worked very well for me in learning functional programming and some computational theory ideas.
Worth it.
It worked very well for me in learning functional programming and some computational theory ideas.
Worth it.
this series of books has an unusual pedagogical style, with a big "bang for the buck" in terms of building up complex systems from basic steps
there is zero fluff, almost zero narration
the books are basically just input output pairs of "now do this, and now that happens"
they are basically a sort of brain data dump for people who can think with computer code
there is zero fluff, almost zero narration
the books are basically just input output pairs of "now do this, and now that happens"
they are basically a sort of brain data dump for people who can think with computer code
Guy Steele's keynote talk at Dan Friedman's 60th-birthday academic festival, "Dan Friedman--Cool Ideas", gives lots of the background: https://youtu.be/IHP7P_HlcBk?t=2198 . Apparently the format was originally a parody or adaptation of that of an IBM Fortran instruction manual based on Skinnerian programmed-instruction ideas.
You can see the FORTRAN texts on Bitsavers: https://bitsavers.org/pdf/ibm/1130/lang/
> people who can think with computer code
In particular, Scheme. If your language of choice is, say, Python, then you'll want to get a primer on Scheme before reading this particular book. Maybe by starting at the beginning of the series.
In particular, Scheme. If your language of choice is, say, Python, then you'll want to get a primer on Scheme before reading this particular book. Maybe by starting at the beginning of the series.
It's special because it ignores the fact that the rest of the world uses Python for deep learning and it's an easy language for beginners too.
Dan Friedman and the Little X-er series.
[deleted]
Here's a preview of the first two chapters: https://mitpress.ublish.com/ebook/the-little-learner-a-strai...