Ask HN: Looking for a book on algorithms and data structures
97 comments
> For example a basic Stack is explained in a page and a half or two, for me this is not enough
Why not? It's a simple data structure. What more do you think can be said about it?
Why not? It's a simple data structure. What more do you think can be said about it?
"Foundations of Computer Science" by Aho and Ullman is an excellent book and freely available:
http://i.stanford.edu/~ullman/focs.html
http://i.stanford.edu/~ullman/focs.html
Came here to recommend the same. From the book's website:
"This book has been taken out of print by W. H. Freeman. You are welcome to use it if you like. We believed in 1992 it was the way to introduce theory in Computer Science, and we believe that today."
The book's introduction to stacks is in chapter 6, link: http://infolab.stanford.edu/~ullman/focs/ch06.pdf
"This book has been taken out of print by W. H. Freeman. You are welcome to use it if you like. We believed in 1992 it was the way to introduce theory in Computer Science, and we believe that today."
The book's introduction to stacks is in chapter 6, link: http://infolab.stanford.edu/~ullman/focs/ch06.pdf
> I want a book that can go into the details of each specific DS or Algorirhm.
I'm not aware of any book like that. At the uni, the process is repetitive and incremental.
First you learn how to implement the algo, then you learn how to count (discrete math) and, finally, you learn how to analyze its complexity. Concurrently, you might see some more details on how those data structure are used in Operating System and learn how to implement them in some lecture about parallel computing.
MIT has a great resource online. They have recorded lecture, exercises, lecture notes with reading suggestion, etc.
Their intro to algo. https://ocw.mit.edu/courses/electrical-engineering-and-compu...
Also, see what it the reading for similar lectures on other universities.
I'm not aware of any book like that. At the uni, the process is repetitive and incremental.
First you learn how to implement the algo, then you learn how to count (discrete math) and, finally, you learn how to analyze its complexity. Concurrently, you might see some more details on how those data structure are used in Operating System and learn how to implement them in some lecture about parallel computing.
MIT has a great resource online. They have recorded lecture, exercises, lecture notes with reading suggestion, etc.
Their intro to algo. https://ocw.mit.edu/courses/electrical-engineering-and-compu...
Also, see what it the reading for similar lectures on other universities.
CLRS + ocw mit course should be good in my opinion. Really helped me a lot
+1, Erik Demaine is a great lecturer.
I read CLRS in the fashion you described, mostly cover to cover, as a textbook with exercises. I found it extremely helpful. It gives both intuitive and mathematical proofs for each of the algorithms and data structures.
Sedgewick's Algorithms [1] is pretty good. I found it way more approachable than CLRS. There's also a free course on Coursera[2] from him.
1. https://www.amazon.com/Algorithms-4th-Robert-Sedgewick/dp/03...
2. https://www.coursera.org/learn/algorithms-part1#syllabus
1. https://www.amazon.com/Algorithms-4th-Robert-Sedgewick/dp/03...
2. https://www.coursera.org/learn/algorithms-part1#syllabus
Came here so say this. The booksite has additional materials and, crucially, solutions to many of the exercises
I learned most of my algorithm knowledge from Sedgewick in the 1990s. There is a series that applies them to various languages (C,C++, Java, etc) that makes them very accessible and easy to experiment with.
I had "walls and mirrors". Recently I met someone who said that was the only book they kept from uni.
I can't say if it is good or not, I lost mine well over a decade ago.
I can't say if it is good or not, I lost mine well over a decade ago.
The Algorithm Design Manual by Steven Skiena
Amazing book. Very readable. I highly recommend it. The book has a section call "War story" at the end of each chapter in which Skiena shares his real life experience of when the contents from that particular chapter came in handy for him.
Go through it. You won't regret
https://www.amazon.com/Algorithm-Design-Manual-Steven-Skiena...
Amazing book. Very readable. I highly recommend it. The book has a section call "War story" at the end of each chapter in which Skiena shares his real life experience of when the contents from that particular chapter came in handy for him.
Go through it. You won't regret
https://www.amazon.com/Algorithm-Design-Manual-Steven-Skiena...
Not OP but I’m interested in this subject as well. I took a look at this book and others that are similar and I’ve realized my foundations on math are really lacking to even understand the given examples. Since I only had the opportunity to finish my high school diploma and this was several years ago I wonder if you could have any suggestion to brush my math knowledge up in order to properly understand the examples of these books. The thing for me is that I’m not sure where I should start. Would you know any resource to introduce me to the math foundations in order to understand such texts? Or maybe math concepts that I should study before I dig in. I really appreciate it. Thanks.
This is a good starting point: https://youtube.com/playlist?list=PL7wAPgl1JVvUEb0dIygHzO469...
>Since I only had the opportunity to finish my high school diploma and this was several years ago I wonder if you could have any suggestion to brush my math knowledge up in order to properly understand the examples of these books.
You should really consider taking a few community college math courses if you're serious. Math is extremely difficult to learn on your own. Not only because of not knowing what you don't know, but because it requires intense effort and repetition which is very hard to force yourself to do. You can work through the concepts and delude yourself into thinking you understand something when really you're just hand waving it. Taking an actual course and being faced with the gaps of your knowledge by someone else is very humbling and essential to actually learning it.
You should really consider taking a few community college math courses if you're serious. Math is extremely difficult to learn on your own. Not only because of not knowing what you don't know, but because it requires intense effort and repetition which is very hard to force yourself to do. You can work through the concepts and delude yourself into thinking you understand something when really you're just hand waving it. Taking an actual course and being faced with the gaps of your knowledge by someone else is very humbling and essential to actually learning it.
That of course assumes the community college is serious about its mission to educate. Most of the students just want to pass because the math course is required for this or that vocational program, and in this age it's wiser to humour the students and wave them through.
> You can work through the concepts and delude yourself into thinking you understand something when really you're just hand waving it
I keep seeing this and I don't quite understand how this one works.
If I were studying math on my own (which I've done and still do), I'd do the following:
1. Pick a book. Say, Rudin's Principles of Mathematical Analysis[0]. Read a section, then attempt problems. Pick a problem. Say, "prove {1/n: n is natural} U{0} is compact directly from the definition(not using Heine-Borel)". It's guaranteed your "proof" is not a proof.
2. Compare your solution to existing solution manuals or ask a question on MSE[1]. Since the given book by Rudin is super-massively famous, each question has probably been asked/answered about a bagillion times each on MSE, so just searching MSE alone would likely to spit out many answers to your questions. People on MSE will tell you exactly why your "solution" is wrong and where you tripped up. Sometimes even the clarifying answers are hard to understand. But then you can ask new questions, think more, correct your misconceptions until it all finally clicks. Do that with all the rest of the problems[2].
I don't see how the process above is delusional.
[0] This book is not a realistic fit for a novice, though. Instead, one would start with books like [3], [4], [5], [6] to learn how to prove things and think like a mathematician would.
[1] https://math.stackexchange.com/
[2] In reality, the more math you see and do, the more mathematically mature and less dependent on others(to check your work) you become. In fact, if you can solve any problem in "B@by Rudin" and some famous abstract algebra textbook (say, Dummit & Foote's "Abstract Algebra") cold, you're way ahead of most any undergrad math major in the world! Because standards on undergrad math majors are not that high, nor that brutal the world over no matter what they say. If, additionally, you can solve any given problem in a book like, say, Hatcher's "Algebraic Topology" or any other famous grad level textbooks on, say, differential geometry or, uhh, functional analysis, you're officially in the big leagues. Again, if you're worried about being delusional about your proofs, you can always present them on MSE.
[3] "Book of Proof" by Richard Hammach. It's online free.
https://www.people.vcu.edu/~rhammack/BookOfProof/
[4] "Discrete Math" by Susanna Epp
https://www.google.com/books/edition/Discrete_Mathematics_wi...
[5] "How to Think About Analysis" by Lara Alcock
https://www.google.com/books/edition/How_to_Think_about_Anal...
[6] "Linear Algebra" by Kuldeep Singh
https://www.google.com/books/edition/Linear_Algebra/BJNoAgAA...
I keep seeing this and I don't quite understand how this one works.
If I were studying math on my own (which I've done and still do), I'd do the following:
1. Pick a book. Say, Rudin's Principles of Mathematical Analysis[0]. Read a section, then attempt problems. Pick a problem. Say, "prove {1/n: n is natural} U{0} is compact directly from the definition(not using Heine-Borel)". It's guaranteed your "proof" is not a proof.
2. Compare your solution to existing solution manuals or ask a question on MSE[1]. Since the given book by Rudin is super-massively famous, each question has probably been asked/answered about a bagillion times each on MSE, so just searching MSE alone would likely to spit out many answers to your questions. People on MSE will tell you exactly why your "solution" is wrong and where you tripped up. Sometimes even the clarifying answers are hard to understand. But then you can ask new questions, think more, correct your misconceptions until it all finally clicks. Do that with all the rest of the problems[2].
I don't see how the process above is delusional.
[0] This book is not a realistic fit for a novice, though. Instead, one would start with books like [3], [4], [5], [6] to learn how to prove things and think like a mathematician would.
[1] https://math.stackexchange.com/
[2] In reality, the more math you see and do, the more mathematically mature and less dependent on others(to check your work) you become. In fact, if you can solve any problem in "B@by Rudin" and some famous abstract algebra textbook (say, Dummit & Foote's "Abstract Algebra") cold, you're way ahead of most any undergrad math major in the world! Because standards on undergrad math majors are not that high, nor that brutal the world over no matter what they say. If, additionally, you can solve any given problem in a book like, say, Hatcher's "Algebraic Topology" or any other famous grad level textbooks on, say, differential geometry or, uhh, functional analysis, you're officially in the big leagues. Again, if you're worried about being delusional about your proofs, you can always present them on MSE.
[3] "Book of Proof" by Richard Hammach. It's online free.
https://www.people.vcu.edu/~rhammack/BookOfProof/
[4] "Discrete Math" by Susanna Epp
https://www.google.com/books/edition/Discrete_Mathematics_wi...
[5] "How to Think About Analysis" by Lara Alcock
https://www.google.com/books/edition/How_to_Think_about_Anal...
[6] "Linear Algebra" by Kuldeep Singh
https://www.google.com/books/edition/Linear_Algebra/BJNoAgAA...
> I keep seeing this and I don't quite understand how this one works.
> I don't see how the process above is delusional.
Because those that are just starting to learn math don't take your approach to learning.
I had a lot of bad habits that I had to break when I was learning math that really caused me to do poorly in many classes. If you are coming at it from a more qualitative field, then it's very easy to read the book and come away with nothing from it.
I'm taking a discrete math course and I was trying to offer up some advice on the school's subreddit to someone struggling in the class. They mentioned something about me taking the easier prof, so that's why I'm doing well. I said that I had fully worked at least 50 problems in each chapter we covered and asked how many they had done. The response was I went to lecture, read the notes once, and attempted the homework exercises; I didn't know you had to do some much work in this class.
Because those that are just starting to learn math don't take your approach to learning.
I had a lot of bad habits that I had to break when I was learning math that really caused me to do poorly in many classes. If you are coming at it from a more qualitative field, then it's very easy to read the book and come away with nothing from it.
I'm taking a discrete math course and I was trying to offer up some advice on the school's subreddit to someone struggling in the class. They mentioned something about me taking the easier prof, so that's why I'm doing well. I said that I had fully worked at least 50 problems in each chapter we covered and asked how many they had done. The response was I went to lecture, read the notes once, and attempted the homework exercises; I didn't know you had to do some much work in this class.
Step 1.5: Decide you're not making progress on the problem, look up the answer on MSE, say, "OK, I get that" and go on.
Alternate: Ask a question, get a not-quite-right answer, and find yourself completely stuck two chapters further in with no way of figuring out where you went wrong.
Alternate: Ask a question, get a not-quite-right answer, and find yourself completely stuck two chapters further in with no way of figuring out where you went wrong.
Art of Computer programming starts at a high school level.
It's very rigorous and considered one of the more difficult reads. But if you start at chapter 1 page 1, it covers all the math you'll ever need for the rest of the books (which is sufficient maths to reach masters or even PH.d level comp sci)
I know I've recommended it to high schoolers. A lot of math is just getting used to the nomenclature and vocabulary. The sooner you get used to rigor the better
It's very rigorous and considered one of the more difficult reads. But if you start at chapter 1 page 1, it covers all the math you'll ever need for the rest of the books (which is sufficient maths to reach masters or even PH.d level comp sci)
I know I've recommended it to high schoolers. A lot of math is just getting used to the nomenclature and vocabulary. The sooner you get used to rigor the better
AoCP is an absurdly inefficient way to learn algorithms that 99.999% of people won’t need
TAOCP is probably the best book if you approach it as infotainment/puzzles casual reading on the weekend reading about the history of trees and their mathematical properties kind of reading not anxiety filled interview passing cramming. Even autodiff online algorithms is in there something heavily used right now.
I've bought them and looked through it but if someone asks how they should go about learning algorithms for leetcode there are so many better options thank Knuth. That is years of effort to get through.
Most algorithm books don't cover the math like summations or generating functions.
Chapter one of TAOCP for better or worse, is a very rigorous mathematics introduction.
I'm not recommending it for it's algorithms (which are unfortunately somewhat out of date). I'm recommending it because it's an incredible mathematics introduction, albeit a very difficult one.
Chapter one of TAOCP for better or worse, is a very rigorous mathematics introduction.
I'm not recommending it for it's algorithms (which are unfortunately somewhat out of date). I'm recommending it because it's an incredible mathematics introduction, albeit a very difficult one.
Can you elaborate on "out of date"? I've seen a lot of books considered out of date because of their publishing date, but to what's exactly being compared to be considered so?
Well, Volume 4 was released in 2016 and is not out of date at all. Its a very good read, but on an obscure subject.
But Volume 1 has lots of out-of-date advice and is somewhat of a shame, because its otherwise an excellent introduction to computing + the mathematics needed to understand computing. For example, the assembly language is based on the 1960s computers, such as decimal computing and 6-bit numbers (based on the days of old, before 8-bits were standardized).
Functions are introduced by self-modifying code first and foremost: by rewriting "jump" instructions at the end of functions as a return. No one does this anymore: pretty much all compilers do the stack-thing instead. (Push return address onto a stack register).
The Fascicle on MMIX updates a lot of those sections to a modern-like 64-bit assembly language. However, the sections on cofunctions, arrays, garbage collection aren't part of that update... and should be updated (Knuth's discussion on these concepts is great, but are told in an "old way" based on 1960s tech)
Volume2: Seminumerical Algorithms (chapter 3 / 4) is again, a decent introduction to the subject. But the RNG stuff is fully obsolete. The statistical tests may have passed muster in the 1970s or 1980s, but today's statistical tests (aka: PractRand) go above and beyond what Knuth discusses.
There's a lot of interesting discussions in the randomness chapters: such as the efficacy of multiplication when it comes to bit-mixing (and I feel like modern RNGs are sleeping on that tidbit), but modern RNGs are generally based on a simpler sequence of ADD / XOR / SHIFT instructions based around the concept of permutations / perfect bijections.
----------------
Ironically, I consider the section on "tape sorting" to be "suddenly up-to-date" again. Modern RAM acts more like a tape (sequential is far faster than random), meaning that thinking of sorting problems in terms of external-sort is surprisingly relevant.
Given that today's cache is 768MB (see AMD's Milan-X CPUs: expected to be a ~$5000 CPU-server chip), we see that L3 cache basically serves as the RAM from Knuth's time, while today's DDR4 RAM really acts as the fast-sequential / slow-random layer that Knuth studied so much.
But Volume 1 has lots of out-of-date advice and is somewhat of a shame, because its otherwise an excellent introduction to computing + the mathematics needed to understand computing. For example, the assembly language is based on the 1960s computers, such as decimal computing and 6-bit numbers (based on the days of old, before 8-bits were standardized).
Functions are introduced by self-modifying code first and foremost: by rewriting "jump" instructions at the end of functions as a return. No one does this anymore: pretty much all compilers do the stack-thing instead. (Push return address onto a stack register).
The Fascicle on MMIX updates a lot of those sections to a modern-like 64-bit assembly language. However, the sections on cofunctions, arrays, garbage collection aren't part of that update... and should be updated (Knuth's discussion on these concepts is great, but are told in an "old way" based on 1960s tech)
Volume2: Seminumerical Algorithms (chapter 3 / 4) is again, a decent introduction to the subject. But the RNG stuff is fully obsolete. The statistical tests may have passed muster in the 1970s or 1980s, but today's statistical tests (aka: PractRand) go above and beyond what Knuth discusses.
There's a lot of interesting discussions in the randomness chapters: such as the efficacy of multiplication when it comes to bit-mixing (and I feel like modern RNGs are sleeping on that tidbit), but modern RNGs are generally based on a simpler sequence of ADD / XOR / SHIFT instructions based around the concept of permutations / perfect bijections.
----------------
Ironically, I consider the section on "tape sorting" to be "suddenly up-to-date" again. Modern RAM acts more like a tape (sequential is far faster than random), meaning that thinking of sorting problems in terms of external-sort is surprisingly relevant.
Given that today's cache is 768MB (see AMD's Milan-X CPUs: expected to be a ~$5000 CPU-server chip), we see that L3 cache basically serves as the RAM from Knuth's time, while today's DDR4 RAM really acts as the fast-sequential / slow-random layer that Knuth studied so much.
The obvious example for me is the chapter on tape merging
Modern "RAM" is far faster at sequential than it is at random access.
So tape merging is in fact, useful again strangely enough. That's one of the sections that suddenly and surprisingly has regained relevance.
So tape merging is in fact, useful again strangely enough. That's one of the sections that suddenly and surprisingly has regained relevance.
Concrete Mathematics by Graham/Knuth/Patashnik is still a great resource on the kind of mathematics and mathematical thinking we need in CS, but I'm not sure if it is fully accessible with a high-school level of Maths.
To be perfectly honest, I doubt I would've ever gotten through college-level maths without being forced to do it, as it can be very frustrating and difficult in the beginning. Unless you are quite confident in your self-discipline and enthusiasm to learn maths, rather than books I'd recommend something interactive (online course, forums, challenges).
If you are interested in a starting point to learn mathematics that are relevant for CS, I'd start with propositional logic and boolean algebra, as well as proofs via induction.
To be perfectly honest, I doubt I would've ever gotten through college-level maths without being forced to do it, as it can be very frustrating and difficult in the beginning. Unless you are quite confident in your self-discipline and enthusiasm to learn maths, rather than books I'd recommend something interactive (online course, forums, challenges).
If you are interested in a starting point to learn mathematics that are relevant for CS, I'd start with propositional logic and boolean algebra, as well as proofs via induction.
I'm not sure if this is universally true or not, but the algorithms courses at the universities I'm familiar with require a course in discrete math as a prerequisite.
You can find Discrete and Combinatorial Mathematics (an Applied Introduction), 5th Edition, Ralph P. Grimaldi online and an answer key can be found online as well. (This book covers two discrete math courses. Chapters 1-5, 7, 8, 12 is a first course in discrete math. I'm not sure what chapters the second course covers, but that requires linear algebra as a prerequisite.)
It's not perfect, but it's a start. I think you need to be familiar with some high school algebra, exponents, and logarithms. You can find some review information in the appendix. If you have troubles with recalling that information, then you can try Khan academy. (It really is an if you don't use it, you lose it situation with much of math.)
You're probably aware of this already, but most people don't read the book and come away with the knowledge required to solve the problems. You'll need to work through the examples in the chapter, be able to recall the definitions and theorems, and then work the exercises at the back of the book.
I think the discussion of proof methods is pretty poor in the book. You can find many intro to proof method type supplemental notes online to help fill in the details.
There really is so much information out there that you can pretty much always find an alternative explanation or viewpoint for undergrad level material. Many profs will post their own lecture notes, homework, and solutions. There are some math forums that have explanations. So find a book you're reasonably comfortable with and supplement it with extra material.
You can find Discrete and Combinatorial Mathematics (an Applied Introduction), 5th Edition, Ralph P. Grimaldi online and an answer key can be found online as well. (This book covers two discrete math courses. Chapters 1-5, 7, 8, 12 is a first course in discrete math. I'm not sure what chapters the second course covers, but that requires linear algebra as a prerequisite.)
It's not perfect, but it's a start. I think you need to be familiar with some high school algebra, exponents, and logarithms. You can find some review information in the appendix. If you have troubles with recalling that information, then you can try Khan academy. (It really is an if you don't use it, you lose it situation with much of math.)
You're probably aware of this already, but most people don't read the book and come away with the knowledge required to solve the problems. You'll need to work through the examples in the chapter, be able to recall the definitions and theorems, and then work the exercises at the back of the book.
I think the discussion of proof methods is pretty poor in the book. You can find many intro to proof method type supplemental notes online to help fill in the details.
There really is so much information out there that you can pretty much always find an alternative explanation or viewpoint for undergrad level material. Many profs will post their own lecture notes, homework, and solutions. There are some math forums that have explanations. So find a book you're reasonably comfortable with and supplement it with extra material.
The best book if you have any programming background is 'Mathematical Modeling and Applied Calculus' by Joel Kilty and Alex M. McAllister reviewed here: https://www.maa.org/press/maa-reviews/mathematical-modeling-...
There's a small workshop for it here: https://learnaifromscratch.github.io/calculus.html throwing in some youtube tutorials. The book presents everything as functions and their parameters, like linear functions, trig, sigmoidal, e and logarithms, you learn all the parameters to these functions and can type into desmos online graph to see what they're doing visually. You don't have to do the whole thing just use it for background material when an algorithm text uses calculus methods like L'Hopital's rule.
Poh-Shen Loh has a discrete math course open on his youtube channel https://www.youtube.com/c/DailyChallengewithPoShenLoh/search... you can use the book he recommends to look up anything that is assumed knowledge in lectures. Discrete Mathematics, by L. Lovász, J. Pelikán, and K. Vesztergombi. A book called Asymptopia by Spencer is well done too, good chapters for learning everything you want about big-O/omega/theta some topics are advanced and some anyone can do.
There's a small workshop for it here: https://learnaifromscratch.github.io/calculus.html throwing in some youtube tutorials. The book presents everything as functions and their parameters, like linear functions, trig, sigmoidal, e and logarithms, you learn all the parameters to these functions and can type into desmos online graph to see what they're doing visually. You don't have to do the whole thing just use it for background material when an algorithm text uses calculus methods like L'Hopital's rule.
Poh-Shen Loh has a discrete math course open on his youtube channel https://www.youtube.com/c/DailyChallengewithPoShenLoh/search... you can use the book he recommends to look up anything that is assumed knowledge in lectures. Discrete Mathematics, by L. Lovász, J. Pelikán, and K. Vesztergombi. A book called Asymptopia by Spencer is well done too, good chapters for learning everything you want about big-O/omega/theta some topics are advanced and some anyone can do.
Check out the course 6.042J from MIT (Mathematics for Computer Science).
https://ocw.mit.edu/courses/electrical-engineering-and-compu...
https://ocw.mit.edu/courses/electrical-engineering-and-compu...
I had exactly the same issue with this book. It brought back memories of seeing teachers solve problems on the board and skipping some obvious (to the teacher at least) step.
This prompted me to check out khan academy. Man, that is an incredible resource. I really envy the schoolchildren of today that have instant access to this incredibly smart tutor, who can be rewinded at the touch of a button.
This prompted me to check out khan academy. Man, that is an incredible resource. I really envy the schoolchildren of today that have instant access to this incredibly smart tutor, who can be rewinded at the touch of a button.
I bought the PDF last year for $7 (a very steep discount) during Apress/Springer's Black Friday sale. If you don't need the book right now or are on a budget, it may be worth waiting until next week in case they repeat the offer.
It's unfortunate that it costs 77$ :(
Edit: unfortunate for me ofc.
Edit: unfortunate for me ofc.
Goodwillbooks.com had it for $11.50. As an aside, I really love that website. I am not too picky about the books I buy being in perfect condition, so it works out well for me. YMMV.
$30 used on ebay
Note, there seems to be a 3rd Edition in 2020
https://www.amazon.com/Algorithm-Design-Manual-Computer-Scie...
https://www.amazon.com/Algorithm-Design-Manual-Computer-Scie...
> The Algorithm Design Manual by Steven Skiena
And his CSE 373 course lectures: https://www3.cs.stonybrook.edu/~skiena/373/videos/
And his CSE 373 course lectures: https://www3.cs.stonybrook.edu/~skiena/373/videos/
Wow. I actually took one of his classes way back at Stony Brook. Great guy.
I came here precisely to suggest Skiena. Excellent book that gives you a good grounding in both theory and practice, and opens the road for further study, if you're so inclined.
just move on to leetcode. you just memorize the bullshit questions and their answers. that is what everyone seems to be doing. actual experience makes no difference. you have 2 minutes to produce the exact answer expected.
"A Common-Sense Guide to Data Structures and Algorithms, 2nd Edition" by Jay Wengrow is one I'd recommend every time. It really makes it easy enough to understand things I previously only considered to be the domain of math-oriented people. Not sure if it goes deep enough for you though...
Yes, wholeheartedly agree. All other classic recommendations from this thread pale in comparison. Jay Wengrow did a fantastic job with this book.
If you want basics and details, the art of computer programming is a good candidate.
Unfortunately, it isn’t done yet (1), and the completed parts have are very basic and have parts that are a bit dated (multi-tape sorting, for example)
(1) and likely never will be. Knuth is turning 84 in two months, and isn’t even halfway writing it, after almost 60 years
Unfortunately, it isn’t done yet (1), and the completed parts have are very basic and have parts that are a bit dated (multi-tape sorting, for example)
(1) and likely never will be. Knuth is turning 84 in two months, and isn’t even halfway writing it, after almost 60 years
It would be fitting if the work were carried on by a committee, like Bourbaki.
That would be harder than Bourbaki, I guess. They had the benefit of a blank slate.
It would have to be done in Knuth’s spirit, and there would be lots to argue over what to include, before even writing about it
How complete are Knuth’s notes? Would he have found Foo important enough to include, even though his notes don’t mention it? Should the completion include stuff discovered after his death? Should earlier volumes be revisited, as Knuth probably would do, if he had an eternity to live?
It would have to be done in Knuth’s spirit, and there would be lots to argue over what to include, before even writing about it
How complete are Knuth’s notes? Would he have found Foo important enough to include, even though his notes don’t mention it? Should the completion include stuff discovered after his death? Should earlier volumes be revisited, as Knuth probably would do, if he had an eternity to live?
The Art of computer programming by Donald Knuth.
TAOCP is too detailed and too deep as a beginner's book; you first want something lighter that you can worth through end-to-end before taking on a 7-volume book that is still not even finished (volume 4b of 7 in progress at the time of writing).
Note I am _not_ saying TAOCP isn't readable, I consider it very well written. But it's terse, hard, too deep and too detailed for a beginner, IMHO.
Note I am _not_ saying TAOCP isn't readable, I consider it very well written. But it's terse, hard, too deep and too detailed for a beginner, IMHO.
Back when I was in high school I learned all leetcode algorithms from doing programming competitions. One of the books I used was: http://acm.cs.buap.mx/downloads/Programming_Challenges.pdf
What really put me to the next level is this amazing free training program which I still recommend anyone who wants to nail the interviews. It teaches each algorithm and data structure and gives you appropriate challenging exercises so you really understand them: https://train.usaco.org/
What really put me to the next level is this amazing free training program which I still recommend anyone who wants to nail the interviews. It teaches each algorithm and data structure and gives you appropriate challenging exercises so you really understand them: https://train.usaco.org/
I would recommend Open Data Structures: https://opendatastructures.org/
It's available for free online, or if you'd like you can buy a physical copy. It's got 14 chapters that take you through a variety of data structures, introduces you to the theory behind them, and teaches you the basics of analyzing them.
I've done many of the exercises from this book, and found them useful for learning the topic.
It's available for free online, or if you'd like you can buy a physical copy. It's got 14 chapters that take you through a variety of data structures, introduces you to the theory behind them, and teaches you the basics of analyzing them.
I've done many of the exercises from this book, and found them useful for learning the topic.
"Algorithms Illuminated" by Roughgarden. Very accessible. Reads like a novel not a textbook. Has exercises throughout the chapter to test knowledge. I recommend it over your suggestion that you want a textbook. This book will serve you much better.
I found his courses rigorous and approachable too. I think they were equivalent to the courses he taught while at Stanford.
https://timroughgarden.org/videos.html
https://timroughgarden.org/videos.html
Is there a way to buy ebooks from the seller other than Amazon (for those of us who hate Jeff Bezos ;) ?
[deleted]
Unfortunately no. Possibly email the author?
http://www.algorithmsilluminated.org/
http://www.algorithmsilluminated.org/
Another vote for Tim Roughgarden’s work. As someone who recently became a software engineer without a cs degree, his Stanford Algorithms I & II lectures (similar content to his books) have been the best instruction I’ve had on DS&A. I’ve dabbled with Skiena’s book, Grokking Algorithms, CTCI, and I now also have a copy CLRS that I use for reference or to go deeper on certain things. But I revisit Tim’s work all the time. Can’t recommend him enough.
Shameless plug: you might want to accompany a book with explained animations of Algorithms and Data Structures: https://www.chrislaux.com
In case you want to go for something aimed at competitive programming have a look at the "Competitive Programmer’s Handbook"
https://cses.fi/book/book.pdf
My personal favourite is Sedgewick & Wayne's _Algorithms_.
https://algs4.cs.princeton.edu/home/
You can even find an excellent two-part MOOC on Coursera:
https://www.coursera.org/learn/algorithms-part1
https://www.coursera.org/learn/algorithms-part2
Maybe it's not the "purest" class or book, but it's engaging and it lets you understand how algos work AND how to use them in practice.
https://algs4.cs.princeton.edu/home/
You can even find an excellent two-part MOOC on Coursera:
https://www.coursera.org/learn/algorithms-part1
https://www.coursera.org/learn/algorithms-part2
Maybe it's not the "purest" class or book, but it's engaging and it lets you understand how algos work AND how to use them in practice.
Sedgewick is a masterpiece but also overkill for interview prep. Interviews are mostly trivia and for that leet codes volume is your friend. Buy sedgewick for your long term growth but it will stand in the way of any near term interview prep, at least it did for me.
The annoying thing as you note is that in the long run Sedgewick >> Leetcode but in the short run it's probably reversed as whiteboard interviews often present algorithm puzzles that are variants of "have you seen this trick before?" and closer to Leetcode questions.
I wonder though if "programming contest" (usually more like "algorithm puzzle contest") books like Skiena's might be useful, though I can't help but think that doing an algorithms course first shouldn't be useless.
I wonder though if "programming contest" (usually more like "algorithm puzzle contest") books like Skiena's might be useful, though I can't help but think that doing an algorithms course first shouldn't be useless.
I usually don't complete online courses(I get bored watching educational videos) except for one. This course is on data structures and is conducted by University of California, San Diego. It is available both on edX[0] and Stepik[1]. The reason this online course is so engaging is because it uses the active-learning way of teaching. I was surprised how well that technique worked!
[0] https://www.edx.org/course/data-structures-an-active-learnin...
[1] https://stepik.org/579
[0] https://www.edx.org/course/data-structures-an-active-learnin...
[1] https://stepik.org/579
Yes there’s lots of algorithm books! Too many to learn them all. I had an anxiety attack and bought most of the introductory algo books mentioned in CLRS chapter 1.
I liked ALGS4 by Sedgewick as my introduction to Algorithms. I still refer to the Coursera course. I actually started with CLRS and doubled back to ALGS4. I wish I did it in reverse order but it was still useful reading CLRS and refreshing my math skills.
I would say grab a few and see which ones you like. CLRS, ALGS4, Algorithm Design Manual, and Algorithm Design by Kleinberg and Tardos are a good start. You can look in the bibliography of each books for more suggestions from the experts themselves.
I liked ALGS4 by Sedgewick as my introduction to Algorithms. I still refer to the Coursera course. I actually started with CLRS and doubled back to ALGS4. I wish I did it in reverse order but it was still useful reading CLRS and refreshing my math skills.
I would say grab a few and see which ones you like. CLRS, ALGS4, Algorithm Design Manual, and Algorithm Design by Kleinberg and Tardos are a good start. You can look in the bibliography of each books for more suggestions from the experts themselves.
Algorithms in a nutshell by George T. Heineman helped me a lot. It's not too dense so it's easy to read, just enough to learn and understand how an algorithm works.
CLRS is what I used in college
If your goal is to be good at LeetCode, then save yourself the time and just go straight to LeetCode. Nobody cares if you actually understand the stuff. They just want you to come in and do the dance (recognize the problem archetype, talk through your thought process for solving it, and adjust based on whatever twists they've thrown in).
If you're interviewing for senior positions, it is worthwhile to read through Designing Data Intensive Systems. This will prepare you for System Design interviews where you do need to actually understand stuff.
If you're interviewing for senior positions, it is worthwhile to read through Designing Data Intensive Systems. This will prepare you for System Design interviews where you do need to actually understand stuff.
> If you're interviewing for senior positions, it is worthwhile to read through Designing Data Intensive Systems. This will prepare you for System Design interviews where you do need to actually understand stuff.
Do you mean "Designing Data Intensive Applications" by Martin Kleppmann? That's what most of my search hits found. If something else, do you have a link? Thanks.
Do you mean "Designing Data Intensive Applications" by Martin Kleppmann? That's what most of my search hits found. If something else, do you have a link? Thanks.
Yep, sorry that's the book I'm referring to.
Thanks, the table of contents looks promising.
That's a good read regardless of your level.
I find the writing style in CLRS to be quite readble. Sure the book is huge but so maybe pick and choose chapters but each chapter can be read "cover to cover". Perhaps some of the stack details you are looking for are left to exercises in CLRS? Most good "textbooks" will do this. Perhaps you don't want a general textbook on data structures but want a deep dive article on stacks.
I admit I haven't explored many other books on this topic but CLRS is very good in my opinion.
I admit I haven't explored many other books on this topic but CLRS is very good in my opinion.
Segwick's book is far more useful in self-learning environment than other text books. He has videos and fully functioning code to help you understand.
A great and quite unconventional book on Algorithms is "INTRODUCTION TO ALGORITHMS - A Creative Approach" by Udi Manber. After leaving academia the author was hired first by Amazon (A9.com) and then by Google. At some point around 2010 he was responsible for all search products at Google [0].
The book not only describes the algorithms per se, but also teaches how to think about the algorithms and the problems they are solving. I enjoyed it very much as a complimentary reading to the classic Knuth volumes.
[0] https://en.wikipedia.org/wiki/Udi_Manber
The book not only describes the algorithms per se, but also teaches how to think about the algorithms and the problems they are solving. I enjoyed it very much as a complimentary reading to the classic Knuth volumes.
[0] https://en.wikipedia.org/wiki/Udi_Manber
Excellent book if you can find it. I kick myself for selling my hardback copy.
"Foundations of Computer Science" by Aho and Ullman is my personal favourite.
CLRS is pretty accessible for beginners, at least as far as computer science textbooks go.
Perhaps you would benefit from pairing it with a MOOC like MIT's Introduction to Algorithms. https://ocw.mit.edu/courses/electrical-engineering-and-compu...
Perhaps you would benefit from pairing it with a MOOC like MIT's Introduction to Algorithms. https://ocw.mit.edu/courses/electrical-engineering-and-compu...
Introduction to Algorithms -Thomas H. Cormen
Grokking Algorithms -Aditya Bhargava
Cracking the Coding Interview Book by Gayle Laakmann McDowell
Grokking Algorithms -Aditya Bhargava
Cracking the Coding Interview Book by Gayle Laakmann McDowell
The good thing about Algorithm Design & Analysis as a topic is that there is no lack of great resources to learn from! And even better, a large portion of them are freely available online (in forms of lecture notes, or entire books, video lectures, programming challenges with online judging, etc).
In addition to what others have mentioned, here are some example resources you might prefer for a beginner-intermediate level intro:
1. (free online) Algorithms by Dasgupta, Papadimitriou, and Vazirani http://algorithmics.lsi.upc.edu/docs/Dasgupta-Papadimitriou-...
2. (free online) Algorithms by Jeff Erickson https://jeffe.cs.illinois.edu/teaching/algorithms/
3. Algorithm design by Kleinberg & Tardos https://www.amazon.com/Algorithm-Design-Jon-Kleinberg/dp/032...
4. Another one specifically for more applied view (esp., how they are used in programming contests such as ICPC) is Skiena & Revilla's "Programming Challenges" book (https://www.amazon.com/Programming-Challenges-Contest-Traini...). Note that this is different than Skiena's other popular book (Algorithm Design manual) which is also pretty good and has a "war story" based perspective to design of algorithms.
5. There are also several resources where lecture notes from university Algorithm & DS courses are very useful. Here is an example from my previous Professor, David Kempe: http://david-kempe.com/teaching/DataStructures.pdf
6. Several programming competition specific tutorials can be found on Topcoder: https://www.topcoder.com/thrive/tracks?track=Competitive%20P... (individual SRM archives are also good place to try problems first hand and then learn from other's approach). In general, if you search for ACM-ICPC resources, you will find a lot more targeted information/problems which will apply not only for leetcode, but also for detailed understanding of the theory too.
In addition to what others have mentioned, here are some example resources you might prefer for a beginner-intermediate level intro:
1. (free online) Algorithms by Dasgupta, Papadimitriou, and Vazirani http://algorithmics.lsi.upc.edu/docs/Dasgupta-Papadimitriou-...
2. (free online) Algorithms by Jeff Erickson https://jeffe.cs.illinois.edu/teaching/algorithms/
3. Algorithm design by Kleinberg & Tardos https://www.amazon.com/Algorithm-Design-Jon-Kleinberg/dp/032...
4. Another one specifically for more applied view (esp., how they are used in programming contests such as ICPC) is Skiena & Revilla's "Programming Challenges" book (https://www.amazon.com/Programming-Challenges-Contest-Traini...). Note that this is different than Skiena's other popular book (Algorithm Design manual) which is also pretty good and has a "war story" based perspective to design of algorithms.
5. There are also several resources where lecture notes from university Algorithm & DS courses are very useful. Here is an example from my previous Professor, David Kempe: http://david-kempe.com/teaching/DataStructures.pdf
6. Several programming competition specific tutorials can be found on Topcoder: https://www.topcoder.com/thrive/tracks?track=Competitive%20P... (individual SRM archives are also good place to try problems first hand and then learn from other's approach). In general, if you search for ACM-ICPC resources, you will find a lot more targeted information/problems which will apply not only for leetcode, but also for detailed understanding of the theory too.
I like:
algorithms.wtf
Algorithms by Jeff Erickson
algorithms.wtf
Algorithms by Jeff Erickson
I visited a friend a while ago and she was reading this amazing book on her tablet, "The AgloDaily Book: Core Essentials"
https://algodaily.gumroad.com/l/yYEqr
I really loved its illustrations; it looks so easy to follow.
https://algodaily.gumroad.com/l/yYEqr
I really loved its illustrations; it looks so easy to follow.
in case you want to code along, Ive been through some of the books mentioned here and solved the algorithms. You can check it out at https://github.com/realpacific/algorithms
I recommend to start with "Elements of Programming Interviews (in Java or Python)". It serves as a nice practical overview of the main algorithms and data structures and when to use them. Personally I found it more complete than the usually recommended "Cracking the Coding Interview" while being equally approachable.
Best I can offer is for data structure think about working backwards if possible.
Every time I've been involved in projects using 10TB+ data it's the processing that tends to expose the most bottlenecks. Nothing competes with making a pseudo dataset about 5%-10% of the size and running benchmarks which roughly match your expected workflow.
Good luck
Every time I've been involved in projects using 10TB+ data it's the processing that tends to expose the most bottlenecks. Nothing competes with making a pseudo dataset about 5%-10% of the size and running benchmarks which roughly match your expected workflow.
Good luck
It really depends. It is a bit difficult to answer this question without understanding your background first. What CS lectures have you already taken? What related courses (e.g., math) have you done?
For example, the Stack data structure that you bring into discussion is quite simple (but interesting and important), so I am not surprised it is treated speedily. Introduction to Algorithms typically has more difficult content.
Depending on this, I could recommend for example that you follow the lectures on the excellent MIT OCW YouTube channel (https://www.youtube.com/c/mitocw). But you could also do one of the numerous algorithms and data structures courses on coursera/edx. It all depends on what you know so far and what your goal is.
For example, the Stack data structure that you bring into discussion is quite simple (but interesting and important), so I am not surprised it is treated speedily. Introduction to Algorithms typically has more difficult content.
Depending on this, I could recommend for example that you follow the lectures on the excellent MIT OCW YouTube channel (https://www.youtube.com/c/mitocw). But you could also do one of the numerous algorithms and data structures courses on coursera/edx. It all depends on what you know so far and what your goal is.
CLRS does about 5 pages on linked lists. If you want more, my advice would be to implement one in your language of choice, then look at the API for the standard lib impl of a linked list in your language of choice (if there is one).
Otherwise, for more reading on linked lists, read through adlist.c from redis, by antirez.
Next, one of the best books (IMO) on algorithmic thinking is SICP. Get through a few chapters, do the exercises, grok recursion, etc.
In no particular order:
Algorithm Design Manual, Skiena
Algorithm Design, by Kleinberg/Tardos
Algorithms, Sedgwick 4th ed. (also Algorithms in C)
If you find you need or would like more math:
Discrete Mathematics (Epp is an easier read, Rosen seems more verbose)
Mathematical Proofs, Charrand
For interview prep:
Elements of Programming Interviews (EPI)
Designing Data Intensive Applications
Otherwise, for more reading on linked lists, read through adlist.c from redis, by antirez.
Next, one of the best books (IMO) on algorithmic thinking is SICP. Get through a few chapters, do the exercises, grok recursion, etc.
In no particular order:
Algorithm Design Manual, Skiena
Algorithm Design, by Kleinberg/Tardos
Algorithms, Sedgwick 4th ed. (also Algorithms in C)
If you find you need or would like more math:
Discrete Mathematics (Epp is an easier read, Rosen seems more verbose)
Mathematical Proofs, Charrand
For interview prep:
Elements of Programming Interviews (EPI)
Designing Data Intensive Applications
EPP is a great intro for those without the preexisting math. Almost nothing is taken for granted, even high school math.
Not a book but I've been working on a project that sounds like it might be what you're looking for. It's algos and data structures courses in code-along style. It's free to audit, so do check it out even if you aren't interested in the certification stuff :)
https://qvault.io
https://qvault.io
There are literally hundreds of threads with this topic on the Web. Most of them contain the same standard references which IMO are not always the best for people without much mathematical maturity and wishing to learn from first principles. That said, here are my recommendations for them (pdfs of almost all are available online and they can also be purchased cheaply as used books);
* 1) Functions and Graphs and 2) The Method of Coordinates by I.M.Gelfand et. al. - These give you the basics on functions and how to visualize them which is crucial for understanding.
* Introductory Logic and Sets for Computer Scientists by Nimal Nissanke - Nice succinct introduction to various required topics for Comp. Sci.
* Compared To What?: Introduction to the Analysis of Algorithms by Gregory Rawlins - This is a classic book which takes you by hand and walks you through Algorithm Analysis and the reqd. mathematics. You will find this more approachable than most other text books and well worth a study.
* How to Think about Algorithms by Jeff Edmonds - Practical techniques without too much mathematical formalism.
* Data Structure Techniques by Thomas Standish - A Classic text; good adjunct to more modern texts.
* Advanced Data Structures by Peter Brass - Nice succinct text but not necessarily "advanced".
One final point; try to get a Algorithms and Data Structures book with code in the language with which you are most familiar. That way you understand the mapping from "Abstract Algorithm" to "Concrete Language Implementation" which is not always trivial.
* 1) Functions and Graphs and 2) The Method of Coordinates by I.M.Gelfand et. al. - These give you the basics on functions and how to visualize them which is crucial for understanding.
* Introductory Logic and Sets for Computer Scientists by Nimal Nissanke - Nice succinct introduction to various required topics for Comp. Sci.
* Compared To What?: Introduction to the Analysis of Algorithms by Gregory Rawlins - This is a classic book which takes you by hand and walks you through Algorithm Analysis and the reqd. mathematics. You will find this more approachable than most other text books and well worth a study.
* How to Think about Algorithms by Jeff Edmonds - Practical techniques without too much mathematical formalism.
* Data Structure Techniques by Thomas Standish - A Classic text; good adjunct to more modern texts.
* Advanced Data Structures by Peter Brass - Nice succinct text but not necessarily "advanced".
One final point; try to get a Algorithms and Data Structures book with code in the language with which you are most familiar. That way you understand the mapping from "Abstract Algorithm" to "Concrete Language Implementation" which is not always trivial.
Are they any algorithms books that don't use data structures backed by linked lists but rather arrays or vectors? Linked data structures are barely used in practice. It can take a student weeks to go through Linked Lists and data structures using them, just to find that they're hardly ever used in practice. Contiguous data structures are the dominant types used today.
I better recommend you to try your hands on code-forces, CP Algorithm etc. If you got stuck in any of the problems just read the discussion and given solution. If you continue this you several months then you quickly learning the data structures and algorithms. Because if you don't apply DSA which you will learn from any book then i guess it doesn't add much value to you.
In the beginning problems might not able to solve but after giving time and looking solution you will start getting things. And I guess its just a myth or we can say a phobia that you need to know each and every piece of mathematics.
And I think graph is a data structure that you should be work upon. For this you can refer this book Introduction to Graph Theory Subsequent Edition by Douglas Brent West. This book you can just read and practice while solving problems on above coding problem platforms.
So, I recommend you first do code-forces DIV3, DIV2 problems, gain some confidence, and then pick any DSA book.
Or one thing you can also do just filter out problems related to math,Combinatorics etc.on code-forces and try to solve them. Through this approach you will learn to apply the concepts.
I hope this comment add some value to you! Happy coding.
And I think graph is a data structure that you should be work upon. For this you can refer this book Introduction to Graph Theory Subsequent Edition by Douglas Brent West. This book you can just read and practice while solving problems on above coding problem platforms.
So, I recommend you first do code-forces DIV3, DIV2 problems, gain some confidence, and then pick any DSA book.
Or one thing you can also do just filter out problems related to math,Combinatorics etc.on code-forces and try to solve them. Through this approach you will learn to apply the concepts.
I hope this comment add some value to you! Happy coding.
I'm only a student, but I personally really liked https://jeffe.cs.illinois.edu/teaching/algorithms/
This is better for more advanced topics, like dynamic programming (well, advanced for me, anyway). I started out taking over an hour to solve the first problem in the problem sets, but they build on each other slowly, so I was soon able to see solutions within minutes. It took me a weekend to go through chapters I was struggling with, and I did really well on my coding interview the next day.
This is better for more advanced topics, like dynamic programming (well, advanced for me, anyway). I started out taking over an hour to solve the first problem in the problem sets, but they build on each other slowly, so I was soon able to see solutions within minutes. It took me a weekend to go through chapters I was struggling with, and I did really well on my coding interview the next day.
Im not good at maths but to get my work done this books helped me a lot:
https://www.amazon.de/Problem-Solving-Algorithms-Structures-... If I remember correctly it is Python 2
https://www.amazon.de/Algorithms-Illuminated-Part-1-Basics/d...
https://www.amazon.de/Algorithms-Illuminated-Part-Graph-Stru...
https://www.amazon.de/Algorithms-Illuminated-Part-Dynamic-Pr...
https://www.amazon.de/Problem-Solving-Algorithms-Structures-... If I remember correctly it is Python 2
https://www.amazon.de/Algorithms-Illuminated-Part-1-Basics/d...
https://www.amazon.de/Algorithms-Illuminated-Part-Graph-Stru...
https://www.amazon.de/Algorithms-Illuminated-Part-Dynamic-Pr...
Truth is, if you want to really learn algorithms and data structures, then you are the one who has to "go into the details". CLRS is the bible, I can't think of a better book, and you don't have to read it sequentially because most chapters are independent of each other.
I’m in the initial phase of structuring a book that aims to teach some simple algorithms and data structures using Python. The idea is that the reader should do the (guided) exercises to learn how to build those data structures. It’s basically “learn by doing”.
For instance, to teach what a stack is I’ll explain the basic idea of a stack, then provide a base class and the reader must implement each method (init, push, pop, peek, etc) given the requirements and intended results of each functionality.
I already have a kind-of “learn by doing” ebook about Python (https://github.com/joaoventura/full-speed-python), so it’s like a follow-up to that one.
Don’t know if people are interested in a book like this, though..
For instance, to teach what a stack is I’ll explain the basic idea of a stack, then provide a base class and the reader must implement each method (init, push, pop, peek, etc) given the requirements and intended results of each functionality.
I already have a kind-of “learn by doing” ebook about Python (https://github.com/joaoventura/full-speed-python), so it’s like a follow-up to that one.
Don’t know if people are interested in a book like this, though..
For algorithms, I can't think of a book better than CLRS, honestly, unless you want to read The Art of Computer Programming by Knuth. For data structures, a great book with a huge amount of detail is Data Structures Using C and C++ by Langsam and Tanenbaum.
tl;dr: No
But I want a structured guide / textbook that I can follow which also contains exercises.
I have checked the CLRS book, but, it's more of a reference than a book that you can read from cover to cover. For example a basic Stack is explained in a page and a half or two, for me this is not enough, I want a book that can go into the details of each specific DS or Algorirhm.
*TL;DR:* Are there any books better than CLRS for DS & Algos?