Ask HN: How to learn algorithms and become a good programmer?
18 comments
Coursera has a couple of courses on algorithms that would be acceptable for beginners but also very challenging (in a good way).
MIT OCW's Intro to Computer Science and Programming (in Python) covers a lot of ground related to programming (more than just Algos), but it covers some essential sorting algorithms and eventually gets into dynamic programming. I don't think anything much simpler than this would be worth your time (not to say this is the best), but the readings themselves (especially those from wikipedia) can actually be a little dense: http://ocw.mit.edu/courses/electrical-engineering-and-comput...
MIT OCW also has a full course on algorithms.
MIT OCW's Intro to Computer Science and Programming (in Python) covers a lot of ground related to programming (more than just Algos), but it covers some essential sorting algorithms and eventually gets into dynamic programming. I don't think anything much simpler than this would be worth your time (not to say this is the best), but the readings themselves (especially those from wikipedia) can actually be a little dense: http://ocw.mit.edu/courses/electrical-engineering-and-comput...
MIT OCW also has a full course on algorithms.
I enjoyed Coursera's two part Algorithms: Design and Analysis series. For me the additional structure of the time box is useful. It's also helpful to have a cohort to discuss the exercises with. Unlike the Sedgewick course, it is language agnostic, so it is also an opportunity to learn more about any language.
I followed Coursera and the Karatsuba Multiplication algorithm made me reignite my interest in algorithms and data structures. Coursera is a good start followed by the Data Structures and Algorithms in Java book. Learning logarithmic analysis is a prerequisite in my opinion.
Thanks for the suggestion! Can you please suggest any book that deals with algorithms in python or other "non-heavy" languages ? I mean to ask, that i haven't invested much time in learning java. All i have learnt about computers is on the job and i am more comfortable working with scripting languages and / or C. Hence the request.
I come from a non-CS background too, and currently my interest in software development is mostly as a hobby and to build a base I might use in a job later on.
Remember that "making stuff work", aka prototyping, is incredibly important, and in fact an extremely desirable skill: in so many situations, it can be impossible to see the forest for the trees when building a new idea or architecture, and the design is still on paper; things frequently need to be built out into working first drafts for edge cases or gotchas to become apparent. Often, the result is that everyone might need to go back to the drawing board and start from scratch, so rapid turnaround is essential.
Going beyond the basics in any domain is a categorical form of specialization, and specialization in any context tends to require a lot of investment, and the return it gives predominantly leans toward inflexibility and a narrow field of application.
When tackling arbitrary, ambiguous problems, the statistical likelihood your specialized knowledge will come in handy is, on average, going to be quite low, unless you're in an academic environment that focuses on your specialty.
So don't shun the skillset you already have in "making things work." :D
By all means dive into algorithms - just remember the info will likely stay in if you do so at a slow, non-anxious pace and keep your experiences positive ^^
Remember that "making stuff work", aka prototyping, is incredibly important, and in fact an extremely desirable skill: in so many situations, it can be impossible to see the forest for the trees when building a new idea or architecture, and the design is still on paper; things frequently need to be built out into working first drafts for edge cases or gotchas to become apparent. Often, the result is that everyone might need to go back to the drawing board and start from scratch, so rapid turnaround is essential.
Going beyond the basics in any domain is a categorical form of specialization, and specialization in any context tends to require a lot of investment, and the return it gives predominantly leans toward inflexibility and a narrow field of application.
When tackling arbitrary, ambiguous problems, the statistical likelihood your specialized knowledge will come in handy is, on average, going to be quite low, unless you're in an academic environment that focuses on your specialty.
So don't shun the skillset you already have in "making things work." :D
By all means dive into algorithms - just remember the info will likely stay in if you do so at a slow, non-anxious pace and keep your experiences positive ^^
Very true. But the problem I face is that the so called cool and big impact companies interview based on CS skillset which I do not have. Also I am trying to move into companies where my work can have an impact. These companies either reject applications based on my non-cs background or since CS core is still lacking and am still working on the same.
there are so many algorithms
in so many different areas.
it may work to start with just one area, an area that is close to something you already are making or solving, and then, literally, in searching for algorithms for that pick the shortest one and solve a small implementation of it.
in finding a reference implementation i found the following helpful :
-- pick code in the same language you will be writing your version in.
-- if you can, find and read the original paper ( try searching google scholar ) describing the algorithm, these are often clearly written and explain the motivating situations that led to the way the algorithm was constructed.
finally some example short algorithms from a few areas and favorites of mine :
string processing :
-- LZW algorithm
-- longest common subsequence algorithm
tree graph processing :
-- post order depth first search
list processing :
-- group elements in an array into sub arrays of length k
integer processing :
-- change of radix algorithm ( change a mumber from the given base ( 10 ) to base 2 ( or even to other bases as well ) )
most of all, just work your way through step by step, like the algorithms your are making.
it may work to start with just one area, an area that is close to something you already are making or solving, and then, literally, in searching for algorithms for that pick the shortest one and solve a small implementation of it.
in finding a reference implementation i found the following helpful :
-- pick code in the same language you will be writing your version in.
-- if you can, find and read the original paper ( try searching google scholar ) describing the algorithm, these are often clearly written and explain the motivating situations that led to the way the algorithm was constructed.
finally some example short algorithms from a few areas and favorites of mine :
string processing :
-- LZW algorithm
-- longest common subsequence algorithm
tree graph processing :
-- post order depth first search
list processing :
-- group elements in an array into sub arrays of length k
integer processing :
-- change of radix algorithm ( change a mumber from the given base ( 10 ) to base 2 ( or even to other bases as well ) )
most of all, just work your way through step by step, like the algorithms your are making.
Thanks for the advice! Trust me, i haven't heard of some of the things you have mentioned above. So it definitely helps!
I suggest that you work patiently through a good text on algorithms and data structures that has code samples in it. I recommend "Algorithms I and II" by Sedgewick or "Data Structures and Algorithm Analysis in Java" (or the C++ version) by Weiss. I also hear good things about "Data Structures and Algorithms in Java" by Lafore.
The above books are quite apt for beginners. You should expect to struggle a little but you must persevere anyway. I remember struggling with sorting and matrix multiplication algorithms when I learned it the first time in school. Do not mistake your unfamiliarity with the subject for its inherent difficulty! Once you write a few sample programs and see how the algorithms play out, you'll get the hang of it.
The above books are quite apt for beginners. You should expect to struggle a little but you must persevere anyway. I remember struggling with sorting and matrix multiplication algorithms when I learned it the first time in school. Do not mistake your unfamiliarity with the subject for its inherent difficulty! Once you write a few sample programs and see how the algorithms play out, you'll get the hang of it.
Is there any book that teaches algorithms in say a language like python ( which i find comfortable to use ). Most books focus on C,C++ and/or java. I am not good at C++ or java. Since most books give examples out in C++, i tend to find them a little intimidating.
If you do not have a math background , I suggest you start with Logarithmic analysis and properties of logarithms ( many tutorials in youtube ), then attend the Coursera algorithm courses and some selected courses on CS50's youtube channel.
Often, the first examples of computer algorithms involve sorting. There will be some discussion of "bad" sorting algorithms that run in O(n^2) time. This will illustrate how to use algorithmic analysis to understand time complexity. Then the faster merge-sort will be introduced as an obvious improvement to O(n log n) time, followed by quicksort with O(n log n) time and O(n) space...and at this point you may reasonably ask, "So what?"
The thing is is that merge-sort was discovered by John von Neumann [1] and QuickSort by CAR Hoare [2]. The less accomplished of the two won a Turing Award and one might quite legitimately walk away with the belief that developing algorithms from scratch may be non-trivial simply because the subject is really hard.
Which is a round about way of coming to some advice: if the very subject of algorithms isn't enough to scare you off, don't fret the fact that the presentation looks difficult. The apparent difficulty is in the subject itself and the only thing a gentle introduction or simplification means is that someone is pretending that the subject of algorithms is not really really hard with the natural consequence that when it does get hard, you're more likely to see the struggle as a failing of the student rather as inherent in the subject matter. And everything interesting and important in algorithms is hard.
This is why I don't hesitate to suggest people consider TAoCP. It covers the "easy" parts as well as or better than anything else I've seen. It just requires being ok with not understanding everything or even most things that Knuth covers because he finds them important and interesting. Because I'm never going to be that smart, I might as well work at my own pace, and celebrate the small victories like grasping the answer to one of the easy exercises that a year ago, I wouldn't have grasped.
The other recommendation is to take an algorithms course on Coursera. It's free.
Good luck.
[1]: https://en.wikipedia.org/wiki/John_von_Neumann
[2]: https://en.wikipedia.org/wiki/Tony_Hoare
The thing is is that merge-sort was discovered by John von Neumann [1] and QuickSort by CAR Hoare [2]. The less accomplished of the two won a Turing Award and one might quite legitimately walk away with the belief that developing algorithms from scratch may be non-trivial simply because the subject is really hard.
Which is a round about way of coming to some advice: if the very subject of algorithms isn't enough to scare you off, don't fret the fact that the presentation looks difficult. The apparent difficulty is in the subject itself and the only thing a gentle introduction or simplification means is that someone is pretending that the subject of algorithms is not really really hard with the natural consequence that when it does get hard, you're more likely to see the struggle as a failing of the student rather as inherent in the subject matter. And everything interesting and important in algorithms is hard.
This is why I don't hesitate to suggest people consider TAoCP. It covers the "easy" parts as well as or better than anything else I've seen. It just requires being ok with not understanding everything or even most things that Knuth covers because he finds them important and interesting. Because I'm never going to be that smart, I might as well work at my own pace, and celebrate the small victories like grasping the answer to one of the easy exercises that a year ago, I wouldn't have grasped.
The other recommendation is to take an algorithms course on Coursera. It's free.
Good luck.
[1]: https://en.wikipedia.org/wiki/John_von_Neumann
[2]: https://en.wikipedia.org/wiki/Tony_Hoare
Hey thanks for your suggestions! I have started learning algorithms multiple times and dropped it mid way since either i find it intimidating or i am stuck and i can't find help. I am giving this another shot. Hopefully i make it.
[deleted]
One thing I really liked was using Project Euler. Step 1 for any given problem was to solve it. Step 2 was to go through code written by others benchmark it, and learn why there code was faster than mine.
https://projecteuler.net
https://projecteuler.net
I found the Khan Academy course on algorithms to be a good introduction. It doesn't get too in depth, but does explain things well and has some interactive coding examples.
I am from a non-CS background. I find myself able to make stuff work and resolve issues in case any pop up, but i do not find myself writing the most optimized code.
How do i learn algorithms ? The problem is that the suggestions posted in various posts in HN are pretty intimidating to start. Is there a beginner friendly book or a youtube lecture that de-mystifies and simplifies learning algorithms ?
I mailed a few people whom i admire if they could provide guidance and / or mentor me, but haven't found help. IRC channels are intimidating too.
I would be grateful to the community for any help.