Ask HN: How to best acquire theoretical computer science knowledge?
121 comments
If you can get a decent prof (not necessarily the most famous or well-published) to take you on to do a M.Sc., that beats the hell out of doing another B.Sc.. (Caveat emptor: Choosing a bad prof will bring you nothing but pain! Your supervisor is not just another prof. He will have an insane impact on you!) B.Sc.'s are the Honda Civics of academia. Universities mass produce them because they bring in the most cash per head. Grad school is more like a bespoke roadster or F1 car. The resources universities spend on grad students and research are disproportionate compared to what they spend on undergrads. Also, just as you learn more than just the core knowledge of your major in undergrad, you learn a lot of additional things in grad-school. Teaching, speaking, etc.. Above all, grad students are placed under pressure to actually think instead of just regurgitating. If you do another B.Sc. you'll be repeating a lot of stuff you already know.
A good prof will figure out where your knowledge is lacking and give you the resources to catch up fast. You'll likely be thrown into some grad-level courses that are way outta your league and will have to work like a crazy bastard to catch up with students who have a CS B.Sc., but you'll have great student-teacher ratios and your best teachers will actually be the other students. You'll probably have to TA material you'll be learning the night before, and teaching something is a fantastic way to learn it well. Eventually, you should do some research, and hopefully you'll have chosen a prof doing something you're interested in. Do NOT do a course-based M.Sc., as those are just upgraded Honda Civics!
You'll be under intense pressure to learn a lot fast, but you'll have the resources to do it. A B.Sc. will likely take longer and you'll spend a lot of time slacking off because undergrad courses are pretty damned easy. Also, a M.Sc. is a better credential to have than another B.Sc..
When you're choosing a prof, talk to the other grad-students. DO NOT just choose the guy who was nominated for a nobel prize. As the saying goes, "Happy students never won anyone a Nobel!".
A good prof will figure out where your knowledge is lacking and give you the resources to catch up fast. You'll likely be thrown into some grad-level courses that are way outta your league and will have to work like a crazy bastard to catch up with students who have a CS B.Sc., but you'll have great student-teacher ratios and your best teachers will actually be the other students. You'll probably have to TA material you'll be learning the night before, and teaching something is a fantastic way to learn it well. Eventually, you should do some research, and hopefully you'll have chosen a prof doing something you're interested in. Do NOT do a course-based M.Sc., as those are just upgraded Honda Civics!
You'll be under intense pressure to learn a lot fast, but you'll have the resources to do it. A B.Sc. will likely take longer and you'll spend a lot of time slacking off because undergrad courses are pretty damned easy. Also, a M.Sc. is a better credential to have than another B.Sc..
When you're choosing a prof, talk to the other grad-students. DO NOT just choose the guy who was nominated for a nobel prize. As the saying goes, "Happy students never won anyone a Nobel!".
Grow Beyond with Google: Guide for Technical Development https://docs.google.com/file/d/0B_Y4kz709TVwaVZfdnl3blI3UWs/...
Universities tbh are more for meeting likeminded people nowadays: most of the stuff taught at CS can be learnt on your own given you're disciplined enough.
Just to underline some of the points others have made, here is a short list of things I found most valuable from my CS degree:
- Algorithms, Datastructures & Computational/space complexity (ADC). <-- This one is huge imo and will allow you to discern efficient code from inefficient ones.
- Discrete Mathematics. It's the lingua franca when talking about CS and you will need it to understand ADC for example. It introduces set theory, graph theory and so forth, which are used in ADC/AI etc... to describe and solve problems.
- Linear Algebra: if you ever want to work with 3d stuff, this one will be essential. It'll also work out nicely for 2d work, and seeing as you mentioned you've got a knack for UI work, you probably will appreciate this one. It'll allow you to get an understanding of for example affine transformations etc... You will need a computer graphics course too, but this is definitely a prerequisite course to do those.
- Probability and statistics. I really hated this course (it's also one of the harder ones coincidentally), but once you "get it", you really do "get it". Ever wondered how gesture recognition works? Or how spam filters work? Well, the math required for this will be addressed here, together with the following subject.
- Artificial Intelligence. Extremely useful if you're interested in game dev, or just want to learn to solve problems in a smart and efficient manner. It combines ADC with discrete mathematics with probability & statistics.
- Calculus. If not for the math, it'll definitely make you look at things in a different way, which imo in turn, contributes to your way of looking at problems. Partial derivation / integrals and how it can relate to 3d surfaces for example. And let's not forget taylor series etc... It'll give you an idea of how mathematical functions such as sin(1) etc... can be implemented etc...
- Operating systems, Computer Architecture/organization. Essential for understanding what happens under the hood. Allows you to reason about what the best code path is for solving a problem. Should you use the GPU for example instead of the CPU etc... and if so, why? etc... It also introduces models such as finite state automatons etc... which are also applicable in language processing for, say, compilers.
And that brings us to the last, but certainly not least subject, my all time favorite: - Compiler construction. Gives you an insight in how languages are being processed, and how to design your own language and build a compiler for it. It should also give you additional insights on how runtimes/virtual machines work.
The software engineering courses were more about processes, philosophies and patterns etc... and even though they're definitely useful, I think most experienced software engineers have already "got this". Either via books, company policy/culture or via experience.
Again, these are my experiences, YMMV :) Good luck!
Just to underline some of the points others have made, here is a short list of things I found most valuable from my CS degree:
- Algorithms, Datastructures & Computational/space complexity (ADC). <-- This one is huge imo and will allow you to discern efficient code from inefficient ones.
- Discrete Mathematics. It's the lingua franca when talking about CS and you will need it to understand ADC for example. It introduces set theory, graph theory and so forth, which are used in ADC/AI etc... to describe and solve problems.
- Linear Algebra: if you ever want to work with 3d stuff, this one will be essential. It'll also work out nicely for 2d work, and seeing as you mentioned you've got a knack for UI work, you probably will appreciate this one. It'll allow you to get an understanding of for example affine transformations etc... You will need a computer graphics course too, but this is definitely a prerequisite course to do those.
- Probability and statistics. I really hated this course (it's also one of the harder ones coincidentally), but once you "get it", you really do "get it". Ever wondered how gesture recognition works? Or how spam filters work? Well, the math required for this will be addressed here, together with the following subject.
- Artificial Intelligence. Extremely useful if you're interested in game dev, or just want to learn to solve problems in a smart and efficient manner. It combines ADC with discrete mathematics with probability & statistics.
- Calculus. If not for the math, it'll definitely make you look at things in a different way, which imo in turn, contributes to your way of looking at problems. Partial derivation / integrals and how it can relate to 3d surfaces for example. And let's not forget taylor series etc... It'll give you an idea of how mathematical functions such as sin(1) etc... can be implemented etc...
- Operating systems, Computer Architecture/organization. Essential for understanding what happens under the hood. Allows you to reason about what the best code path is for solving a problem. Should you use the GPU for example instead of the CPU etc... and if so, why? etc... It also introduces models such as finite state automatons etc... which are also applicable in language processing for, say, compilers.
And that brings us to the last, but certainly not least subject, my all time favorite: - Compiler construction. Gives you an insight in how languages are being processed, and how to design your own language and build a compiler for it. It should also give you additional insights on how runtimes/virtual machines work.
The software engineering courses were more about processes, philosophies and patterns etc... and even though they're definitely useful, I think most experienced software engineers have already "got this". Either via books, company policy/culture or via experience.
Again, these are my experiences, YMMV :) Good luck!
Coursera, edX and Udacity.
I have myself taken half a course on Udacity(did not finish) and a Startup Engineering Course on Coursera. I've started a new course on Coursera and really enjoy the material on there.
Also, I'd suggest you read a lot, the internet is a treasure trove of information. I've learnt from long blog posts and free online books. Have a go and explore, you'll reach a point where you don't have enough time to study everything you want.
Goodluck
I have myself taken half a course on Udacity(did not finish) and a Startup Engineering Course on Coursera. I've started a new course on Coursera and really enjoy the material on there.
Also, I'd suggest you read a lot, the internet is a treasure trove of information. I've learnt from long blog posts and free online books. Have a go and explore, you'll reach a point where you don't have enough time to study everything you want.
Goodluck
[deleted]
I'm going to keep this reply shallow but my advice is dig up a curriculum and course guide for a CS degree (there are many publicly discoverable with Google), establish what all of the main topics and threads are, grab the best/most recommended textbooks for a couple of topics at a time, work through the books a few pages at a time and for anything you don't understand, Google and learn about it. Take copious notes as you go to refer back to. Read papers that pop up from time to time, get a feel for what you know you don't understand and quickly get up to speed with them (even if it's just reading Wikipedia for 20 minutes).
The above is deliberate practice and learning but also keep an eye out for CS-level posts linked on HN and http://www.reddit.com/r/compsci and relate them to what you're learning. Read a post about AI data structures in Prolog? Don't understand Prolog that sounds interesting? Get a feel for what Prolog is. Write a simple program. Get a feel for implementing the data structure in a language you do understand. Then try and bring both Prolog and the structure together. Rinse and repeat with anything that interests you. Again, be sure to TAKE NOTES or you'll forget things using this scattershot approach.
It'll take a TON of time but if you keep your sessions relatively short, you'll get a shallow knowledge of all of the major CS areas quickly, and then you can use your job or natural interest to drive you into going deep on areas that affect or motivate you.
I'm starting an MSc in software engineering later this year but my own CS knowledge has mostly been learned in the above ways (that is, I don't have a bachelors in CS).
Clarification: As cliveowen got me to realize (below), the above approach might not be for you if you have a very specific target in your career development. My experiences and recommendations are specifically around getting a broad level of experience rather than aiming at a specific role (which, perhaps, you should be doing).
The above is deliberate practice and learning but also keep an eye out for CS-level posts linked on HN and http://www.reddit.com/r/compsci and relate them to what you're learning. Read a post about AI data structures in Prolog? Don't understand Prolog that sounds interesting? Get a feel for what Prolog is. Write a simple program. Get a feel for implementing the data structure in a language you do understand. Then try and bring both Prolog and the structure together. Rinse and repeat with anything that interests you. Again, be sure to TAKE NOTES or you'll forget things using this scattershot approach.
It'll take a TON of time but if you keep your sessions relatively short, you'll get a shallow knowledge of all of the major CS areas quickly, and then you can use your job or natural interest to drive you into going deep on areas that affect or motivate you.
I'm starting an MSc in software engineering later this year but my own CS knowledge has mostly been learned in the above ways (that is, I don't have a bachelors in CS).
Clarification: As cliveowen got me to realize (below), the above approach might not be for you if you have a very specific target in your career development. My experiences and recommendations are specifically around getting a broad level of experience rather than aiming at a specific role (which, perhaps, you should be doing).
Frankly I think that's the worst advice you could give anyone for whatever subject: i.e. "when you find something you don't know about, study it". The best course would be to pursue a targeted learning, and by that I mean, do you wanna be a data scientist? Then you should study probability theory and statistics along with the relative algorithms instead of reading something about Prolog. Do you wanna become a software engineer? Then you should study software engineering, programming, algorithms, advanced algorithms and data structures, patterns, and so on. Getting a "shallow knowledge of all of the major CS areas quickly" with a "scattershot approach" is a recipe for disaster. The ultimate goal is to gain skills needed for your dream job.
I concede it's not the best approach if someone has a specific goal like, as you suggest, becoming a "data scientist", a "compiler developer", or a quant, etc.
In my case (and what I felt was the case for the OP) a general, broad level of knowledge is paramount. I then study deeper on an as-needed basis off the back of my broad but shallow knowledge. Being able to tie together the basics of many unrelated areas really helps in my work, but is surely not for everyone. I already have my dream job for life so my goal is more on becoming well rounded rather than getting a specific job.
From a career POV, should the OP specifically focus on a specific target rather than just aim to get a general CS understanding? Quite possibly! But that's a different discussion/recommendation that I think you could kick off in your own response to OP. So I'm voting you up not because I agree with your criticism, but because perhaps the OP does need to think more solidly about the end goal.
In my case (and what I felt was the case for the OP) a general, broad level of knowledge is paramount. I then study deeper on an as-needed basis off the back of my broad but shallow knowledge. Being able to tie together the basics of many unrelated areas really helps in my work, but is surely not for everyone. I already have my dream job for life so my goal is more on becoming well rounded rather than getting a specific job.
From a career POV, should the OP specifically focus on a specific target rather than just aim to get a general CS understanding? Quite possibly! But that's a different discussion/recommendation that I think you could kick off in your own response to OP. So I'm voting you up not because I agree with your criticism, but because perhaps the OP does need to think more solidly about the end goal.
The goal is simply acquire a much deeper understanding of computing that I feel I missed out on by not studying CS in college. I see myself as extremely employable doing app and web development for the immediate future. Under the hood everything is like magic to me and I don't want to feel like that. I also feel like I don't know enough to know what my dream job in computing is yet. Your thoughts are limited by your knowledge.
You might like to try the old "go down the stack" process as a first exercise then. There was a post about it recently on here. A Web app developer who knows a lot about things work right down to the metal is a lot more valuable than one who doesn't.
How do the processes behind your webapps work? Why or why can't your app support 10,000 clients hitting it at once? What's happening at the HTTP level? The TCP/IP level? Just understanding some of the algorithms behind routing and TCP congestion avoidance could give a ton of food for thought, still get you into learning algorithms, notation, and reading CS papers, but still be more directly beneficial to the day job. I wish you luck however you end up approaching it! :-) The only bit I want to repeat is take a lot of notes.. do blog posts, write articles.. anything to ensure you retain that knowledge because it's so easily lost as you get older.
How do the processes behind your webapps work? Why or why can't your app support 10,000 clients hitting it at once? What's happening at the HTTP level? The TCP/IP level? Just understanding some of the algorithms behind routing and TCP congestion avoidance could give a ton of food for thought, still get you into learning algorithms, notation, and reading CS papers, but still be more directly beneficial to the day job. I wish you luck however you end up approaching it! :-) The only bit I want to repeat is take a lot of notes.. do blog posts, write articles.. anything to ensure you retain that knowledge because it's so easily lost as you get older.
Taking notes has always been my weakest point as a student so that is something I will definitely have to focus on. I think it's sound advice though.
I'm hearing a lot of companies talking about the desire for "T-Shaped" Developers. I believe this means they do have deep knowledge in at least one area, but also have more general knowledge in a range of topics besides their specialty.
[deleted]
What's the MSc in software engineering course you're going to do? What's the curriculum for it?
Read. Focus on foundational texts. I'm talking about texts like K&R, SICP, Lisp In Small Pieces, RFC 791. I'll add Gödel, Escher, Bach, because I genuinely believe it will make anyone a better computer programmer. There are far more than I could mention in a paragraph; L.I.S.P is not as well known as the others I mention, and got added mostly because I'm reading it at present. It's really good.
Also, read actual code. Want to know how diff works? Read diff. Getting better at code forensics is the second-best way for a self-taught programmer to learn real CS.
EDIT: I am a largely self-taught programmer.
Also, read actual code. Want to know how diff works? Read diff. Getting better at code forensics is the second-best way for a self-taught programmer to learn real CS.
EDIT: I am a largely self-taught programmer.
At the end of the day I think its a question of what you value more "money" or "time". If money is your main concern, use books and online courses. If time is your main concern, get an MS at a good CS program.
My problem with teaching myself is that it's hard for me to stay motivated after the first couple of weeks. Now this just me -- it's not you. So maybe you don't have this problem at all, but I've found school to be a good motivator.
Plus having a good professor can help get through the tough spots. I studied complexity under Papadimitriou and he was great at explaining things I struggled with.
One problem with school though is you'll have less flexibility to skip things you don't care about, and investigate those things you do. That can be a blessing or a curse.
My problem with teaching myself is that it's hard for me to stay motivated after the first couple of weeks. Now this just me -- it's not you. So maybe you don't have this problem at all, but I've found school to be a good motivator.
Plus having a good professor can help get through the tough spots. I studied complexity under Papadimitriou and he was great at explaining things I struggled with.
One problem with school though is you'll have less flexibility to skip things you don't care about, and investigate those things you do. That can be a blessing or a curse.
I think it's far and away self-study or if you can find a good online course among the free solutions you can start there. The Art of Computer Programming by Knuth has a lot of the theory included and doesn't hold back on the Math. Similarly, Introduction to Algorithms by Cormen, Leiserson, Rivest and Stein is quite good for the basis of Algorithm theory.
That being said, theory plays a small role in everyday development. If your primary goal is to increase your knowledge as it relates to development you don't want to focus on theory until it becomes a pain point. Algorithm theory helps with scalability, but practice is far more useful to programming. Conferences in your field that can expose you to new things to learn and different tool chains in your language are valuable. Learning a new related language can also be valuable you might consider learning iOS and then JavaScript which would let you work on Android, iOS and Cordova (formerly PhoneGap) applications at a level where you'd be comfortable going from JavaScript to native to write unavailable features.
If you're happy and have good opportunities on Android, focus on getting a deeper understanding: what are good open source repos, what coding conventions do they use, what frameworks and approaches could you know better. Are there parts of the OS you haven't worked with and wanted to? For instance, I built a demo app for LG designed for in-store promo devices where we hacked around in the OS to intercept the display of screens that would let the user delete apps from the phone and throw up a password lock that would kick them out to the main screen. Try and get a sense of where your best (however you measure it) opportunities are and focus your learning in that direction.
That being said, theory plays a small role in everyday development. If your primary goal is to increase your knowledge as it relates to development you don't want to focus on theory until it becomes a pain point. Algorithm theory helps with scalability, but practice is far more useful to programming. Conferences in your field that can expose you to new things to learn and different tool chains in your language are valuable. Learning a new related language can also be valuable you might consider learning iOS and then JavaScript which would let you work on Android, iOS and Cordova (formerly PhoneGap) applications at a level where you'd be comfortable going from JavaScript to native to write unavailable features.
If you're happy and have good opportunities on Android, focus on getting a deeper understanding: what are good open source repos, what coding conventions do they use, what frameworks and approaches could you know better. Are there parts of the OS you haven't worked with and wanted to? For instance, I built a demo app for LG designed for in-store promo devices where we hacked around in the OS to intercept the display of screens that would let the user delete apps from the phone and throw up a password lock that would kick them out to the main screen. Try and get a sense of where your best (however you measure it) opportunities are and focus your learning in that direction.
One way of improving your knowledge is something you have already done.
Get another job that will force you to expand your knowledge. I am sure your job experience with your first start up is serving you better than a CS degree.
Similarly, a job which involves doing low level computer programming would force you to learn a lot of very practice CS, and you would be getting paid for it instead of the other way around.
Get another job that will force you to expand your knowledge. I am sure your job experience with your first start up is serving you better than a CS degree.
Similarly, a job which involves doing low level computer programming would force you to learn a lot of very practice CS, and you would be getting paid for it instead of the other way around.
I have definitely been considering this approach. The issue with this is I don't think I am currently that desirable for anything that isn't Java.
Sorry if this is obvious, but: read books. A good place to start is here: http://cstheory.stackexchange.com/questions/3253/what-books-...
There was another reading list that I remember seeing, I think it was from Stanford's TCS website, but I no longer can find it.
Self-teaching is fun because you get to choose your own curriculum, but it's often frustrating too because if you get stuck, there is no professor or TA to unstick you. This issue can somewhat mitigated via the internet.
There was another reading list that I remember seeing, I think it was from Stanford's TCS website, but I no longer can find it.
Self-teaching is fun because you get to choose your own curriculum, but it's often frustrating too because if you get stuck, there is no professor or TA to unstick you. This issue can somewhat mitigated via the internet.
Is it theoretical CS as in "CS topics that are more theoretical" (NLP? AI? Search engines? Software engineering?), or as in "Theoretical CS" (the mathematical subdisciplines: combinatorics, type theory, lambda calculus, etc.)?
In any case you first need to investigate what is interesting to you, free resources are far from rare and you can probably build yourself a decent curriculum from MIT OCW, Coursera, Udacity and similar websites which you can then study at your own pace with videos and books. If finding a job is not your concern I wouldn't advise putting yourself into debt. See http://www.saylor.org/majors/computer-science/ for an example of curriculum.
In any case you first need to investigate what is interesting to you, free resources are far from rare and you can probably build yourself a decent curriculum from MIT OCW, Coursera, Udacity and similar websites which you can then study at your own pace with videos and books. If finding a job is not your concern I wouldn't advise putting yourself into debt. See http://www.saylor.org/majors/computer-science/ for an example of curriculum.
If you're OK learning on your own, you might try buying a book before committing to the cost of tuition, also. If you really mean "theoretical CS" I don't think you can go wrong with Sipser. The latest edition[1] is expensive, but a used copy of the second edition [2] isn't TOO bad, considering the cost of another few years of education.
[1] http://www.amazon.com/Introduction-Theory-Computation-Michae...
[2] http://www.amazon.com/Introduction-Theory-Computation-Michae...
[1] http://www.amazon.com/Introduction-Theory-Computation-Michae...
[2] http://www.amazon.com/Introduction-Theory-Computation-Michae...
I second Sipser's book. Probably the best textbook I used in university. You can find international editions for a very low price.
International Editions:
http://used.addall.com/SuperRare/submitRare.cgi?author=sipse...
http://used.addall.com/SuperRare/submitRare.cgi?author=sipse...
So, what is the best way for a proven and largely self taught developer to take their knowledge to the next level?
It depends on how you learn. Some people will recommend Coursera et al., which are great. I tried some Coursera courses, but found that I found the pace too slow and became bored quickly.
For me, reading always works the best, since I can adjust it to my pace. You could look up a curriculum (as someone else suggests) and compose a reading list from that. Also, there are some works where you really can't go wrong. Some examples:
- Structure and Interpretation of Computer Programs, Abelson and Sussman
- Algorithms, Sedgewick, Wayne
- Introduction to Automata Theory, Languages, and Computation, Hopcroft, Motwani, Ullman
Lesser known, but incredibly fun books:
- Purely Functional Data Structures, Chris Okasaki
- The Reasoned Schemer, Friedman, Byrd, Kiselyov (or a good Prolog book).
After learning the foundations, you could branch out to a subfield that interests you.
(Or as Frank Zappa has bluntly put it: “If you want to get laid, go to college. If you want an education, go to the library.”)
It depends on how you learn. Some people will recommend Coursera et al., which are great. I tried some Coursera courses, but found that I found the pace too slow and became bored quickly.
For me, reading always works the best, since I can adjust it to my pace. You could look up a curriculum (as someone else suggests) and compose a reading list from that. Also, there are some works where you really can't go wrong. Some examples:
- Structure and Interpretation of Computer Programs, Abelson and Sussman
- Algorithms, Sedgewick, Wayne
- Introduction to Automata Theory, Languages, and Computation, Hopcroft, Motwani, Ullman
Lesser known, but incredibly fun books:
- Purely Functional Data Structures, Chris Okasaki
- The Reasoned Schemer, Friedman, Byrd, Kiselyov (or a good Prolog book).
After learning the foundations, you could branch out to a subfield that interests you.
(Or as Frank Zappa has bluntly put it: “If you want to get laid, go to college. If you want an education, go to the library.”)
>It depends on how you learn. Some people will recommend Coursera et al., which are great. I tried some Coursera courses, but found that I found the pace too slow and became bored quickly.
You can watch the lectures at 2x speed. ;)
You can watch the lectures at 2x speed. ;)
Yeah, but you can't jump to the next lesson - unless you wait until the end of the course and see them all at the time, but then you won't get a certificate because you miss the quiz deadlines.
One thing I've noticed with Coursera is that if you join before the course starts, there's often a preview button that seems to show at least the entire course's video lectures. If the content was useful enough to you, you could take the time to download all the videos and could watch them at any pace, and since you'd always be ahead because of your quicker pace you could still do the quizzes, albeit not straight after watching the lectures.
You can download the video, and then you can extract the audio and listen to the mp3 wherever you are, as often as you want. At least, that's what I did.
I don't think the OP is in it for the certificates.
I don't think the OP is in it for the certificates.
You can do several courses at the same time if you have the spare time.
The SICP videos are available somewhere and are pretty intense.
SICP Videos - http://www.youtube.com/playlist?list=PLB63C06FAF154F047
I think the first video in this series is probably the best intro to CS lecture ever.
Any reason to go for Sedgewick and Wayne over CLRS? I haven't read the first one so just curious.
I've read both and they are VERY different algorithms books. Some differences:
* Sedgewick has no proofs in his book (or at least very few). CLRS has a lot of proofs in it.
* CLRS uses pseudocode, while Sedgewick uses actual code. I think they use Java now (it was Pascal in the edition that I had used).
* Sedgewick is really a practitioners introduction. It views the algorithmic problem as the jumping off point. Whereas CLRS is more an intro for people who are going to study computer science -- so it focuses on the methods.
Both good texts. I think for most programmers I'd recommend Sedgewick, but if your goal is to be a computer scientist, then I'd recommend CLRS.
* Sedgewick has no proofs in his book (or at least very few). CLRS has a lot of proofs in it.
* CLRS uses pseudocode, while Sedgewick uses actual code. I think they use Java now (it was Pascal in the edition that I had used).
* Sedgewick is really a practitioners introduction. It views the algorithmic problem as the jumping off point. Whereas CLRS is more an intro for people who are going to study computer science -- so it focuses on the methods.
Both good texts. I think for most programmers I'd recommend Sedgewick, but if your goal is to be a computer scientist, then I'd recommend CLRS.
Thanks for the list. I was definitely the type of student that just read the book and didn't go lectures too often. I also love that quote.
"I am not really worried about getting a job. This is about gaining knowledge."
Even so you want to be smart about it, and gain the knowledge in such a way such that employers will recognize your expertise. It is for this reason that I would rather recommend the MS. It will at least get you an interview next time you want to change jobs.
Even so you want to be smart about it, and gain the knowledge in such a way such that employers will recognize your expertise. It is for this reason that I would rather recommend the MS. It will at least get you an interview next time you want to change jobs.
CS or SE? Not clear from you statement.
I studied finance in college. A few years later I asked a similar question, ended up enrolling as a post-bac student, and taught myself web development.
First, you need to figure out exactly what you want to learn. By theoretical, what do you mean? If your goal is to be on the same level as other programmers who are CS grads, then you might consider digging into algorithms and data structures and discrete math. This is pretty much the base of anything theoretical you'll do in computer science.
If you're in it to learn, go the udacity route. If, after completing all of their courses, you still feel you have some gaps, then you can go back to school if need be. If, on the other hand, your goal is to go into research, then you'll just have to bone up and go back to grad school to be taken seriously. In research, credentials matter.
First, you need to figure out exactly what you want to learn. By theoretical, what do you mean? If your goal is to be on the same level as other programmers who are CS grads, then you might consider digging into algorithms and data structures and discrete math. This is pretty much the base of anything theoretical you'll do in computer science.
If you're in it to learn, go the udacity route. If, after completing all of their courses, you still feel you have some gaps, then you can go back to school if need be. If, on the other hand, your goal is to go into research, then you'll just have to bone up and go back to grad school to be taken seriously. In research, credentials matter.
It sounds like you're not quite sure what specifically you are interested in. CS is a very broad field and you could probably study it for a lifetime without scratching the surface.
Maybe better to choose a particular hard problem that is interesting and focus on that. Something that would be neat to do but isn't as simple as just calling a few library functions.
Maybe better to choose a particular hard problem that is interesting and focus on that. Something that would be neat to do but isn't as simple as just calling a few library functions.
I feel like I need to get the foundation that most people get in a BS before i can answer the question specifically. At this point I only know that I have a strong desire to know how computers work on a deep level.
Uff, there's so much to go through there. It's turtles all the way down.
Start with a course on computer architecture / digital design. Something that'll have you put together a simple 8 bit CPU. Move your way up with a course on operating systems - processes, swapping, cache invalidation, file systems, drivers.
Then, maybe something on compilers? Or how to write your own interpreter.
Finally, a good networking course is pretty important.
Very little of this is theoretical compsci, but it's something that will get covered in a degree program. That said, knowing how to recognize what kind of class of algorithm we're looking at, and data structures all around.
Start with a course on computer architecture / digital design. Something that'll have you put together a simple 8 bit CPU. Move your way up with a course on operating systems - processes, swapping, cache invalidation, file systems, drivers.
Then, maybe something on compilers? Or how to write your own interpreter.
Finally, a good networking course is pretty important.
Very little of this is theoretical compsci, but it's something that will get covered in a degree program. That said, knowing how to recognize what kind of class of algorithm we're looking at, and data structures all around.
There isn't really a standard foundation that everyone with a BS in CS has. Curriculum are different and different colleges will emphasize different things, not to mention that many courses are elective.
It also depends upon what you consider to be a deep level?
Are you interested in electronics and signal processing, mathematics or the structure/design of programming languages? Are you interested in individual computers or distributed systems? Are you just interested in computers or are you interested in using computers to model other things, like economics, physics or virtual worlds?
At a minimum you could brush up on algorithm / data structure understanding and some of the math behind that. Problem is that as soon you gain understanding of something it will simply expose you to deeper subjects that you don't fully understand. You will never reach the bottom of all of these rabbit holes.
It also depends upon what you consider to be a deep level?
Are you interested in electronics and signal processing, mathematics or the structure/design of programming languages? Are you interested in individual computers or distributed systems? Are you just interested in computers or are you interested in using computers to model other things, like economics, physics or virtual worlds?
At a minimum you could brush up on algorithm / data structure understanding and some of the math behind that. Problem is that as soon you gain understanding of something it will simply expose you to deeper subjects that you don't fully understand. You will never reach the bottom of all of these rabbit holes.
I have no illusions that I could possibly learn it all. I just want to learn enough so that I can go deeper on something more specific, such as virtual worlds, and no how to proceed.
Having a general CS education does not necessarily make you able to explore arbitrary fields. CS programs are ultimately designed by course administrators and professors and can be slow to change, so there's no guarantee that they will prepare you sufficiently to explore whatever avenues you wish to explore later.
Ultimately it depends on where you see your shortcomings that are preventing you from doing what you want. Do you wish you had better credentials so that you could get a job with better pay or more interesting work? Or do you find yourself frustrated because you are not able to sufficiently grok a certain subject area.
Ultimately it depends on where you see your shortcomings that are preventing you from doing what you want. Do you wish you had better credentials so that you could get a job with better pay or more interesting work? Or do you find yourself frustrated because you are not able to sufficiently grok a certain subject area.
Theoretical computer science is probably not what you mean. Theoretical computer science are things like complexity theory (up to arithmetical hierarchy and randomized and approximation algorithms), theory of recursion (up to e.g. Gödel's theorems), (various kinds of) logic, algorithms and datastructures (full of hardcore probability and statistics), etc.
Subjects like that give you a firm grounding in the area of computer science but you could study them for ages especially on your own. These subjects are also not very practical in terms of everyday engineering and on top of that before you even get to that level, a top school would require you to study a bunch of mathematics to gain a certain level of rigor of your thinking. My guess is that this is hard to do on your own.
On the other hand, since you have a related degree already and since you know programming, I would heartily recommend you to study on your own combining two approaches: bottom-up to learn the basics and top-down to immediately start increasing your market value and to start opening doors and to keep yourself motivated to go forward with the more tedious study of the basics.
E.g. say you are interested in distributed systems - you can start reading up on them (top-down) while digressing here and there to learn some of the basics.
Make use of online resources like Coursera for the necessary basics (bottom-up): at least some maths heavy on proofs [1], at least some complexity theory, at least some graph theory, ....
I think a purely bottom-up approach (going to a university) could be a waste of time and money in your case. But you need to emulate it a bit because a purely top-down approach could be too superficial and wouldn't teach you some of the more rigorous thinking you might need.
Whatever you do and learn try to do it in-depth. Superficial knowledge of many things will not benefit you much in the long term while in-depth study of many computer science fields will likely lead you through a series of small enlightenments.
[1] My professor of linear algebra used to insist that we as computer scientists have to know all the proofs of the theorems we were learning - in contrast to mathematicians learning the same subject - simply because we have to know how things are done to the last detail and we have to develop that kind of thinking.
Subjects like that give you a firm grounding in the area of computer science but you could study them for ages especially on your own. These subjects are also not very practical in terms of everyday engineering and on top of that before you even get to that level, a top school would require you to study a bunch of mathematics to gain a certain level of rigor of your thinking. My guess is that this is hard to do on your own.
On the other hand, since you have a related degree already and since you know programming, I would heartily recommend you to study on your own combining two approaches: bottom-up to learn the basics and top-down to immediately start increasing your market value and to start opening doors and to keep yourself motivated to go forward with the more tedious study of the basics.
E.g. say you are interested in distributed systems - you can start reading up on them (top-down) while digressing here and there to learn some of the basics.
Make use of online resources like Coursera for the necessary basics (bottom-up): at least some maths heavy on proofs [1], at least some complexity theory, at least some graph theory, ....
I think a purely bottom-up approach (going to a university) could be a waste of time and money in your case. But you need to emulate it a bit because a purely top-down approach could be too superficial and wouldn't teach you some of the more rigorous thinking you might need.
Whatever you do and learn try to do it in-depth. Superficial knowledge of many things will not benefit you much in the long term while in-depth study of many computer science fields will likely lead you through a series of small enlightenments.
[1] My professor of linear algebra used to insist that we as computer scientists have to know all the proofs of the theorems we were learning - in contrast to mathematicians learning the same subject - simply because we have to know how things are done to the last detail and we have to develop that kind of thinking.
IamA graduate student in theoretical computer science.
I don't know that I agree with the poster above with respect to:
"a top school would require you to study a bunch of mathematics to gain a certain level of rigor of your thinking"
Most good schools require only two courses in "pure theory": first an introduction to discrete mathematics, where you learn about logic and how to prove stuff, a smattering of number theory/crypto, graph theory, probability, and automata theory; second an algorithms course that teaches you to analyse and construct algorithms according to certain design methods. Together this is really only a year of mathematics, and both courses are quite fun.
Note that discrete mathematics as taught by a good CS program is NOTHING like the math you learned in high school or elementary school, it teaches you to think creatively about mathematical structures and then justify that creativity with logic.
If you're already a developer, you'll probably have a bunch of latent structures hanging around in your brain that will let you have an easy/fun time with the material in these two courses. I suggest that you study discrete mathematics and algorithms in depth, to the level of some decent undergrad course on each. Learning this material will give your thinking the sort of quantitative/logical "edge" you might want -- these two courses contain the basic mathematical tools common to much of computer science. I do agree with the post above that they are not the most practical things in terms of everyday engineering, though.
Addendum: if you find that you really enjoy the mathematical angle and you think machine learning is cool, also learn linear algebra, because a bunch of machine learning theory and implementation relies on linear algebraic algorithms and concepts. Linear algebra actually would be practical to know if you ever do anything with data analysis.
I don't know that I agree with the poster above with respect to:
"a top school would require you to study a bunch of mathematics to gain a certain level of rigor of your thinking"
Most good schools require only two courses in "pure theory": first an introduction to discrete mathematics, where you learn about logic and how to prove stuff, a smattering of number theory/crypto, graph theory, probability, and automata theory; second an algorithms course that teaches you to analyse and construct algorithms according to certain design methods. Together this is really only a year of mathematics, and both courses are quite fun.
Note that discrete mathematics as taught by a good CS program is NOTHING like the math you learned in high school or elementary school, it teaches you to think creatively about mathematical structures and then justify that creativity with logic.
If you're already a developer, you'll probably have a bunch of latent structures hanging around in your brain that will let you have an easy/fun time with the material in these two courses. I suggest that you study discrete mathematics and algorithms in depth, to the level of some decent undergrad course on each. Learning this material will give your thinking the sort of quantitative/logical "edge" you might want -- these two courses contain the basic mathematical tools common to much of computer science. I do agree with the post above that they are not the most practical things in terms of everyday engineering, though.
Addendum: if you find that you really enjoy the mathematical angle and you think machine learning is cool, also learn linear algebra, because a bunch of machine learning theory and implementation relies on linear algebraic algorithms and concepts. Linear algebra actually would be practical to know if you ever do anything with data analysis.
I am talking mostly from my own experience (from two or three European universities). A lot of my undergraduate CS programme was very mathematics heavy with a lot of focus on proofs (a lot of calculus, linear algebra, discrete mathematics, graph theory, algebra, even set theory for some, first order and predicate logic) and then of course a lot of foundational CS subjects like automata theory, basic datastructures, crypto, .... I would argue that proper rigorous proofs and thinking are maybe easier taught on the example of calculus or set theory rather than graph theory (I am not talking about discrete mathematics such as combinatorics without much of graphs, etc.). In graph theory, you often resort to doing proofs "by drawing" assuming that a skilled reader would be able to fill in the dots in a formal manner. That said, it's just my opinion and your course may have ways around it.
I think your off base with this. A good school with a good theoretical cs program will always require up to multivariable calc, linear algebra, stats with calc, and discrete math. These are required because they will almost always be essential for doing theoretical work. Sometimes you just need it to properly express and explain complexity of a problem but many times in theoretical work knowledge of advanced mathematics is integral to to the actual research. Your advice is more geared towards what most good schools consider a general cs degree.
My advice is indeed geared towards general CS, it was unclear to me if the original poster wanted to get into heavy-duty theory or just gain a more rigorous understanding of CS in general.
Even at good schools (in the US, anyway) the calc up to multivar that is required isn't done with what I would call good proofs. They'll generally do epsilon-delta definitions of the limit and a few other fun things, but will fall far short of a truly rigorous development of the calculus. Same goes for many introductions to linear algebra and stats with calc courses. Some math departments use linear algebra to introduce proofs and creative mathematical thinking, though, so this isn't always the case.
So if someone is looking to develop how they think in mathematics and they already have a good programming background, they are much better served by taking a discrete math course than any of those sorts of math courses. If they then want to learn calculus, I would recommend self-study from Spivak's "Calculus" book which is a fun and elementary but rigorous treatment of the subject. You get to see not just how calculus works, but why it works, which is sorely lacking in most undergrad calc courses, even up to multivar.
Even at good schools (in the US, anyway) the calc up to multivar that is required isn't done with what I would call good proofs. They'll generally do epsilon-delta definitions of the limit and a few other fun things, but will fall far short of a truly rigorous development of the calculus. Same goes for many introductions to linear algebra and stats with calc courses. Some math departments use linear algebra to introduce proofs and creative mathematical thinking, though, so this isn't always the case.
So if someone is looking to develop how they think in mathematics and they already have a good programming background, they are much better served by taking a discrete math course than any of those sorts of math courses. If they then want to learn calculus, I would recommend self-study from Spivak's "Calculus" book which is a fun and elementary but rigorous treatment of the subject. You get to see not just how calculus works, but why it works, which is sorely lacking in most undergrad calc courses, even up to multivar.
What do you call good proofs? I think what you consider a "good school" is not what I consider a good school. In my calc classes we studied tons of awesome proofs that were very applicable to theoretical cs stuff. Plus most of my linear algebra classes and stats classes were just engineering, cs, and math majors so we once again focused on very useful real world applications and fields of active research.
If someone wants to develop their mathematical thinking and is good at programming descrete math isn't going to help them much because they have probably already seen much of it. They need to be exposed to high level calc and linear algebra to develop their mathematical thinking.
If someone wants to develop their mathematical thinking and is good at programming descrete math isn't going to help them much because they have probably already seen much of it. They need to be exposed to high level calc and linear algebra to develop their mathematical thinking.
Thanks for this. I think this is very practical advice.
The other responses are great. But one meta point I think is worth making. Don't overestimate how much competency you gain from just getting a CS degree. Lots of people coasted their way through undergrad without internalizing a heck of a lot. The fact that you are motivated to go out and get knowledge puts you, IMO, at a great advantage.
My background: I did a BS in CS/Economics but spent most of my time playing online poker and skipping class. I managed to get a job I love on mostly potential, and I've had a great time re-learning all the stuff I was superficially exposed to but didn't work that hard at.
The most important thing is caring about what you do and taking ownership over your own development.
My background: I did a BS in CS/Economics but spent most of my time playing online poker and skipping class. I managed to get a job I love on mostly potential, and I've had a great time re-learning all the stuff I was superficially exposed to but didn't work that hard at.
The most important thing is caring about what you do and taking ownership over your own development.
I agree.
Education will not teach you anything, but it will give a great opportunity to learn. Many will simply learn enough to get through exams and then promtly forget everything again.
You will also get introduced to concepts and areas you would never even think of exploring otherwise. I thought graph theory sounded boring, but now I enjoy it. Formal languages and automata theory? Physics? Electronics? Hardware design? These are topics I would never have explored if it wasn't for going to university.
Artificial intelligence, compilers and operating systems are topics I probably would have explored, but I sincerely doubt I would go as deep as I have.
Also do not underestimate the effect deadlines can have. Procrastination is a constant evil in my life somewhat negated by school forcing me to do things.
Education will not teach you anything, but it will give a great opportunity to learn. Many will simply learn enough to get through exams and then promtly forget everything again.
You will also get introduced to concepts and areas you would never even think of exploring otherwise. I thought graph theory sounded boring, but now I enjoy it. Formal languages and automata theory? Physics? Electronics? Hardware design? These are topics I would never have explored if it wasn't for going to university.
Artificial intelligence, compilers and operating systems are topics I probably would have explored, but I sincerely doubt I would go as deep as I have.
Also do not underestimate the effect deadlines can have. Procrastination is a constant evil in my life somewhat negated by school forcing me to do things.
This is why I am considering the BS. I already had the college experience at Penn State. Meaning parties, friends, girls, football, etc. This time it would be purely about going to class and learning. I bet I wouldn't even have to do Gen Eds since I already have a BS.
I think the marginal benefit of a second BS-level degree is small given the cost in time and money.
If you're going back to school then you should definitely consider an MS. Maybe you're worried that you wouldn't be accepted or that it would be too challenging? In that case, narrow down a list of universities/departments where you would like to enroll, and identify what you need to study in order to get accepted.
Alternatively, teach yourself, and try to join a top tech company that focus on making the kind of software you're interested in. Working with great programmers has a large impact on your learning. Perhaps you're underestimating this impact because your first job wasn't challenging enough.
If you're going back to school then you should definitely consider an MS. Maybe you're worried that you wouldn't be accepted or that it would be too challenging? In that case, narrow down a list of universities/departments where you would like to enroll, and identify what you need to study in order to get accepted.
Alternatively, teach yourself, and try to join a top tech company that focus on making the kind of software you're interested in. Working with great programmers has a large impact on your learning. Perhaps you're underestimating this impact because your first job wasn't challenging enough.
This is very true, having been exposed to something 10 years ago to a sufficient level to score 50% on an exam paper is very different from having an applicable knowledge.
I am suggesting an approach rather than a source. Sources, as someone already said it, are far from rare.
I would say pick up a single system, say database (relational or document) or routing, OS, DNS, Web server, A compiler, Filesystem, NFS, etc. and study the hell out of it.
Once you know how to understand a particular system in depth, everything else will start falling in place.
I would say pick up a single system, say database (relational or document) or routing, OS, DNS, Web server, A compiler, Filesystem, NFS, etc. and study the hell out of it.
Once you know how to understand a particular system in depth, everything else will start falling in place.
Read papers.
Read lots of papers. Start with easy ones to learn how to best digest highly academic technical writing then keep pushing.
CS is remarkable in how much of its academic knowledge is online, available, for free. Taking advantage of that resource is beyond key.
If you know roughly what you want to learn, search out a seminal paper in that area. If it's deep CS, you'll probably find a good one from the 80s. Seek out every citation that's proximal to a sentence in that paper that confuses or excites you. Search on Google Scholar for all the recent papers that cite the one you found and begin eating up the chain that direction. When you find a deep topic you want to learn more of—set theory, logic, formal languages, discrete math, matrix analysis, automata theory, geometry, topology, &c.—don't be afraid to look for a good book on the topic. There are resources around the internet answering "What's a good introductory book to X" for all kinds of X.
If you don't know the general area you're interested in then I recommend the exact same strategy but with greater emphasis on following citations that make connections to other areas of CS. Additionally, pick a few disjoint topics (Category theory, system architecture, graph theory) and follow them all simultaneously looking for connections.
Formal languages, automata, and process algebras form a really fundamental mathematical course of study that I'd highly recommend to anyone interested in "why" CS works. Optimization, matrix analysis/linear algebra, and diffeq form a great basis for simulation and machine learning.
Generally, for any of these routes, learn to find your boundary of knowledge. Studying source material is very challenging because any given paper will tend to assume many things about your background. Most of those assumptions will be false and lead to density of the paper. You can often power through without them, but you can also take it as an indication of a new place to study.
Finally, see if you can find a study group or journal club. I'm not there, so I don't have firsthand experience, but I'm sure there are many in the bay area.
(Oh, and if you read mathematical papers/books don't skip the proofs and exercises. That'd be like reading the iOS documentation without ever programming anything.)
Read lots of papers. Start with easy ones to learn how to best digest highly academic technical writing then keep pushing.
CS is remarkable in how much of its academic knowledge is online, available, for free. Taking advantage of that resource is beyond key.
If you know roughly what you want to learn, search out a seminal paper in that area. If it's deep CS, you'll probably find a good one from the 80s. Seek out every citation that's proximal to a sentence in that paper that confuses or excites you. Search on Google Scholar for all the recent papers that cite the one you found and begin eating up the chain that direction. When you find a deep topic you want to learn more of—set theory, logic, formal languages, discrete math, matrix analysis, automata theory, geometry, topology, &c.—don't be afraid to look for a good book on the topic. There are resources around the internet answering "What's a good introductory book to X" for all kinds of X.
If you don't know the general area you're interested in then I recommend the exact same strategy but with greater emphasis on following citations that make connections to other areas of CS. Additionally, pick a few disjoint topics (Category theory, system architecture, graph theory) and follow them all simultaneously looking for connections.
Formal languages, automata, and process algebras form a really fundamental mathematical course of study that I'd highly recommend to anyone interested in "why" CS works. Optimization, matrix analysis/linear algebra, and diffeq form a great basis for simulation and machine learning.
Generally, for any of these routes, learn to find your boundary of knowledge. Studying source material is very challenging because any given paper will tend to assume many things about your background. Most of those assumptions will be false and lead to density of the paper. You can often power through without them, but you can also take it as an indication of a new place to study.
Finally, see if you can find a study group or journal club. I'm not there, so I don't have firsthand experience, but I'm sure there are many in the bay area.
(Oh, and if you read mathematical papers/books don't skip the proofs and exercises. That'd be like reading the iOS documentation without ever programming anything.)
In case you want to read papers but don't really know where to get started, here's my collection I've read over the years: http://dl.dropboxusercontent.com/u/315/articles/index.html ... feel free to browse through it and pick up anything that looks interesting. You'll probably want to scroll all the way to the bottom and work your way up the list.
Many of the classics are in there, such as the Plan9 paper and Paxos Made Practical.
If the images are distracting, here's the list without them: http://dl.dropboxusercontent.com/u/315/articles/index_no_ima...
Actually, looking over it, that list isn't very curated. Just Ctrl-F "unix" and that'll take you to some good papers.
Many of the classics are in there, such as the Plan9 paper and Paxos Made Practical.
If the images are distracting, here's the list without them: http://dl.dropboxusercontent.com/u/315/articles/index_no_ima...
Actually, looking over it, that list isn't very curated. Just Ctrl-F "unix" and that'll take you to some good papers.
Any chance of some sort of tar of the papers?
Thanks for this. I actually like that it's not curated, it's a good way to discover things I didn't know about. The fluid simulation stuff looks especially interesting.
Sure, no problem. Let me know if you have any questions or want to chat about random compsci topics.
I'll probably have some questions after looking over some more of the papers. We seem to have a healthy intersection of common interests. What's the best way to get in touch?
Cool! You can mail me at sillysaurus2@gmail
What sort of projects are you interested in?
What sort of projects are you interested in?
I agree with this, and would add that if you go this route, make one of the first papers you read "How to Read a Paper"[1] by Keshav.
[1]: http://ccr.sigcomm.org/online/files/p83-keshavA.pdf
[1]: http://ccr.sigcomm.org/online/files/p83-keshavA.pdf
I like fogus' list quite a bit:
http://blog.fogus.me/2011/09/08/10-technical-papers-every-pr...
http://blog.fogus.me/2011/09/08/10-technical-papers-every-pr...
That's a great kickoff point.
The previous suggestions are all good, but here's another tack, possibly better for self-teaching.
Tackle a big project: write an extensible text editor, design and implement a language and runtime, design and implement a new garbage collection system, or the currently quite popular RSS reader and feed provider.
Tackle a big project: write an extensible text editor, design and implement a language and runtime, design and implement a new garbage collection system, or the currently quite popular RSS reader and feed provider.
It might be worth your time to look at the course slides for "Gems of Theoretical Computer Science":
http://www.ccs.neu.edu/home/viola/classes/gems-08/
http://www.ccs.neu.edu/home/viola/classes/gems-08/
[deleted]
Same way you get good everything else, really. Train, learn, rest.
In answer to the first part of the question (how to get more theoretical CS knowledge), I studied an extremely theoretical degree and the materials are online to dig through - if I were to self-teach now, I'd start at the final year's course list (http://www.cl.cam.ac.uk/teaching/1314/part3.html) and work backwards to chart a path to the modules that sounded most interesting.
In terms of "taking knowledge to the next level", though, it could mean anything from you wish you knew what red-black trees and big-O notation were, to you'd love to be able to design your own programming language and write a compiler, to you wish you could administer a Linux box. I would actually suggest the best approach here is to find someone with the job you want and talk with them 1:1 to figure out where your knowledge could be most improved.
Also, for Coursera, etc, there are local study groups at places like Hacker Dojo you could join to get some of the benefits of college without the expense.
In terms of "taking knowledge to the next level", though, it could mean anything from you wish you knew what red-black trees and big-O notation were, to you'd love to be able to design your own programming language and write a compiler, to you wish you could administer a Linux box. I would actually suggest the best approach here is to find someone with the job you want and talk with them 1:1 to figure out where your knowledge could be most improved.
Also, for Coursera, etc, there are local study groups at places like Hacker Dojo you could join to get some of the benefits of college without the expense.
This is the core of a CS degree, each one is class, sometimes multiple classes that go into more detail. All interview questions will basically only cover these topics.
1. Algorithms & Data Structures <- The biggest one
2. Baremetal Hardware. ASM, goes into the design of a physical cpu with ALUs, etc. Sometimes you make your own basic CPU. I suggest using a course with ARM ASM, simpler than x86 ASM.
3. Operating Systems & Multithreading Theory. You usually do a bunch of C language work here. Maybe combine it with a project on an ardunio or similar.
4. Discrete Mathematics
5. Databases
6. (Optional) Compilers. Write your own compiler, do this after the ASM course.
7. (Optional) Artificial Intelligence. Learn cool things such as machine learning. AI can be very statistical at times, so I suggest adding a Statistics & Probability course to supplement. Statistics is used a lot in business, so it's useful to know for life in general.
8. (Optional) Computer Graphics, using OpenGL!
On top of that I suggest you learn a functional language like Clojure or Haskell. Or both of them. Also learn a language where you have to do memory management and deal with pointers (like C, C++, Objective-C) if you don't include it in your Operating Systems course. After that you can basically do anything in software. Any new language and platform will be just covering concepts you already understand at that point, so you'll be able to learn them extremely quickly.
1. Algorithms & Data Structures <- The biggest one
2. Baremetal Hardware. ASM, goes into the design of a physical cpu with ALUs, etc. Sometimes you make your own basic CPU. I suggest using a course with ARM ASM, simpler than x86 ASM.
3. Operating Systems & Multithreading Theory. You usually do a bunch of C language work here. Maybe combine it with a project on an ardunio or similar.
4. Discrete Mathematics
5. Databases
6. (Optional) Compilers. Write your own compiler, do this after the ASM course.
7. (Optional) Artificial Intelligence. Learn cool things such as machine learning. AI can be very statistical at times, so I suggest adding a Statistics & Probability course to supplement. Statistics is used a lot in business, so it's useful to know for life in general.
8. (Optional) Computer Graphics, using OpenGL!
On top of that I suggest you learn a functional language like Clojure or Haskell. Or both of them. Also learn a language where you have to do memory management and deal with pointers (like C, C++, Objective-C) if you don't include it in your Operating Systems course. After that you can basically do anything in software. Any new language and platform will be just covering concepts you already understand at that point, so you'll be able to learn them extremely quickly.
I would add computability and complexity theory to that list. Some computational complexity should be covered in algorithms and data structures, but theory of computation is not really (as algorithms by definition solves only computable problems). A fundamental knowledge of what is and what isn't computable is quite important in theoretical computer science IMO.
I can't remember if formal languages and automata was an elective or a requirement for me... I think it might have been required. The textbook was the one by Sipser, and I thought it was a pretty good book.
Yes I forgot about that. In my degree it was just the second 'algorithms & data structures' class.
+1 for Algorithms & Data Structures
In my experience with non-CS developers, this is one of their biggest shortcomings when trying to solve non-trivial programming issues in the real world.
However, if you have a design background and a knack for UI, then don't sell yourself short. There are lots of programmers and plenty of great artists, but talented people with both skill sets are seriously hard to find. Take a look at this article for an idea of what most developers (myself included) come up with for UI: http://www.codinghorror.com/blog/2006/11/this-is-what-happen...
If you have the ability to prevent atrocious designs like the one in that article then you are solving important and challenging problems. There's a reason user experience (UX)design is such a buzzword right now.
In my experience with non-CS developers, this is one of their biggest shortcomings when trying to solve non-trivial programming issues in the real world.
However, if you have a design background and a knack for UI, then don't sell yourself short. There are lots of programmers and plenty of great artists, but talented people with both skill sets are seriously hard to find. Take a look at this article for an idea of what most developers (myself included) come up with for UI: http://www.codinghorror.com/blog/2006/11/this-is-what-happen...
If you have the ability to prevent atrocious designs like the one in that article then you are solving important and challenging problems. There's a reason user experience (UX)design is such a buzzword right now.
A precursor to the compilers class is the Formal Automata one. This is very useful in understanding fundamental computation models (what is computation, why, how can we do it etc..). I was recently reading [1] to refresh some concepts and found it to be more engaging than some of the older books on the topic.
[1] http://www.valorebooks.com/textbooks/automata-theory-and-app...
[1] http://www.valorebooks.com/textbooks/automata-theory-and-app...
4: I've found nothing substitutes for a good teacher. Maybe try Coursera or something of the like?
On 6: You may want to write an interpreter as a first step. Google "bootstrap scheme" for a guy who implemented a scheme-like language in 22 steps. It's pretty cool, I'm following it right now.
If you're looking Look into LLVM. It's a great backend and is practical knowledge if you like what you're doing.
Adding onto languages, if you want the esoteric stuff, look at Factor, or Smalltalk (for more image based programming). Different paradigms are cool.
On 6: You may want to write an interpreter as a first step. Google "bootstrap scheme" for a guy who implemented a scheme-like language in 22 steps. It's pretty cool, I'm following it right now.
If you're looking Look into LLVM. It's a great backend and is practical knowledge if you like what you're doing.
Adding onto languages, if you want the esoteric stuff, look at Factor, or Smalltalk (for more image based programming). Different paradigms are cool.
When I went to CMU, databases was not in the core curriculum, but statistics and programming languages was. I don't think there was even an undergraduate-level databases course in the CS department.
Forgot to add that computer graphics can use linear algebra & calculus.
Could we try to add some books for each of these topics?
Don't forget about the math side, too. Discretet math and linear algebra should be the minimum. Probability and statistics is also useful. Abstract algebra has interesting applications in CS, too.
I'm a Devbootcamp alumni, now working in a full time position. Just like you, I wanted to continue my learning and get more knowledge on algorithms, data-structures, and other things DevBootcamp doesn't teach you.
I strongly recommend - Algorithms in a nutshell from O'Reilly (http://www.amazon.com/Algorithms-Nutshell-In-OReilly/dp/0596...)
and - Cracking the Coding interview (http://www.amazon.com/Cracking-Coding-Interview-Programming-...)
Those two books are perfectly concise and straight to the point to understand and learn exactly what you feel you are lacking as a self-taught programmer.
The first one will teach you what you have to know, putting everything into work context, making it really easy to understand why and how this or that algorithm is useful.
Cracking the coding interview then offers really good challenges to practice and master those algorithms. And of course, if you are interested in getting a job, will perfectly prepare you for that ;)
I strongly recommend - Algorithms in a nutshell from O'Reilly (http://www.amazon.com/Algorithms-Nutshell-In-OReilly/dp/0596...)
and - Cracking the Coding interview (http://www.amazon.com/Cracking-Coding-Interview-Programming-...)
Those two books are perfectly concise and straight to the point to understand and learn exactly what you feel you are lacking as a self-taught programmer.
The first one will teach you what you have to know, putting everything into work context, making it really easy to understand why and how this or that algorithm is useful.
Cracking the coding interview then offers really good challenges to practice and master those algorithms. And of course, if you are interested in getting a job, will perfectly prepare you for that ;)
In addition to all the great advice here, if you want a bit of a more structured approach, check out Scott Young's MIT challenge: http://www.scotthyoung.com/blog/mit-challenge/
I am a law major, who did 1 year of engineering a long time ago. I always been a math geek, but 0 in CS. In my last year of college, i created a startup with some friends. It was a ERP for small lawyers firms. That didnt work out, but turned me on into CS.
Now i work as a product manager in a bigger firm (also law related), and i'm learning how to code through Coursera and Edx. I HIGHLY recommend them.
They are the future. The classes are very good. Way better than a regular class over here (Brazil).
If you are intereste in theoretical CS, here are 2 classes that you may be interested:
Programming Languages (https://www.coursera.org/course/proglang): It focuses on functional programming. The first edition was VERY high recommended. Here you will learn about functional paradigm, and the differences in using it in several languages (some purely functional, like SML, some hybrid, like Ruby)
Automata (https://www.coursera.org/course/automata): It cant get more theoretical than this. The professor is Jeff Ullman. A legend. For free.
Machine Learning: A second version of Andrew Ng just ended. The third edition will be offered soon.
see.stanford.edu is also a very nice place to learn. They are actual stanford classes taped, and offered for free, online. I`m taking the three introdutory classes, and the machine learning classes.
Also, there are several architecture, compilers and algorithms classes too. After my first course (cs50.net), i realized that colleges (at least for CS) are redundant right now. They can be awsome. But, if you are short of money, or are already working, these online classes can fill the gap, easily.
add me on skype if you want to talk about more classes: lucasribeiropereira
See you!
PS. They can be VERY challenge. Take one, at most two at a time. 10-20 hours per week per course is a good rule of thumb
Now i work as a product manager in a bigger firm (also law related), and i'm learning how to code through Coursera and Edx. I HIGHLY recommend them.
They are the future. The classes are very good. Way better than a regular class over here (Brazil).
If you are intereste in theoretical CS, here are 2 classes that you may be interested:
Programming Languages (https://www.coursera.org/course/proglang): It focuses on functional programming. The first edition was VERY high recommended. Here you will learn about functional paradigm, and the differences in using it in several languages (some purely functional, like SML, some hybrid, like Ruby)
Automata (https://www.coursera.org/course/automata): It cant get more theoretical than this. The professor is Jeff Ullman. A legend. For free.
Machine Learning: A second version of Andrew Ng just ended. The third edition will be offered soon.
see.stanford.edu is also a very nice place to learn. They are actual stanford classes taped, and offered for free, online. I`m taking the three introdutory classes, and the machine learning classes.
Also, there are several architecture, compilers and algorithms classes too. After my first course (cs50.net), i realized that colleges (at least for CS) are redundant right now. They can be awsome. But, if you are short of money, or are already working, these online classes can fill the gap, easily.
add me on skype if you want to talk about more classes: lucasribeiropereira
See you!
PS. They can be VERY challenge. Take one, at most two at a time. 10-20 hours per week per course is a good rule of thumb
[deleted]
Personally I would recommend doing a ms if you can afford it but if you can't I would recommend that you study the following areas
Algorithms and data structures (this is a must)
Systems organizations and architecture
Systems programming
Discrete math
Math for cs books (generally proof based descrete and calculus based)
Statistics and probability
Then Start diving into areas that interest you like
Metaheuristics
Ai
Machine learning
Computer vision
Compilers
Device drivers
Graph theory
Computer Graphics (if your working in theoretical computer graphics strong maths skills are recommended)
And the list goes on. Also read papers were in areas of interest. Papers are a great way to learn high level theoretical concepts.
My story is somewhat similar.
I took a BA in Linguistics, worked for a while, then discovered computers. I spent a year on prerequisites and then did an MS in CS. I got good grades and did a thesis and it opened a lot of doors for me.
Some things that an MS CS degree will teach you that you may not have learned as a seasoned developer are: How to invent solutions to cutting edge problems. How to operate at scale using efficient algorithms and appropriate data structures. How to think about problem spaces. How to research. How to operate at the correct level of abstraction and how to get a level deeper when the abstraction leaks. The intersection between math, stats, and software. How to think like a computer.
To me it sounds like that's exactly what you want to do. One way or another you want these skills.
You may want to separate your desire for these skills from your desire to be credentialed for these skills. It is going to take a while to see how employers react to the Georgia Tech online MS. I believe the vast majority will treat it as second-class for a while. The judgment of HR departments is a lagging indicator. But the top tier employers may not care and just test you to see if you walk the walk.
You don't need a top tier school to get a good job. My MS is from a state university, and I work for Amazon.
I took a BA in Linguistics, worked for a while, then discovered computers. I spent a year on prerequisites and then did an MS in CS. I got good grades and did a thesis and it opened a lot of doors for me.
Some things that an MS CS degree will teach you that you may not have learned as a seasoned developer are: How to invent solutions to cutting edge problems. How to operate at scale using efficient algorithms and appropriate data structures. How to think about problem spaces. How to research. How to operate at the correct level of abstraction and how to get a level deeper when the abstraction leaks. The intersection between math, stats, and software. How to think like a computer.
To me it sounds like that's exactly what you want to do. One way or another you want these skills.
You may want to separate your desire for these skills from your desire to be credentialed for these skills. It is going to take a while to see how employers react to the Georgia Tech online MS. I believe the vast majority will treat it as second-class for a while. The judgment of HR departments is a lagging indicator. But the top tier employers may not care and just test you to see if you walk the walk.
You don't need a top tier school to get a good job. My MS is from a state university, and I work for Amazon.
In order to get into an MS program would I need to do a BS in CS or just do the prerequisites I am missing. Off the top of my head I know that I am missing Calc 2, Calc 3, Linear Algebra, Algorithms, and Operating Systems. Does it matter where I do the prerequisites? I imagine I could just do the maths at a community college.
I did the exact same thing. You only need to do the prerequisites. No, it doesn't matter where you do them, but it's obviously best if you do them at the same school you intend to attend, as you'll be able to get recommendations from professors in the same department, and obviously the courses will transfer fine.
EDIT: FWIW, at the University of Colorado, I had to do 2 courses above calc 1 - for me that was calc 2 and statistics, as well as 5-6 CS courses.
EDIT: FWIW, at the University of Colorado, I had to do 2 courses above calc 1 - for me that was calc 2 and statistics, as well as 5-6 CS courses.
That's a really a good point about the letters of recommendation. I think this is the route I will be going down. I had no idea how I was going to get the letters otherwise.
Do you have to apply to take the prerequisites or do you just pay them and go?
Do you have to apply to take the prerequisites or do you just pay them and go?
At most schools, you can just take courses without a competitive admission process until you intend to enroll in a degree program.
My college really just wanted algorithms and multivariable calculus. It probably varies from program to program.
Another important consideration is what emphases the MS program will offer. Mine had tracks in AI, Software, Distributed Systems, and Bioinformatics. CS is very broad and you won't have time to do everything in the course of two years. Think about your goals and your focus a little.
Another important consideration is what emphases the MS program will offer. Mine had tracks in AI, Software, Distributed Systems, and Bioinformatics. CS is very broad and you won't have time to do everything in the course of two years. Think about your goals and your focus a little.
You might be interested in my school, http://hackreactor.com. Our course is 50% CS (data structures, algorithm design and analysis) and software engineering (modularity, refactoring) and 50% web fundamentals and writing a lot of app code, and students are constantly working together on extracurricular study. This weekend, they're throwing a machine learning workshop (developed by students; alums invited), and in two weeks we're playing host to a PL design/implementation class (http://proglangmasterclass.com/). You can reach out to me at [email protected].
One of the biggest factors I think you should consider if you want to go the self learning route is if you'll have the time to while juggling a full time job as well. You might be able to take a few courses, but will you be able to dedicate your full attention to it, and really dig into the subjects?
If I were you I wouldn't add on any more debt until I've settled my current one. With a good CS job, that shouldn't take more than 2-3 years, considering you have no dependants and presumably relatively low living costs. In the meantime, maybe take part time courses where you can get class credit, and if you plan it right you might be able to complete your degree in a year or so, considering you already have your basic requirements done. Just do some research into making sure your class credits will be able to transfer. Good luck.
If I were you I wouldn't add on any more debt until I've settled my current one. With a good CS job, that shouldn't take more than 2-3 years, considering you have no dependants and presumably relatively low living costs. In the meantime, maybe take part time courses where you can get class credit, and if you plan it right you might be able to complete your degree in a year or so, considering you already have your basic requirements done. Just do some research into making sure your class credits will be able to transfer. Good luck.
Doing a BS in CS would be the best, otherwise you will need a ton of self-discipline to learn even a half of what you would learn in a degree program. If you decide to self study, and have enough "mathematical maturity", you will find everything you need on MIT OpenCourseWare, they have all the core courses available online with video lectures, solved homeworks etc. I would start with going through those two first:
Structure And Interpretation Of Computer Programs:
http://ocw.mit.edu/courses/electrical-engineering-and-comput...
Mathematics For Computer Science:
http://ocw.mit.edu/courses/electrical-engineering-and-comput...
The first one is pretty much a master class in software engineering, the other prepares you for studying algorithms, I would consider those the two main more theoretical facets of programming at a professional level. With that background you can try to work on their algorithms course:
http://ocw.mit.edu/courses/electrical-engineering-and-comput...
It would also be very valuable to eventually take all the core mathematics subjects from OCW: Single+multivariable calculus, linear algebra and probability theory - the profit is less immediate but those things do come very useful. All those six courses on the MIT OCW are top notch.
Structure And Interpretation Of Computer Programs:
http://ocw.mit.edu/courses/electrical-engineering-and-comput...
Mathematics For Computer Science:
http://ocw.mit.edu/courses/electrical-engineering-and-comput...
The first one is pretty much a master class in software engineering, the other prepares you for studying algorithms, I would consider those the two main more theoretical facets of programming at a professional level. With that background you can try to work on their algorithms course:
http://ocw.mit.edu/courses/electrical-engineering-and-comput...
It would also be very valuable to eventually take all the core mathematics subjects from OCW: Single+multivariable calculus, linear algebra and probability theory - the profit is less immediate but those things do come very useful. All those six courses on the MIT OCW are top notch.
This is how I am doing it. Currently working through CLRS, but I don't really like the lectures. Also taking Linear Algebra, which is just amazing, btw.
Others have suggested Coursera, but so far, I don't think it is prime time for those who are really serious. I've found that many of the classes are somewhat watered down and I feel as though I am reviewing concepts I already know instead of learning. Too bad there is no mention of EdX here. Their classes are much more intense, complete, and CS-heavy than anything I've seen from Coursera. The grading system is far superior as well. I wish EdX had more love, but maybe the problem is that it is too intense.
A note one SICP. There is so much in that book and it is certainly worth every moment. Just expect to grind long time with it. I wasn't sure how much I really learned from the book until I started making my own projects. I was quite astounded to see how much really did sink in and how much I just understood without any extra searching. I was also saddened to see how certain programming languages don't bend to the ideals found in that book.
Others have suggested Coursera, but so far, I don't think it is prime time for those who are really serious. I've found that many of the classes are somewhat watered down and I feel as though I am reviewing concepts I already know instead of learning. Too bad there is no mention of EdX here. Their classes are much more intense, complete, and CS-heavy than anything I've seen from Coursera. The grading system is far superior as well. I wish EdX had more love, but maybe the problem is that it is too intense.
A note one SICP. There is so much in that book and it is certainly worth every moment. Just expect to grind long time with it. I wasn't sure how much I really learned from the book until I started making my own projects. I was quite astounded to see how much really did sink in and how much I just understood without any extra searching. I was also saddened to see how certain programming languages don't bend to the ideals found in that book.
+1 to this, they distill the 'tricky bits' into examples and homework problems so you move through the common cases as efficiently as possible.
If you actually get the degree it will save you a lot of talk time in interviews and take away your primary disadvantage with the 'big guns' (who basically won't hire you to do theoretical CS type problems without one since they have a bunch of people with CS degrees already applying and so the risk that you're assertion you've done all the work is eliminated by hiring someone with a certificate from a third party saying that you have done all the work)
Without a CS degree you're path is to join doing something non-CS critical and then using networking and influence inside the company to move into the role you want. Takes longer and not as quick until you get to the 'fun stuff.'
If you actually get the degree it will save you a lot of talk time in interviews and take away your primary disadvantage with the 'big guns' (who basically won't hire you to do theoretical CS type problems without one since they have a bunch of people with CS degrees already applying and so the risk that you're assertion you've done all the work is eliminated by hiring someone with a certificate from a third party saying that you have done all the work)
Without a CS degree you're path is to join doing something non-CS critical and then using networking and influence inside the company to move into the role you want. Takes longer and not as quick until you get to the 'fun stuff.'
It is hard to beat a good CS curriculum with real teachers, peers, assignments, tests and grades. These things don't work well for everyone, and the pressure can be pretty intense at times, but damnit that pressure can really burn stuff into your neurons that would otherwise go in one ear and out the other. The overlapping of topics and repetition of important concepts are also typically lacking in a self-learning environment, and these things really helped me to pick out what was important in terms of building blocks for more complex things. Coursera et al. are great, they fill a niche, just probably not what you are really looking for.
It is hard to beat a good CS curriculum with real teachers, peers, assignments, tests and grades.
I dunno... I think it depends on the individual a bit. OK, to be fair, I did drop out before finishing my B.S., but up to where I was at, I can honestly say that I learned 90% or more of what I learned, outside of class, reading and practicing on my own. The only class I really remember the teacher's explanations helping a whole lot, was in certain parts of Discrete Math.
I could be wrong, but I feel like some people are just "wired' for teaching themselves stuff, learning from books and videos and websites, and then throwing the occasional question up on the Theoretical Computer Science Stack Exchange[1], or /r/askcomputerscience or whatever.
At the very least, I would posit that most people can learn an awful lot through self-study. Maybe not enough to do ground breaking research in C.S., but certainly enough to bump up their knowledge of the "basics" to where they aren't clueless when somebody else in the room starts talking about algorithmic complexity or finite state machines or whatever.
[1]: cstheory.stackexchange.com
[2]: http://www.reddit.com/r/askcomputerscience
I dunno... I think it depends on the individual a bit. OK, to be fair, I did drop out before finishing my B.S., but up to where I was at, I can honestly say that I learned 90% or more of what I learned, outside of class, reading and practicing on my own. The only class I really remember the teacher's explanations helping a whole lot, was in certain parts of Discrete Math.
I could be wrong, but I feel like some people are just "wired' for teaching themselves stuff, learning from books and videos and websites, and then throwing the occasional question up on the Theoretical Computer Science Stack Exchange[1], or /r/askcomputerscience or whatever.
At the very least, I would posit that most people can learn an awful lot through self-study. Maybe not enough to do ground breaking research in C.S., but certainly enough to bump up their knowledge of the "basics" to where they aren't clueless when somebody else in the room starts talking about algorithmic complexity or finite state machines or whatever.
[1]: cstheory.stackexchange.com
[2]: http://www.reddit.com/r/askcomputerscience
Would you recommend taking the math classes before starting the classes you pointed out? I only have had Calc 1 and discrete math before.
There is an alternative approach that inadvertently worked for me that may work well for you, particularly if your goal is to learn things beyond a conventional computer science curriculum. I have been working in high-end theoretical computer science for many years, with significant advances to my credit, but I went to school for chemical engineering. People are often curious as to how I acquired so much esoteric domain expertise in theoretical computer science without spending any time in a computer science class.
Pick an interesting "hard" problem in computer science and attempt to solve it. This works best if either there are no published solutions (e.g. massively parallelizing ad hoc graph search) or the solution space varies widely under the possible parameters (e.g. the design of a unique high-performance database kernel). Iterate, be creative, search the literature for interesting ideas that you can borrow from, and evaluate new designs you come up with. If something does not work, understand how and why and try to fix it. Wash, rinse, repeat.
This sounds like a lot of work, and it is, but there are three unique upsides. First, the challenge of it can be quite a bit of fun in its own right and there is a clear goal of what you are trying to achieve which helps keep you motivated and measures progress. Second, you will cover more of the phase space and explore some tangential theoretical areas while hunting for solutions to narrow sub-problems that you would never be exposed to in a more structured setting. Third, you will invariably explore some unusual or unorthodox ideas that you simply would not be exposed to in a directed, formal program. From the standpoint of being an effective theoretical computer scientist, it will help you develop a unique perspective of the problem space that is different than the perspective that arises from more structured programs.
Also, you might actually solve one of the unsolved theoretical computer science problems you have as your challenge in addition to developing a deep understanding of the surrounding problem space. That happened to me (and then replicated later once I realized it was possible) and it would never happen if I was simply taking courses on the subject matter.
This was an effective way for me to gain in-depth understanding of a diverse range of theoretical computer science areas on an informal basis. One of the downsides is that your rote knowledge will not perfectly overlap what is taught in CS curricula even though what you do know will likely be more valuable (e.g. in spatial indexing some CS courses teach priority trees even though they have no memorable theoretical value).
Pick an interesting "hard" problem in computer science and attempt to solve it. This works best if either there are no published solutions (e.g. massively parallelizing ad hoc graph search) or the solution space varies widely under the possible parameters (e.g. the design of a unique high-performance database kernel). Iterate, be creative, search the literature for interesting ideas that you can borrow from, and evaluate new designs you come up with. If something does not work, understand how and why and try to fix it. Wash, rinse, repeat.
This sounds like a lot of work, and it is, but there are three unique upsides. First, the challenge of it can be quite a bit of fun in its own right and there is a clear goal of what you are trying to achieve which helps keep you motivated and measures progress. Second, you will cover more of the phase space and explore some tangential theoretical areas while hunting for solutions to narrow sub-problems that you would never be exposed to in a more structured setting. Third, you will invariably explore some unusual or unorthodox ideas that you simply would not be exposed to in a directed, formal program. From the standpoint of being an effective theoretical computer scientist, it will help you develop a unique perspective of the problem space that is different than the perspective that arises from more structured programs.
Also, you might actually solve one of the unsolved theoretical computer science problems you have as your challenge in addition to developing a deep understanding of the surrounding problem space. That happened to me (and then replicated later once I realized it was possible) and it would never happen if I was simply taking courses on the subject matter.
This was an effective way for me to gain in-depth understanding of a diverse range of theoretical computer science areas on an informal basis. One of the downsides is that your rote knowledge will not perfectly overlap what is taught in CS curricula even though what you do know will likely be more valuable (e.g. in spatial indexing some CS courses teach priority trees even though they have no memorable theoretical value).
Pick an interesting "hard" problem in computer science and attempt to solve it. This works best if either there are no published solutions (e.g. massively parallelizing ad hoc graph search) or the solution space varies widely under the possible parameters (e.g. the design of a unique high-performance database kernel). Iterate, be creative, search the literature for interesting ideas that you can borrow from, and evaluate new designs you come up with. If something does not work, understand how and why and try to fix it. Wash, rinse, repeat.
How do you verify your attempted solutions and talk to other researchers? I've always found it hard enough getting a hold of my own advisor or someone else in our department, let alone someone at an official research institution if I weren't in one.
How do you verify your attempted solutions and talk to other researchers? I've always found it hard enough getting a hold of my own advisor or someone else in our department, let alone someone at an official research institution if I weren't in one.
That part has always proven to be pretty easy. For most interesting problem areas of theoretical computer science, there is rarely more than a handful of deep experts. Anyone else you talk to would likely need to invest a lot of effort understanding the nuances of the theoretical problem before they could have an informed opinion, which they aren't going to do. Identify the few real experts on the topic and talk to them.
These individuals have two useful characteristics. First, they can identify 99% of non-solutions immediately upon casual inspection. Second, if you show them something novel in their area of expertise that they cannot immediately dismiss, it piques their interest and they will invest the time to actually understand what you've done. I've never run into a case where I developed a novel algorithm or similar where the other experts working in the field were not happy to study it if I sent them a cold email.
However, a big part of the learning experience is being able to show that a particular algorithm must have a desired set of characteristics solely by its description before you show it to another expert. The unique skill you are learning by doing things this way is how to take an arbitrary set of complex behavioral constraints and designing an algorithm or system that conforms to those constraints. This is very different than the more orthodox course approach where you learn the constraints within which a given set of well-known algorithms exist and then select the appropriate algorithm for a use case. There are many real-world cases where all well-known algorithms cannot fit within the necessary constraints. Designing custom algorithms is not particularly difficult, but it requires grokking all the theoretical knobs that can be turned for a problem space rather than memorizing variants of a particular solutions.
These individuals have two useful characteristics. First, they can identify 99% of non-solutions immediately upon casual inspection. Second, if you show them something novel in their area of expertise that they cannot immediately dismiss, it piques their interest and they will invest the time to actually understand what you've done. I've never run into a case where I developed a novel algorithm or similar where the other experts working in the field were not happy to study it if I sent them a cold email.
However, a big part of the learning experience is being able to show that a particular algorithm must have a desired set of characteristics solely by its description before you show it to another expert. The unique skill you are learning by doing things this way is how to take an arbitrary set of complex behavioral constraints and designing an algorithm or system that conforms to those constraints. This is very different than the more orthodox course approach where you learn the constraints within which a given set of well-known algorithms exist and then select the appropriate algorithm for a use case. There are many real-world cases where all well-known algorithms cannot fit within the necessary constraints. Designing custom algorithms is not particularly difficult, but it requires grokking all the theoretical knobs that can be turned for a problem space rather than memorizing variants of a particular solutions.
I couldn't imagine diving into open problems on your own without even undergraduate-level training. Open problems are what you work on for your PhD thesis, and before starting your thesis you take PhD-level courses to get up to date on the state of knowledge, and then a faculty member advises you on your thesis. Without knowing what you don't know, you'd end up reinventing wheels, most likely.
> I have been working in high-end theoretical computer science for many year
I think you may be discounting the on-the-job training you surely received. Maybe even more important would be the fact that your job helped steer you toward problems worth solving.
> I have been working in high-end theoretical computer science for many year
I think you may be discounting the on-the-job training you surely received. Maybe even more important would be the fact that your job helped steer you toward problems worth solving.
Before you dive in to hard problem, don't you need to create certain basic CS foundation? For example, can you start working on database kernel without studying even basic algorithms like red-black trees and B-trees? It would be interesting to know what are your go-to books for CS and other resources you use to dive in to particular topic? For example, do you start with Arxive or plain old web searches or taking cues from expert you know in person or look for books on Amazon or attend conferences?
Another thing is that creating algorithms are easy, especially the ones with bugs :). The difference between a joe-the-programmer creating an algorithm and a CS PhD creating an algorithm is that later produces a proof of correctness and performance while former relies on intuition for same thing. This is the hardest and most critical part of being a CS researcher. It would be interesting to know your approach for acquiring these skills.
Another thing is that creating algorithms are easy, especially the ones with bugs :). The difference between a joe-the-programmer creating an algorithm and a CS PhD creating an algorithm is that later produces a proof of correctness and performance while former relies on intuition for same thing. This is the hardest and most critical part of being a CS researcher. It would be interesting to know your approach for acquiring these skills.
Yes and no. You do need to have broad enough experience with programming that you are comfortable with the basic implementation algorithms even if not up to speed on the nuances of them. However, a modern database kernel typically has few or no B-trees or red-black trees in it, using more specialized and purpose-built algorithms instead. Probably one of the most important lessons I learned when I first started poking at database kernels is that the algorithms in the literature were demonstrably not being used in commercial databases because the behavior did not match what you would expect. The algorithms you learn in a CS curriculum are often not really correct for any real system built with a high degree of skill.
I've only ever owned a few CS-related books (e.g. Li and Vitanyi). Trawling search engines for CS papers and references to CS papers with potentially useful information is how I did most of my work. Honestly, Wikipedia is a good starting point to find search terms but you still have to follow threads in the literature that are poorly or non-existently documented on the Internet. There are important areas of literature that still have very limited presence on the searchable web. This is a huge bias I see with many people fresh out of school -- if it is not on the web it doesn't exist.
On your last point, that is the most difficult part. To be frank, most PhDs in computer science do not have the skills to successfully design new algorithms either. Oddly, chemical engineering provided a very good conceptual framework for thinking about this; there are no pure mathematical solutions and you optimize complex systems in part by understanding every knob in a holistic context. One of the things I've noticed about most people with a computer science education is that they make a large number of implicit assumptions about the nature of computational and algorithmic systems that are not required and sometimes not correct, often without realizing it. Most of the algorithm work I've done amounts to violating implicit but foolish assumptions in the orthodox literature that no one had seriously examined in decades.
It is not just computer science, in most fields of endeavor the vast majority of things are done the way they are because that is the way they've always been done as set forth by a smart person a long time ago. The underlying assumptions and implicit constraints are rarely reexamined. I found a lot of upside to studying theoretical knobs that had been essentially frozen in literature since at least the 1980s.
I've only ever owned a few CS-related books (e.g. Li and Vitanyi). Trawling search engines for CS papers and references to CS papers with potentially useful information is how I did most of my work. Honestly, Wikipedia is a good starting point to find search terms but you still have to follow threads in the literature that are poorly or non-existently documented on the Internet. There are important areas of literature that still have very limited presence on the searchable web. This is a huge bias I see with many people fresh out of school -- if it is not on the web it doesn't exist.
On your last point, that is the most difficult part. To be frank, most PhDs in computer science do not have the skills to successfully design new algorithms either. Oddly, chemical engineering provided a very good conceptual framework for thinking about this; there are no pure mathematical solutions and you optimize complex systems in part by understanding every knob in a holistic context. One of the things I've noticed about most people with a computer science education is that they make a large number of implicit assumptions about the nature of computational and algorithmic systems that are not required and sometimes not correct, often without realizing it. Most of the algorithm work I've done amounts to violating implicit but foolish assumptions in the orthodox literature that no one had seriously examined in decades.
It is not just computer science, in most fields of endeavor the vast majority of things are done the way they are because that is the way they've always been done as set forth by a smart person a long time ago. The underlying assumptions and implicit constraints are rarely reexamined. I found a lot of upside to studying theoretical knobs that had been essentially frozen in literature since at least the 1980s.
Solve any 300 problems of the ACM problemset.
http://uva.onlinejudge.org/index.php?option=com_onlinejudge&...
http://uva.onlinejudge.org/index.php?option=com_onlinejudge&...
There's a guy called Scott H. Young who, supposedly, did the entire (4 year) MIT EECS curriculum in a year. So, if you'd like to do it the REALLY hard way, try something like this: http://www.scotthyoung.com/blog/mit-challenge/
I'm a current senior Design & Development option at Penn State. The degree is basically worthless in terms of teaching you real software development. I did learn a lot about the social aspects of working for clients and working in teams, but I did not learn the proper structure to actually work on software. I literally spend all of my free time studying software development and computer science, because I know going to school will only yield me a piece of paper that will get me taken more seriously.
Here's a list of what I've done:
- Learn more languages. Penn State only teaches Java. I've worked professionally with Python, have open source projects in Go, and have read books on Erlang, Haskell, Scala, Clojure. The more languages you know the more abstract concepts you'll know that you can apply in any language.
- Watch online lectures in your free time. MIT Open Courseware, UC Berekely, Coursera, YouTube, InfoQ are all really good resources.
- Start a reading collection. I have tons of books and white papers downloaded that I read.
- Get an interest. I like distributed systems. I honestly don't know much about them, but I think the problems are interesting and I've read some of the important white papers that establish the field. Establishing an interest will help guide you down the rabbit hole.
- Get out there and do stuff. Find a project on GitHub or make one with a circle of friends. Pick something outside your comfort zone. Even if the project never sees the light of production use, you'll learn from the experience. Having your name in open source projects AUTHORS/CONTRIBUTORS files is also very nice.
I'm getting closer to graduating, so I've been reading up on algorithms and data structures again because I know I need brushing up on it. I'm close to getting a compsci minor and was recently debating just getting a masters in it instead. If you want someone to vent your experiences at PSU, feel free to drop me an email.
Here's a list of what I've done:
- Learn more languages. Penn State only teaches Java. I've worked professionally with Python, have open source projects in Go, and have read books on Erlang, Haskell, Scala, Clojure. The more languages you know the more abstract concepts you'll know that you can apply in any language.
- Watch online lectures in your free time. MIT Open Courseware, UC Berekely, Coursera, YouTube, InfoQ are all really good resources.
- Start a reading collection. I have tons of books and white papers downloaded that I read.
- Get an interest. I like distributed systems. I honestly don't know much about them, but I think the problems are interesting and I've read some of the important white papers that establish the field. Establishing an interest will help guide you down the rabbit hole.
- Get out there and do stuff. Find a project on GitHub or make one with a circle of friends. Pick something outside your comfort zone. Even if the project never sees the light of production use, you'll learn from the experience. Having your name in open source projects AUTHORS/CONTRIBUTORS files is also very nice.
I'm getting closer to graduating, so I've been reading up on algorithms and data structures again because I know I need brushing up on it. I'm close to getting a compsci minor and was recently debating just getting a masters in it instead. If you want someone to vent your experiences at PSU, feel free to drop me an email.
I'm doing the Algorithms course on Coursera. It's fantastic. Interesting and practical, and taught by Princeton's Sedgewick. I have a BSc in AI and 20 years professional experience yet I'm still learning a lot from this course each week.
honestly? take a grad class in algorithms and sit in in as many relevant seminars as you can. conversation is much stronger for theoretical knowledge than it is for memorization. the idea is to work through the problems yourself with strong mentor support, so there really isn't a shortcut here.
There are some good suggestions for topics and what all to learn, here are some links to actually go and learn em!
http://stackoverflow.com/questions/194812/list-of-freely-ava...
and
http://hackershelf.com/browse/
Should have everything you need, they're all free / open-source books!
http://stackoverflow.com/questions/194812/list-of-freely-ava...
and
http://hackershelf.com/browse/
Should have everything you need, they're all free / open-source books!
I would also suggest professor Matt Might's blog post "What every CS major should know?" http://matt.might.net/articles/what-cs-majors-should-know/ also, check out his other posts, they're excellent.
I ended becoming the first employee at Bay Area startup. They were a bit desperate and I convinced them I could make their Android apps. I did and they are both highly rated. I also made one of my own that ended up getting featured on Google Play and receiving a lot of press. I have a bit of knack for it but I am so bored with GUIs and UIs. I want to work on harder problems and not just use other peoples libraries. The problem is that my lack of knowledge limits my thinking. I want to fix this. I am torn between going back to school for another BS in CS, pursuing a MS in CS, or teaching myself from things like MIT Open Courseware and Udacity.
I am not really worried about getting a job. This is about gaining knowledge. Although having a "real" CS would certainly help keep me from being pigeon holed as an app developer and open a few more doors. I do think that going back to school would be the quickest route to gaining the knowledge I want. I am just worried that I wouldn't get into a top school and I don't know if I should try to do a BS, MS, or just take classes. I really don't want another $50k in debt either. Especially when things like the Georgia Tech online MS CS is coming out for $7k.
So, what is the best way for a proven and largely self taught developer to take their knowledge to the next level?