Great ideas in theoretical computer science(cs251.com)
cs251.com
Great ideas in theoretical computer science
https://www.cs251.com/
47 comments
Hi, I am one of those code monkey you mentioned. I never took affinity to computer science/math, but I love building real world products with software. I built some really hard and interesting products. Basically, I love playing with tools and building tools from programming paradigms I know. Right now, I am struggling with all the interviews that have these leetcode problems. What sort of career in IT do you think would be best fit for me?
Well what I'll say is this: my job never had leetcode. Embedded engineering, especially if you do FPGA work, is very different from what leetcode has. Honestly if recruiters are using it for jobs like mine, they really don't get it. But I don't know you nearly enough to know. There are so many different fields up and down the stack. Front end, backend, embedded, cloud, edge, consumer, IoT... The list goes on. I would cast a wider net, I guess.
There are three parts of building products in software just like everywhere else:
1. understanding what product needs to be built
2. Having the technical expertise to build them
3. how much effort is needed to build it
And all three things can be hard in themselves. The product can technically be just straightforward (even if somewhat tedious) but you should know what to build because you should know what the customers need.
The product can be technically so challenging that without the theoretical background, you would not even know that such a product would be possible. (An example here would be something say that requires distributed and independent time clocks).
Finally the product can be something that is technically simple, obvious in its customer demand but requires quite a bit of effort and therefore requires you to be good at procuring resources and managing them.
You need to figure out which of these three things made you successfull with the product you call really hard and interesting and pursue that line of industry (even if it is not software!). And then slowly try to become good at other two things.
1. understanding what product needs to be built
2. Having the technical expertise to build them
3. how much effort is needed to build it
And all three things can be hard in themselves. The product can technically be just straightforward (even if somewhat tedious) but you should know what to build because you should know what the customers need.
The product can be technically so challenging that without the theoretical background, you would not even know that such a product would be possible. (An example here would be something say that requires distributed and independent time clocks).
Finally the product can be something that is technically simple, obvious in its customer demand but requires quite a bit of effort and therefore requires you to be good at procuring resources and managing them.
You need to figure out which of these three things made you successfull with the product you call really hard and interesting and pursue that line of industry (even if it is not software!). And then slowly try to become good at other two things.
> Right now, I am struggling with all the interviews that have these leetcode problems.
There's no need to struggle with them. You'll cover fully half of them simply with experience with trees. Build them. Traverse them top-down. Then bottom-up. Search it, Balance it, then rebalance on modification. Invert it, prune it, Roundtrip it to JSON, etc.
After mastering that, given a problem of "develop a flood-fill algorithm to this specification" should be a walk in the park.
There's no need to struggle with them. You'll cover fully half of them simply with experience with trees. Build them. Traverse them top-down. Then bottom-up. Search it, Balance it, then rebalance on modification. Invert it, prune it, Roundtrip it to JSON, etc.
After mastering that, given a problem of "develop a flood-fill algorithm to this specification" should be a walk in the park.
Off the top of my head, possible problems here might be:
1. To handle trees easily you have to understand recursion. Somebody with no CS experience (and no parser experience) might never ever used recursion in anger, even if they launched zillion commercial projects.
2. No Big-O notation. Interviewers usually ask questions about time/space complexity.
And the elephant in the room is that leetcodes are getting trickier and trickier, that's LLM era for you.
1. To handle trees easily you have to understand recursion. Somebody with no CS experience (and no parser experience) might never ever used recursion in anger, even if they launched zillion commercial projects.
2. No Big-O notation. Interviewers usually ask questions about time/space complexity.
And the elephant in the room is that leetcodes are getting trickier and trickier, that's LLM era for you.
I haven't looked at leetcode in a long time but if the problems require e.g. rebalancing a tree, I honestly don't remember how to do it and might not be able to reason it out on the spot either. I have no problems with concepts like recursion or computational complexity though.
It sounds like leetcode problems require either memorization of a significant number of algorithm design patterns or seriously sharp algorithmic problem solving skills.
It sounds like leetcode problems require either memorization of a significant number of algorithm design patterns or seriously sharp algorithmic problem solving skills.
> Off the top of my head, possible problems here might be:
> 1. To handle trees easily you have to understand recursion.
Well that's the point of the exercises I suggest - to learn recursion :-/
> No Big-O notation. Interviewers usually ask questions about time/space complexity
Well, that's the other half of the interview, I only promised that my method will cover half the questions :-)
> 1. To handle trees easily you have to understand recursion.
Well that's the point of the exercises I suggest - to learn recursion :-/
> No Big-O notation. Interviewers usually ask questions about time/space complexity
Well, that's the other half of the interview, I only promised that my method will cover half the questions :-)
> Well that's the point of the exercises I suggest - to learn recursion :-/
Then we must have been talking past each other. We are considering this problem here:
>> I never took affinity to computer science/math, but I love building real world products with software. [...] I am struggling with all the interviews that have these leetcode problems.
Then your advice to these interview problem is to learn some CS - the useful parts of it? E.g. recursion is CS.
Then we must have been talking past each other. We are considering this problem here:
>> I never took affinity to computer science/math, but I love building real world products with software. [...] I am struggling with all the interviews that have these leetcode problems.
Then your advice to these interview problem is to learn some CS - the useful parts of it? E.g. recursion is CS.
>>>but I love building real world products with software.
> Then your advice to these interview problem is to learn some CS - the useful parts of it?
Theory was not my advice.
My advice was to do practical things with trees; I felt it would fit his love of building real things.
I mean, I didn't recommend doing problems out of a textbook, did I?
> Then your advice to these interview problem is to learn some CS - the useful parts of it?
Theory was not my advice.
My advice was to do practical things with trees; I felt it would fit his love of building real things.
I mean, I didn't recommend doing problems out of a textbook, did I?
> radicalized me to believing we needed programming and computer science to be different majors.
Some universities offer Software Engineering as a BEng as well as CompSci as a BSc. At least in Canada.
Some universities offer Software Engineering as a BEng as well as CompSci as a BSc. At least in Canada.
But software engineering as a concept still isn't writing code. I'm a bit of a stickler about it as somebody who has an engineering degree, but when programmers with a CS degree say they're a software engineer, they're not. Software engineering as far as I understand it from the little bit I did in school is actually engineering. Requirements analysis, breaking down the problem, following methodology, etc. It's not just that they're writing somewhere.
So really there should be 3 fields of study: 1. The theory - computer science 2. How to apply the theory - software engineering 3. How to turn those designs into reality - programmers
It's like the mech engineering side. You have materials science and stuff, then mechanical engineers, then machinists.
So really there should be 3 fields of study: 1. The theory - computer science 2. How to apply the theory - software engineering 3. How to turn those designs into reality - programmers
It's like the mech engineering side. You have materials science and stuff, then mechanical engineers, then machinists.
For what it's worth, my computer science degree also had courses and projects that included requirements analysis, breaking down the problem, and elements of software engineering methodology and project management. (I believe we had a course titled "software engineering" even though the university doesn't award engineering degrees.)
I suppose in some schools computer science programmes might be fairly distinct from engineering ones. However, it seems that in lots of places a bachelor's in computer science is rather an generalist degree that covers lots of (mostly software) tech topics and some CS theory.
I'd still have trouble calling myself a software engineer, though, since I don't technically have an engineering degree, even though in lots of places my job might be described as such.
I also don't know a single programmer/developer whose job is distinct from field 2.
I suppose in some schools computer science programmes might be fairly distinct from engineering ones. However, it seems that in lots of places a bachelor's in computer science is rather an generalist degree that covers lots of (mostly software) tech topics and some CS theory.
I'd still have trouble calling myself a software engineer, though, since I don't technically have an engineering degree, even though in lots of places my job might be described as such.
I also don't know a single programmer/developer whose job is distinct from field 2.
I agree with this. As someone who took 3 degrees in computer science, one called "systems developer" and another called "software engineer", I can confidently say we have a taxonomy problem in computer science education.
It makes me crazy that companies labels there entire programmer workforce as "software engineers" when there are no engineering concepts involved at all. Other fields (medical, mechanical, civil engineering) are a lot more mature in this area and have solved this issue long ago.
It makes me crazy that companies labels there entire programmer workforce as "software engineers" when there are no engineering concepts involved at all. Other fields (medical, mechanical, civil engineering) are a lot more mature in this area and have solved this issue long ago.
I was one of those students initially but then came to appreciate the beauty in the science and nature of computation. I always tell people that computer science is not about programming -- its about studying the dynamics of problems in nature.
Unfortunately, I realized this a little late, and it wasn't until towards the end of my final year when I started appreciating these courses. I long for a day I can return to studying these topics.
Unfortunately, I realized this a little late, and it wasn't until towards the end of my final year when I started appreciating these courses. I long for a day I can return to studying these topics.
> we needed programming and computer science to be different majors.
Neither are needed anymore if people will be able to tell a chatbot what to do.
Instead you’ll need a Problem Solution Verification minor.
Neither are needed anymore if people will be able to tell a chatbot what to do.
Instead you’ll need a Problem Solution Verification minor.
Is code monkey derogatory here?
How about "Software Simian"?
https://ia601605.us.archive.org/view_archive.php?archive=/23...
https://ia601605.us.archive.org/view_archive.php?archive=/23...
No. As somebody who lives in the trenches, I'm not too far behind. I've met people who say they can only do theory and can't do the practical side of this area. I think they might use it derogatorily but anybody who does clearly doesn't understand that both sides are necessary. So if anybody says it in that tone, just walk away.
Discussion 2 years ago: https://news.ycombinator.com/item?id=39720388
The similiar course in
- Stanford: https://web.stanford.edu/class/cs208e/cgi-bin/main.cgi/
- MIT https://ocw.mit.edu/courses/6-080-great-ideas-in-theoretical...
- Harvard https://beta.my.harvard.edu/course/COMPSCI1/2026-Spring/001
The similiar course in
- Stanford: https://web.stanford.edu/class/cs208e/cgi-bin/main.cgi/
- MIT https://ocw.mit.edu/courses/6-080-great-ideas-in-theoretical...
- Harvard https://beta.my.harvard.edu/course/COMPSCI1/2026-Spring/001
Randomized algorithms are so damn cool. They really feel like cheating your way out of NP problems. I highly recommend that anyone interested in algorithms studies them.
https://www.wisdom.weizmann.ac.il/~oded/PDF/rnd.pdf
Then you reach derandomization and it hits you once again, it shows you that maybe you can "cheat" in the same way without randomness. Not for free, you usually trade randomness for a bit more running time, but your algorithms stay deterministic. Some believe all probabilistic algorithms can be derandomized (BPP = P), which would be a huge miracle if true.
Then you reach derandomization and it hits you once again, it shows you that maybe you can "cheat" in the same way without randomness. Not for free, you usually trade randomness for a bit more running time, but your algorithms stay deterministic. Some believe all probabilistic algorithms can be derandomized (BPP = P), which would be a huge miracle if true.
+1. My favourite bit is when pivots are chosen randomly in quicksort, we get linearithmic expected complexity. The CLRS proof using indicator random vars was a oh-shit moment.
Btw, you can make quicksort deterministically O(n log n) if you pick the pivot point with linear median search algorithm. It's impressive how randomness lets you pick a balanced pivot, but even more impressive that you could do the same without randomness.
Similarly, quickselect with random pivots runs in expected linear time.
For a proof of this, see https://github.com/tmoertel/practice/blob/master/dailycoding...
For a proof of this, see https://github.com/tmoertel/practice/blob/master/dailycoding...
[deleted]
Earlier discussions:
March 15, 2024 Great ideas in theoretical computer science
https://news.ycombinator.com/item?id=39720388
93 comments
March 15, 2024 Great ideas in theoretical computer science
https://news.ycombinator.com/item?id=39720388
93 comments
Module 8 is worded in a way that surprised me: “if we could decide the languages in NP efficiently, this would lead to amazing applications.”
My understanding of P=?NP is that all intuition points to the answer being “no”. The openness of the question doesn’t give us hope that a magical algorithm might one day arrive. It just means that despite a lot of suspicion that NP cannot be solved in polynomial time, we cannot yet prove this to be the case.
I suspect the answer to BANKACCOUNT=?1_000_000 is “no”, but although my card stops working every month, it starts working again on the last Friday of the month! My research team and I hope to visit an ATM one day to prove my bank balance is meager but until then it remains an open question (albeit likely false) in Gorgoiler Science.
My understanding of P=?NP is that all intuition points to the answer being “no”. The openness of the question doesn’t give us hope that a magical algorithm might one day arrive. It just means that despite a lot of suspicion that NP cannot be solved in polynomial time, we cannot yet prove this to be the case.
I suspect the answer to BANKACCOUNT=?1_000_000 is “no”, but although my card stops working every month, it starts working again on the last Friday of the month! My research team and I hope to visit an ATM one day to prove my bank balance is meager but until then it remains an open question (albeit likely false) in Gorgoiler Science.
There were surprising results in computational complexity before. We can't prove that P /= PSPACE which should be much easier to prove and yet people are stuck on trying to prove P /= NP. Personally I don't think that P = NP and if it were it would be very surprising, but "very surprising" in not "out of the question".
It's worth keeping in mind that regardless of whether P = NP, we might never prove it one way or another. We already know from Godel that some statements can't be proven true or false in a given logical system (without introducing inconsistencies), so it's certainly possible that this is one of them. At the risk of stating the obvious, this quite different from the hypothetical you give where we definitively know that it's provable.
>
Module 8 is worded in a way that surprised me: “if we could decide the languages in NP efficiently, this would lead to amazing applications.”
> My understanding of P=?NP is that all intuition points to the answer being “no”. The openness of the question doesn’t give us hope that a magical algorithm might one day arrive. It just means that despite a lot of suspicion that NP cannot be solved in polynomial time, we cannot yet prove this to be the case.
I admit that I am slightly biased towards that P=NP does indeed hold. I know that this is a non-mainstream view; even though I could give arguments to actual experts in the field that (surprisingly) did convince them that it is much less "clear" than they thought all the time that P \neq NP "must" hold.
On the other hand, I consider it to be plausible that the fastest algorithm that might exist for a "natural" NP-complete problem (e.g. 3-SAT) that exists could have a runtime of \Theta(n^(2^(2^(2^1000000)))).
Or, it is also possible that the proof of P=NP might be highly non-constructive, and obtaining an actual algorithm from this non-constructive proof might be even harder than proving P=NP. Such a situation actually exists in the Robertson-Seymour theorem ("every family of graphs that is closed under taking minors can be defined by a finite set of forbidden minors"):
> https://en.wikipedia.org/w/index.php?title=Robertson%E2%80%9...
Unluckily,
> https://en.wikipedia.org/w/index.php?title=Robertson%E2%80%9...
"The Robertson–Seymour theorem has an important consequence in computational complexity, due to the proof by Robertson and Seymour that, for each fixed graph H, there is a polynomial time algorithm for testing whether a graph has H as a minor. [...] As a result, for every minor-closed family F, there is polynomial time algorithm for testing whether a graph belongs to F: check whether the given graph contains H for each forbidden minor H in the obstruction set of F.
However, this method requires a specific finite obstruction set to work, and the theorem does not provide one. The theorem proves that such a finite obstruction set exists, and therefore the problem is polynomial because of the above algorithm. However, the algorithm can be used in practice only if such a finite obstruction set is provided. As a result, the theorem proves that the problem can be solved in polynomial time, but does not provide a concrete polynomial-time algorithm for solving it."
So, it is in my opinion a very "religious" belief that even if we could prove P=NP, a practically relevant algorithm will arise from this proof.
> My understanding of P=?NP is that all intuition points to the answer being “no”. The openness of the question doesn’t give us hope that a magical algorithm might one day arrive. It just means that despite a lot of suspicion that NP cannot be solved in polynomial time, we cannot yet prove this to be the case.
I admit that I am slightly biased towards that P=NP does indeed hold. I know that this is a non-mainstream view; even though I could give arguments to actual experts in the field that (surprisingly) did convince them that it is much less "clear" than they thought all the time that P \neq NP "must" hold.
On the other hand, I consider it to be plausible that the fastest algorithm that might exist for a "natural" NP-complete problem (e.g. 3-SAT) that exists could have a runtime of \Theta(n^(2^(2^(2^1000000)))).
Or, it is also possible that the proof of P=NP might be highly non-constructive, and obtaining an actual algorithm from this non-constructive proof might be even harder than proving P=NP. Such a situation actually exists in the Robertson-Seymour theorem ("every family of graphs that is closed under taking minors can be defined by a finite set of forbidden minors"):
> https://en.wikipedia.org/w/index.php?title=Robertson%E2%80%9...
Unluckily,
> https://en.wikipedia.org/w/index.php?title=Robertson%E2%80%9...
"The Robertson–Seymour theorem has an important consequence in computational complexity, due to the proof by Robertson and Seymour that, for each fixed graph H, there is a polynomial time algorithm for testing whether a graph has H as a minor. [...] As a result, for every minor-closed family F, there is polynomial time algorithm for testing whether a graph belongs to F: check whether the given graph contains H for each forbidden minor H in the obstruction set of F.
However, this method requires a specific finite obstruction set to work, and the theorem does not provide one. The theorem proves that such a finite obstruction set exists, and therefore the problem is polynomial because of the above algorithm. However, the algorithm can be used in practice only if such a finite obstruction set is provided. As a result, the theorem proves that the problem can be solved in polynomial time, but does not provide a concrete polynomial-time algorithm for solving it."
So, it is in my opinion a very "religious" belief that even if we could prove P=NP, a practically relevant algorithm will arise from this proof.
I seem to remember this specific class at the CMU School of Computer Science being described as a “weed-out class”.
When I took the theory of computation class at CMU in the mid-80s it was in the philosophy department. The professor knew almost nothing about actual computers. Which was pretty cool, honestly.
"Computer science is no more about computers than astronomy is about telescopes."
Usually attributed to Dijkstra.
Usually attributed to Dijkstra.
> When I took the theory of computation class at CMU in the mid-80s it was in the philosophy department.
To be fair, theory of computation is more logic/proofs/formalism than what is mistakenly associated with CS nowadays - programming. Besides isn't everything just philosophy at the end of that day?
> The professor knew almost nothing about actual computers.
The foundations of theory of computation were laid before actual computers.
> Which was pretty cool, honestly.
I can't imagine my philosophy professors teaching theory of computation as they too didn't know anything about computers. But I'm sure they would have made it interesting.
To be fair, theory of computation is more logic/proofs/formalism than what is mistakenly associated with CS nowadays - programming. Besides isn't everything just philosophy at the end of that day?
> The professor knew almost nothing about actual computers.
The foundations of theory of computation were laid before actual computers.
> Which was pretty cool, honestly.
I can't imagine my philosophy professors teaching theory of computation as they too didn't know anything about computers. But I'm sure they would have made it interesting.
>philosophy at the end of the day
More like DIscrete Math
I always thought it should be called “ComputING Science” as opposed to “ComputER Science”.
I am old enough to remember when it was 15-199, taught by Steven Rudich, titled "How to think like a computer scientist".
They had to run it for a few years before they realized CS kids who did poorly in the class dropped the major - the implicit signal being "you don't know how to think like a computer scientist".
They had to run it for a few years before they realized CS kids who did poorly in the class dropped the major - the implicit signal being "you don't know how to think like a computer scientist".
Theory is certainly a weed-out class. I think algorithms is certainly more difficult for a dedicated student tho.
It is indeed a weed-out class for the CS majors. It's fairly difficult the whole way through, and the difficulty jump afterwards for required classes is much more manageable. I never struggled with a required CS class after I managed to get an A in this one.
There used to be a 'sophomore college' course at stanford called "Great Ideas in Computer Science" which I completed a while ago. I have great memories from that! https://stanforddaily.com/2015/11/02/classy-classes-cs-54n-g...
Who reaches for a tool they forgot about?
Few organizations invest in solutions only understood by one or two individuals on their team. This is actually what prevents undergraduate cs knowledge from having an impact. It makes me sad.
Undergraduate CS isn't about the things you do most of the time. It's about enabling the occasions where the alternative is to give up and shrug, or perhaps speculate instead of evaluate.
Undergrads, read this before taking a theory of computation class: https://swtch.com/~rsc/regexp/regexp1.html
Few organizations invest in solutions only understood by one or two individuals on their team. This is actually what prevents undergraduate cs knowledge from having an impact. It makes me sad.
Undergraduate CS isn't about the things you do most of the time. It's about enabling the occasions where the alternative is to give up and shrug, or perhaps speculate instead of evaluate.
Undergrads, read this before taking a theory of computation class: https://swtch.com/~rsc/regexp/regexp1.html
I notice "highlights" is essentially empty, which seems to be the referent of the title of this post
The title of the submission is the title of the page, it's not referring to just the one section but the entire course.
I see. Where can we see these great ideas?
The course's title set great expectations.
The actual course has a narrower scope, usually this is just called "Formal Language, Automata and Complexity" or "Introduction to Theoretical Computer Science".
There are many genius ideas in computer science, e.g. divide and conquer, the various forms of abstraction (see SICP), some astonishing algorithms like Dijstra's, Kalman filters, MCMC, Viterbi's or Integer-Bresenham, or data structures like the Bloom filter or Kohonen maps; for an intro, have a look at A. K. Dewdney's book "The Turing Omnibus" for a fairly non-technical exposition of a broad range beyond TCS (beginner level, does not include most from my list here).
The actual course has a narrower scope, usually this is just called "Formal Language, Automata and Complexity" or "Introduction to Theoretical Computer Science".
There are many genius ideas in computer science, e.g. divide and conquer, the various forms of abstraction (see SICP), some astonishing algorithms like Dijstra's, Kalman filters, MCMC, Viterbi's or Integer-Bresenham, or data structures like the Bloom filter or Kohonen maps; for an intro, have a look at A. K. Dewdney's book "The Turing Omnibus" for a fairly non-technical exposition of a broad range beyond TCS (beginner level, does not include most from my list here).
I guess I was just confused by the structure of the course; I didn't intend to imply the stuff it covered wasn't crucial.
Other things not really worth mentioning were that we had some useless digital logic section at the start where we made a full adder and called it a computer. As a CompE, it was weird. The CS students thought they knew all there was to how a computer worked from that. Also, he was only a lecturer because our processor got sick right before the class and they found a grad student to do it. He was ok but took shortcuts and our TA would be like "oh, he did this fast and loose, so now I have to teach you the real way it's done".
It was a great class in retrospect and certainly pushed my boundary on theoretical computing but you could feel the code monkeys regretting their decisions each homework and exam. It's what radicalized me to believing we needed programming and computer science to be different majors.