Mathematics for Computer Science (2024)(ocw.mit.edu)
ocw.mit.edu
Mathematics for Computer Science (2024)
https://ocw.mit.edu/courses/6-1200j-mathematics-for-computer-science-spring-2024/
105 comments
It's unbelievable that the average human being has access to the lectures of some of the best universities in the world for free. 31 hours of in-depth mathematics by some of the best people in their field.
Although I have always been struggling with keeping up with long lecture playlists. I always try to find shorter videos which explain the concept faster (although probably lacking depth). And end up ditching it halfway as well. Perhaps the real motivation to keep up with the material comes from actually enrolling the university? Has anyone completed such type of lectures by themselves? How do you stay consistent and disciplined?
I find courses in some platforms (coursera/khanacademy) a bit more motivating because they kind of push me with deadlines. I guess I am used to deadline-oriented studying.
If anyone else is struggling with attention span and is looking for shorter lectures (although they may not have the same depth): https://www.youtube.com/@ProfessorDaveExplains/playlists
Although I have always been struggling with keeping up with long lecture playlists. I always try to find shorter videos which explain the concept faster (although probably lacking depth). And end up ditching it halfway as well. Perhaps the real motivation to keep up with the material comes from actually enrolling the university? Has anyone completed such type of lectures by themselves? How do you stay consistent and disciplined?
I find courses in some platforms (coursera/khanacademy) a bit more motivating because they kind of push me with deadlines. I guess I am used to deadline-oriented studying.
If anyone else is struggling with attention span and is looking for shorter lectures (although they may not have the same depth): https://www.youtube.com/@ProfessorDaveExplains/playlists
I love math, completed a PhD, and am very self-disciplined. But even so, I don't think I would have been able to learn much on my own with video lectures, at least not at the start. For some reason, it seems like you need to reach a "critical mass" of knowledge first before you can do that, and I've observed that a crucial component is being in a program with others, and definitely having a very experienced mentor.
Without a very experienced mentor, I think it's very difficult to get to the independent-learning stage with math. That's the key. You need someone to go through your work, correct you, and make sure you don't go off in a very wrong direction.
So my advice is find at least a graduate student in math to help you. It's like a piano teacher, if you've ever taken piano, you know it's absolutely mandatory to have a teacher. People who self-learn from the start end up being able to play but not very well.
Edit: one other crucial component is time. If you're really interested in knowing something like linear algebra, analysis, or calculus with fluency, expect to spend at least 10 hours per week on it for a year. Two hours per week will give you a cursory and very weak understanding only.
Without a very experienced mentor, I think it's very difficult to get to the independent-learning stage with math. That's the key. You need someone to go through your work, correct you, and make sure you don't go off in a very wrong direction.
So my advice is find at least a graduate student in math to help you. It's like a piano teacher, if you've ever taken piano, you know it's absolutely mandatory to have a teacher. People who self-learn from the start end up being able to play but not very well.
Edit: one other crucial component is time. If you're really interested in knowing something like linear algebra, analysis, or calculus with fluency, expect to spend at least 10 hours per week on it for a year. Two hours per week will give you a cursory and very weak understanding only.
> But even so, I don't think I would have been able to learn much on my own with video lectures, at least not at the start.
This was exactly my situation. Videos can give you a lot of structured, well presented information. And for MIT courses you'd get this knowledge from the very best. The problem is that no matter how well the subject matter is presented, I would hit some conceptual snag that I couldn't resolve just by repeating the sections in the video.
Now, years ago, to clear up the concepts, I would go to math stack exchange, write down exactly what I wanted to understand using mathjax and hope that someone will provide a detailed enough explanation. Most of the time I did learn from the answers, but sometimes the answer would be too succinct. In such cases there would be a need for a back and forth and stackexchange is not really designed around that usage pattern. This hassle would eventually make me give up the whole endeavor.
Now however there are LLMs. They don't need mathjax to understand what I am talking about and they are pretty good at back and forth. In the past 6 months I have gone through 2 full MIT courses with practice sheets and exams.
So I would encourage anyone who went through the route of self learning via videos and found it to be too cumbersome and lacking to give it another go with your favorite LLM.
This was exactly my situation. Videos can give you a lot of structured, well presented information. And for MIT courses you'd get this knowledge from the very best. The problem is that no matter how well the subject matter is presented, I would hit some conceptual snag that I couldn't resolve just by repeating the sections in the video.
Now, years ago, to clear up the concepts, I would go to math stack exchange, write down exactly what I wanted to understand using mathjax and hope that someone will provide a detailed enough explanation. Most of the time I did learn from the answers, but sometimes the answer would be too succinct. In such cases there would be a need for a back and forth and stackexchange is not really designed around that usage pattern. This hassle would eventually make me give up the whole endeavor.
Now however there are LLMs. They don't need mathjax to understand what I am talking about and they are pretty good at back and forth. In the past 6 months I have gone through 2 full MIT courses with practice sheets and exams.
So I would encourage anyone who went through the route of self learning via videos and found it to be too cumbersome and lacking to give it another go with your favorite LLM.
My only concern with using LLMs to learn new material is being certain that it's not leading me astray.
Too many times I've used LLMs for tasks at work and some of the answers I've gotten back are subtlety wrong. I can skip past those suggestions because the subject is one I'm strong/experienced in and I can easily tell that the LLM is just wrong or speaking nonsense.
But if I didn't have that level of experience, I don't think I would be able to tell where the LLM was wrong/mistaken.
I think LLMs are great for learning new things, but I also think you have to be skeptical of everything it says and need to double check the logic of what it's telling you.
Too many times I've used LLMs for tasks at work and some of the answers I've gotten back are subtlety wrong. I can skip past those suggestions because the subject is one I'm strong/experienced in and I can easily tell that the LLM is just wrong or speaking nonsense.
But if I didn't have that level of experience, I don't think I would be able to tell where the LLM was wrong/mistaken.
I think LLMs are great for learning new things, but I also think you have to be skeptical of everything it says and need to double check the logic of what it's telling you.
I have the same doubts, it's like the old rule of reading a newspaper story. When it's outside your area of expertise you think they're a genius. When it's something you know a lot about you think it's an idiot.
But it might still help, especially if you think about the LLM as a fellow student rather than as a teacher. You try to catch it out, spot where it's misunderstood. Explain to it what you understand and see if it corrects you?
But it might still help, especially if you think about the LLM as a fellow student rather than as a teacher. You try to catch it out, spot where it's misunderstood. Explain to it what you understand and see if it corrects you?
LLMs are indeed excellent as conversation partners for helping with difficult concepts or for working through problem sheets. They’re really opened up self-learning for me again in math. You can use them to go much deeper with concepts much deeper than the course you’re taking - e.g. I was relearning some basic undergrad probability and stats but ended up exploring a bit of measure theory using Gemini as well. I would go so far as to say that an LLM can be more effective for explaining things than a randomly selected graduate student (though some grad students with a particular talent for teaching will be better).
What the LLM still does not provide is accountability (a LLM isn’t going to stop you from skipping a problem set) and the human social component. But you could potentially get that from a community of other self-learners covering the same material if you’re able to pull one together.
What the LLM still does not provide is accountability (a LLM isn’t going to stop you from skipping a problem set) and the human social component. But you could potentially get that from a community of other self-learners covering the same material if you’re able to pull one together.
When it comes to math and CS, doing exercises is a much stronger indicator of self-learning then reading/watching.
Like piano it’s important to tackle courses that are appropriate for your level
Self taught people often skip too much of the basics so they struggle to properly tackle the fancy stuff
Self taught people often skip too much of the basics so they struggle to properly tackle the fancy stuff
Even if they don't skip, they adopt weird hand positions that are hard to correct. There is just too much motor movement that needs to be done right that cannot really be explained or learned by watching a video or reading a book. It's actually similar to math in a certain way, where motor memory is replaced by subtle steps in logical reasoning.
Maybe what’s common is that self taught people do not even know what to look for to improve on.
They only see the superficial or easy-to-spot goals and lack the eye for detail to build foundations and technique.
They only see the superficial or easy-to-spot goals and lack the eye for detail to build foundations and technique.
> completed a PhD... But even so,
Not sure why you added "but even so", getting a PhD is fundamentally about believing in the necessity of the mentor/mentee relationship for learning. It's not at all surprising that you would find:
> You need someone to go through your work, correct you, and make sure you don't go off in a very wrong direction.
I've learned enough to publish (well received) technical books in areas I've never taken a single course in, and have personally found that in-classroom experiences were never as valuable as I had hoped they would be. Of course starting from absolute 0 is challenging, but one good teacher early on can be enough.
Though I also don't think video lectures alone are adequate. Rather than focusing on "exercises", I've found I get the biggest boost in learning when I need to build something or solve a real problems with the mathematical tools I'm studying. Learning a bit, using it to build a real project, and then coming back when you need to unblock the next hurdle is very effective.
On top of this, books are just better for learning than videos (or lectures in general). Lectures are only useful for getting the lay of the land, and getting a feel for how types of problems are worked out. Especially with mathematics, you need time to look at an equation, read ahead, flip back, write it in a notebook, etc until you really start to get it.You really can't possibly get any of these ideas in 45-60 minutes of someone talking about it.
That's why, for me, online lectures don't really change the autodidact game all that much. Reading books and solving problems seems to have been the standard way to learn things well for at least the last several hundred years, and lectures don't improve on that too much.
Not sure why you added "but even so", getting a PhD is fundamentally about believing in the necessity of the mentor/mentee relationship for learning. It's not at all surprising that you would find:
> You need someone to go through your work, correct you, and make sure you don't go off in a very wrong direction.
I've learned enough to publish (well received) technical books in areas I've never taken a single course in, and have personally found that in-classroom experiences were never as valuable as I had hoped they would be. Of course starting from absolute 0 is challenging, but one good teacher early on can be enough.
Though I also don't think video lectures alone are adequate. Rather than focusing on "exercises", I've found I get the biggest boost in learning when I need to build something or solve a real problems with the mathematical tools I'm studying. Learning a bit, using it to build a real project, and then coming back when you need to unblock the next hurdle is very effective.
On top of this, books are just better for learning than videos (or lectures in general). Lectures are only useful for getting the lay of the land, and getting a feel for how types of problems are worked out. Especially with mathematics, you need time to look at an equation, read ahead, flip back, write it in a notebook, etc until you really start to get it.You really can't possibly get any of these ideas in 45-60 minutes of someone talking about it.
That's why, for me, online lectures don't really change the autodidact game all that much. Reading books and solving problems seems to have been the standard way to learn things well for at least the last several hundred years, and lectures don't improve on that too much.
> Not sure why you added "but even so"
Because the "even so" was for the "self-motivated" part, not the "getting the PhD" part.
> I've learned enough to publish (well received) technical books in areas I've never taken a single course in,
I'm talking about pure math here, not other technical fields which are more hands on and don't require as much mentorship. Programming is easier to self-learn than math for sure, because it is not very abstract compared to math. It's also guided by whether the code works or not.
Because the "even so" was for the "self-motivated" part, not the "getting the PhD" part.
> I've learned enough to publish (well received) technical books in areas I've never taken a single course in,
I'm talking about pure math here, not other technical fields which are more hands on and don't require as much mentorship. Programming is easier to self-learn than math for sure, because it is not very abstract compared to math. It's also guided by whether the code works or not.
> I'm talking about pure math here
Well the post is "Mathematics for Computer Science" which I don't think anyone considers "pure math". Most of my writing has been in the area of applied mathematics, the closest I've gotten to pure math would be some stuff on measure theory.
So yea, it might be a challenge to self teach something like cluster algebras, but at that level much of the work in the field is academic communication anyway.
Well the post is "Mathematics for Computer Science" which I don't think anyone considers "pure math". Most of my writing has been in the area of applied mathematics, the closest I've gotten to pure math would be some stuff on measure theory.
So yea, it might be a challenge to self teach something like cluster algebras, but at that level much of the work in the field is academic communication anyway.
I would say that you need to start at a lower level when self learning with a simpler resource. Something like Openstax. People get far too obsessed with the name attached to a resource than whether it is the right method of learning.
I am about finished with my CS PhD and I taught databases at the university during covid. I, personally, would have failed in the remote learning environment we were providing.
I am amazed at those wo fought or even flourished through that.
I am amazed at those wo fought or even flourished through that.
I’m currently enrolled in an online MS program, and I had never struggled so much in courses. The lack of social component might be what’s causing that. The material is mostly a recap of undergrad and things I already knew, so the coursework should not be so difficult for me, but it’s been incredibly difficult.
Then again, William & Mary had some incredible teachers, and maybe the online program through a different school just isn’t very good at designing assignments and teaching by comparison. But I feel that there was a difference in how I could succeed at challenging assignments when I was among other students in a social setting. The work in undergrad was highly rigorous, though exploring it alongside other real-life students made it a very different undertaking.
Then again, William & Mary had some incredible teachers, and maybe the online program through a different school just isn’t very good at designing assignments and teaching by comparison. But I feel that there was a difference in how I could succeed at challenging assignments when I was among other students in a social setting. The work in undergrad was highly rigorous, though exploring it alongside other real-life students made it a very different undertaking.
I'm a fourth-year W&M student considering an online MSCS program post-grad (possibly the same one you're in) - I'd love to hear more about your experience in it, as compared to traditional undergrad, if you'd be willing to share?
[deleted]
LLM could be the teacher, one of the best already..
I've found you have to be very careful with LLM as teacher since, especially when it's the one explaining, it is wrong more often then you might think, and there's no way to know.
The best use of an LLM I've found in learning is for when I explain to it my understanding of what I learned and have it critique what I've said. This has greatly reduced the amount of backtracking I need to do as I start to realize I've misunderstand a foundational concept later on when things stop making sense. Often simply having the model response with "Not quite, ..." is enough to make me realize I need to go back and re-read a section.
The other absolute godsend is just being able to take a picture of an equation in a book and ask for some help understanding it notationally. This is especially helpful when going between fields that use different notation (e.g. statistics -> physics)
The best use of an LLM I've found in learning is for when I explain to it my understanding of what I learned and have it critique what I've said. This has greatly reduced the amount of backtracking I need to do as I start to realize I've misunderstand a foundational concept later on when things stop making sense. Often simply having the model response with "Not quite, ..." is enough to make me realize I need to go back and re-read a section.
The other absolute godsend is just being able to take a picture of an equation in a book and ask for some help understanding it notationally. This is especially helpful when going between fields that use different notation (e.g. statistics -> physics)
I don't think "one of the best" would be wrong that often.
Sure, it’s not perfect — but most of the time, it gets things right. Also it can answer instantly, and always patiently...
LLMs dont have a deep enough understanding of theory of mind to see how someone is stuck and help them get unstuck.
You need to guide your own study and you might not know what you need to learn to get unstuck
You need to guide your own study and you might not know what you need to learn to get unstuck
> LLMs dont have a deep enough understanding of theory of mind to see how someone is stuck and help them get unstuck.
Many teachers cannot do that either.
I find ChatGPT and the Gemini model quite good at problems whose solutions are already known. We just need the Wille—the will—to ask it.
Many teachers cannot do that either.
I find ChatGPT and the Gemini model quite good at problems whose solutions are already known. We just need the Wille—the will—to ask it.
> Many teachers cannot do that either.
Of course there are bad teachers out there. The question wasnt "are there human yeachers as bad as an LLM" it was whether an LLM is as good as a good human teacher
> We just need the Wille—the will—to ask it.
Thats the thing. Its is a very good search resource. But thats not what a teacher is. A good teacher will help you get to the right questions, not just get you the right answers. And the student often wont know the right questions until they already know quite a bit. You need a sufficiently advanced, if incomplete, mental model of the sybject to know what you dont know. An LLM cant really model what your thinking, what your stuck on, and what questions you should be asking
Of course there are bad teachers out there. The question wasnt "are there human yeachers as bad as an LLM" it was whether an LLM is as good as a good human teacher
> We just need the Wille—the will—to ask it.
Thats the thing. Its is a very good search resource. But thats not what a teacher is. A good teacher will help you get to the right questions, not just get you the right answers. And the student often wont know the right questions until they already know quite a bit. You need a sufficiently advanced, if incomplete, mental model of the sybject to know what you dont know. An LLM cant really model what your thinking, what your stuck on, and what questions you should be asking
> You need a sufficiently advanced, if incomplete, mental model of the sybject to know what you dont know.
I believe that through a few common prompts and careful reflection on the LLM's responses, this challenge can be easily overcome. Also, nobody truly knows what you're stuck on or thinking, unless you figure out the existence of unknown and seek it out. However, I do agree with your point that "a good teacher will help you get to the right questions," since a great teacher is an active agent; they can present the unknown parts first, actively forcing you to think about them.
- when people see some things as beautiful(best), other things become ugly(ordinary)....Being and non-being create each other. — Laozi, Tao Te Ching
Perhaps the emphasis on the greatness of an LLM gives the impression that it undermines the greatness of a great human teacher, which has already led to a few downvotes. I want to clarify that I never intended to undermine that. I have encountered a few great teachers in my life, whether during my school years or those teaching in the form of MOOCs. A great teacher excels at activating the students' wille to seek the unknown and teaching more than just knowledge. Also, the LLM relies heavily on these very people to create the useful materials it trains on.
Metaphorically speaking, the LLM is learning from almost all great teachers to become a great 'teacher' itself. In that sense, I find no problem saying "LLM could be the teacher, one of the best already."
I believe that through a few common prompts and careful reflection on the LLM's responses, this challenge can be easily overcome. Also, nobody truly knows what you're stuck on or thinking, unless you figure out the existence of unknown and seek it out. However, I do agree with your point that "a good teacher will help you get to the right questions," since a great teacher is an active agent; they can present the unknown parts first, actively forcing you to think about them.
- when people see some things as beautiful(best), other things become ugly(ordinary)....Being and non-being create each other. — Laozi, Tao Te Ching
Perhaps the emphasis on the greatness of an LLM gives the impression that it undermines the greatness of a great human teacher, which has already led to a few downvotes. I want to clarify that I never intended to undermine that. I have encountered a few great teachers in my life, whether during my school years or those teaching in the form of MOOCs. A great teacher excels at activating the students' wille to seek the unknown and teaching more than just knowledge. Also, the LLM relies heavily on these very people to create the useful materials it trains on.
Metaphorically speaking, the LLM is learning from almost all great teachers to become a great 'teacher' itself. In that sense, I find no problem saying "LLM could be the teacher, one of the best already."
[deleted]
>Perhaps the real motivation to keep up with the material comes from actually enrolling the university?
For most people in most situations, the real motivation to keep up with the material comes from the wage premium one gets after getting the sheepskin. It is unsurprising you, a humble autodidact, are having a lot more trouble than an actual MIT student, because unlike an actual MIT student, you will not walk out of this course any closer to having an MIT degree.
>I guess I am used to deadline-oriented studying.
You can always reverse the curse, and promise to pay someone if you don't finish X material by Y date. You probably also want some kind of proof mechanism to show that you actually did it, like eg a graded test.
>Has anyone completed such type of lectures by themselves? How do you stay consistent and disciplined?
I've read through several textbooks cover to cover including problem sets since graduating. My motivation is mostly just burning curiosity. I can't stand the feeling of not only not knowing a thing, but knowing that I don't know it or feeling like I'm faking it every time I do act on what I know.
For most people in most situations, the real motivation to keep up with the material comes from the wage premium one gets after getting the sheepskin. It is unsurprising you, a humble autodidact, are having a lot more trouble than an actual MIT student, because unlike an actual MIT student, you will not walk out of this course any closer to having an MIT degree.
>I guess I am used to deadline-oriented studying.
You can always reverse the curse, and promise to pay someone if you don't finish X material by Y date. You probably also want some kind of proof mechanism to show that you actually did it, like eg a graded test.
>Has anyone completed such type of lectures by themselves? How do you stay consistent and disciplined?
I've read through several textbooks cover to cover including problem sets since graduating. My motivation is mostly just burning curiosity. I can't stand the feeling of not only not knowing a thing, but knowing that I don't know it or feeling like I'm faking it every time I do act on what I know.
At first your comment rubbed me the wrong way, too cynical.
But it is completely true. No one would ‘learn’ the way college courses are structured. The only reason these courses get completed is the pace/cadence, GPA requirements to get jobs and the degree.
In the ‘real world’ you just learn enough to solve the problem in front of you and as you face more and more your knowledge tree expands.
No one in their right mind would go through a syllabus-like sequence - it is just boring, dull as hell.
But it is completely true. No one would ‘learn’ the way college courses are structured. The only reason these courses get completed is the pace/cadence, GPA requirements to get jobs and the degree.
In the ‘real world’ you just learn enough to solve the problem in front of you and as you face more and more your knowledge tree expands.
No one in their right mind would go through a syllabus-like sequence - it is just boring, dull as hell.
>At first your comment rubbed me the wrong way, too cynical.
It's only cynical if you think making money is bad! I think it's terrific that your average B student and up is mature enough to reliably take on tens of thousands of dollars in debt, and work hard for several years without any immediate reward, in exchange for a pretty reliable pathway towards high paying specialized labor for the rest of their lives. It spits in the face of the narrative that young people are too stupid, or too naive or whatever to have agency in their own lives.
>The only reason these courses get completed is the pace/cadence, GPA requirements to get jobs and the degree.
I cite The Case Against Education, as usual. [1]
>In the ‘real world’ you just learn enough to solve the problem in front of you and as you face more and more your knowledge tree expands. No one in their right mind would go through a syllabus-like sequence - it is just boring, dull as hell.
I cite too John D. Cook's "Just-in-case versus just-in-time" blog post. [2] I don't work through actual syllabi, but I love working through textbooks from start to finish. But you are also correct that I am emphatically not in my right mind, and my career has suffered for it. ;)
[1]: https://www.youtube.com/watch?v=_t957pTcJ0E&t=2s
[2]: https://www.johndcook.com/blog/2010/03/03/just-in-case-versu...
It's only cynical if you think making money is bad! I think it's terrific that your average B student and up is mature enough to reliably take on tens of thousands of dollars in debt, and work hard for several years without any immediate reward, in exchange for a pretty reliable pathway towards high paying specialized labor for the rest of their lives. It spits in the face of the narrative that young people are too stupid, or too naive or whatever to have agency in their own lives.
>The only reason these courses get completed is the pace/cadence, GPA requirements to get jobs and the degree.
I cite The Case Against Education, as usual. [1]
>In the ‘real world’ you just learn enough to solve the problem in front of you and as you face more and more your knowledge tree expands. No one in their right mind would go through a syllabus-like sequence - it is just boring, dull as hell.
I cite too John D. Cook's "Just-in-case versus just-in-time" blog post. [2] I don't work through actual syllabi, but I love working through textbooks from start to finish. But you are also correct that I am emphatically not in my right mind, and my career has suffered for it. ;)
[1]: https://www.youtube.com/watch?v=_t957pTcJ0E&t=2s
[2]: https://www.johndcook.com/blog/2010/03/03/just-in-case-versu...
I echo this sentiment. One of my favorite periods of my life was college, actually getting to learn about some advanced topics in CS. Then I graduated and got a job and now I struggle so hard to learn new things (despite lecture videos and textbooks and LLMs existing) without a professor grading assignments/giving exams/that you can talk to, or classmates.
I’m thinking about enrolling in an online college just for fun. Though the problem I have is that I think the Venn diagram of colleges that are online, aren’t expensive, have advanced CS/ML courses, have an experienced professor that you get to interact with is pretty much zero. If anyone has suggestions, do let me know.
I’m thinking about enrolling in an online college just for fun. Though the problem I have is that I think the Venn diagram of colleges that are online, aren’t expensive, have advanced CS/ML courses, have an experienced professor that you get to interact with is pretty much zero. If anyone has suggestions, do let me know.
Try not to beat yourself up too much about it, I certainly have and it hasn't been very useful to do so.
You have a finite amount of energy in a day and learning takes a lot of energy. It's why a kid's job is the learn.
You could try front running the learning, but it will impact your energy levels at work. It still takes a monumental amounts of discipline, but you may have the energy to make it work.
You have a finite amount of energy in a day and learning takes a lot of energy. It's why a kid's job is the learn.
You could try front running the learning, but it will impact your energy levels at work. It still takes a monumental amounts of discipline, but you may have the energy to make it work.
Maybe the public commitment method would help. I kind of think that's fundamentally what makes college inherently motivating.
This is about weight loss, but I think it can be applied to anything:
https://www.medicspot.co.uk/weight-loss/behaviour-change/pub...
This is about weight loss, but I think it can be applied to anything:
https://www.medicspot.co.uk/weight-loss/behaviour-change/pub...
Georgia Tech has a great online MSc CS program (OMSCS) that's very affordable for what it is, though the amount of direct interaction with the professor varies from class to class.
This course has been a _huge_ help to me in my current project (a G-code previewer and programmatic 3D modeling system written for (Open)PythonSCAD).
I would recommend pairing it with:
https://ocw.mit.edu/courses/6-001-structure-and-interpretati...
and some additional online resources which I've found very helpful:
- https://mathcs.clarku.edu/~djoyce/java/elements/elements.htm...
- https://www.motionmountain.net/
and of course https://librivox.org/ and https://www.gutenberg.org/ --- for a benchmark on why, well, when my father retired to a rural Virginia county, the library was a metal carrel of books in the basement of the old courthouse, and my favourite books during the summer (when I didn't have access to the school libraries) were Hal Clement's _Space Lash_ (which my father found in a tower at the prison where he worked where reading material was forbidden) and an English textbook containing a number of short stories which my mother purchased from a table of remaindered books in a department store in a town 26 miles away to which we might drive once a month or so.
I would recommend pairing it with:
https://ocw.mit.edu/courses/6-001-structure-and-interpretati...
and some additional online resources which I've found very helpful:
- https://mathcs.clarku.edu/~djoyce/java/elements/elements.htm...
- https://www.motionmountain.net/
and of course https://librivox.org/ and https://www.gutenberg.org/ --- for a benchmark on why, well, when my father retired to a rural Virginia county, the library was a metal carrel of books in the basement of the old courthouse, and my favourite books during the summer (when I didn't have access to the school libraries) were Hal Clement's _Space Lash_ (which my father found in a tower at the prison where he worked where reading material was forbidden) and an English textbook containing a number of short stories which my mother purchased from a table of remaindered books in a department store in a town 26 miles away to which we might drive once a month or so.
I got through a few lectures by recognizing that I didn’t have the mathematical training/practice to finish up one video in one sitting. Often times I would need to scurry on over to have some basics explained to me on another site. I did one lecture over several days (weeks if I had to). I think most of the discipline comes from expectation management. Expect to get stuck and need a few moments or days or weeks to mull something over until it becomes more intuitive. Keep a list of things you do and don’t understand (a simple text file / paper is enough) and keep doing it for a few months if you have to and you’ll get there.
Part of the value of a university is exactly that. It builds momentum and incentives. Self paces lectures can be available, but it's extremely hard to follow them if you don't have a good evaluation at the end, or if you don't have deadlines to give assignments.
But also remember, many of those lectures are at a slower peace, so one or two lectures per week. It takes time to internalize the material. People that don't follow university usually try to binge watch them, but this leads to low outcomes.
I think the best strategy is to put deadlines and risks for yourself, and follow them at a natural peace. And, do the exercices.
But also remember, many of those lectures are at a slower peace, so one or two lectures per week. It takes time to internalize the material. People that don't follow university usually try to binge watch them, but this leads to low outcomes.
I think the best strategy is to put deadlines and risks for yourself, and follow them at a natural peace. And, do the exercices.
I completed an earlier version of this class and found structure to be helpful. Found consistent time and place each day to spend some time learning and that helped a ton, but still had weeks of not touching it so the struggle is real :)
A bit of a side note but I find that the lectures are not the most interesting/useful part of those courses. The problem sets and the time spent trying to solve them ended up solidifying so many ideas that I had fooled myself into believing I understood. So I highly recommend heads-down solving some problems. It sinks much more time than the lectures but you come out of it better off
A bit of a side note but I find that the lectures are not the most interesting/useful part of those courses. The problem sets and the time spent trying to solve them ended up solidifying so many ideas that I had fooled myself into believing I understood. So I highly recommend heads-down solving some problems. It sinks much more time than the lectures but you come out of it better off
Its so hard to be self disciplined like that, I find I have to enroll to force myself to engage. I know its a weakness.
In my experience, coursera/khan academy courses have never been able to compete with a rigorous university course. They're great resources when you need alternative explanations, but never stood up on their own.
I think long lecture playlist is a feature, not a bug. It's much harder to commit to such material when you're not full timing education.
I think long lecture playlist is a feature, not a bug. It's much harder to commit to such material when you're not full timing education.
My 5 cents, the value of KA is that it gives you some sort of basic curriculum you can follow. To finish calculus (the "basic", single variable) I've had to pull in lots of other books, youtube channels, courses from other universities, but it still has it's worth. It's like a rope bridge over a high river.
Major weaknesses are some cool sections like Linear Algebra that have no exercises in their respective "tree", but that's very rare.
Major weaknesses are some cool sections like Linear Algebra that have no exercises in their respective "tree", but that's very rare.
Discipline is just making the same choice every time no matter how you feel or what thoughts enter your mind. Your mind will lie to you with thoughts and feelings about why you can’t attend to a lecture. Treat your mind like a child pretending to be sick to get out of school.
Even if it is true that in the moment you aren’t focused or whatever other excuses your mind comes up with, so what? “Go to class” anyway. At worst you learn nothing but improve your discipline skills.
Set a regular time to watch the lectures so you can’t lie to yourself about doing it later.
I want to be clear, it isn’t about willing yourself through it despite everything even if it can read that way. It’s about honoring the choice you made to attend to the lectures and not accepting excuses from your mind.
Recognizing that you can choose, recognizing that past you had good intentions for you and deserves you to honor those intentions, and recognizing that your thoughts and feelings in the moment may not be true and “aren’t the boss of you” even if they are true helps tremendously.
Even if it is true that in the moment you aren’t focused or whatever other excuses your mind comes up with, so what? “Go to class” anyway. At worst you learn nothing but improve your discipline skills.
Set a regular time to watch the lectures so you can’t lie to yourself about doing it later.
I want to be clear, it isn’t about willing yourself through it despite everything even if it can read that way. It’s about honoring the choice you made to attend to the lectures and not accepting excuses from your mind.
Recognizing that you can choose, recognizing that past you had good intentions for you and deserves you to honor those intentions, and recognizing that your thoughts and feelings in the moment may not be true and “aren’t the boss of you” even if they are true helps tremendously.
Explore use of LLM instead of passive viewing of videos.
Pass the link to LLM and ask to summarize it and generate a synopsis and quiz you.
We don't learn from lectures, we learn from problem solving
Also, be modest and assume you're dumber than you think you are - start with courses where you already know at least 50% of materials covered.
Also, be modest and assume you're dumber than you think you are - start with courses where you already know at least 50% of materials covered.
I miss iTunes U from Apple (not iTunes, iTunes U, as in "University").
So many cool (free) courses and so easy to find, download.
So many cool (free) courses and so easy to find, download.
“A world class education available for free, to the undistractable.”
Hmm. One could think of the cost as a hack to force you to not be distracted so much.
The page listing topics (just like the playlist):
https://ocw.mit.edu/courses/6-1200j-mathematics-for-computer...
Lecture notes:
https://ocw.mit.edu/courses/6-1200j-mathematics-for-computer...
There are a few unusual parts, like the last lecture ("Large Deviations"). I'm not familiar with the entire course, but IMO the lecture on state machines is very good; it discusses invariants and uses an approchable example (the 15-puzzle).
Text (last revised 2018): https://courses.csail.mit.edu/6.042/spring18/mcs.pdf
If you have never looked at it, the problems there are very nice. For example, instead of some dry boolean logic problem about A and Not(B), you have Problem 3.17 on page 81, which begins:
https://ocw.mit.edu/courses/6-1200j-mathematics-for-computer...
Lecture notes:
https://ocw.mit.edu/courses/6-1200j-mathematics-for-computer...
There are a few unusual parts, like the last lecture ("Large Deviations"). I'm not familiar with the entire course, but IMO the lecture on state machines is very good; it discusses invariants and uses an approchable example (the 15-puzzle).
Text (last revised 2018): https://courses.csail.mit.edu/6.042/spring18/mcs.pdf
If you have never looked at it, the problems there are very nice. For example, instead of some dry boolean logic problem about A and Not(B), you have Problem 3.17 on page 81, which begins:
This problem examines whether the following specifications are satisfiable:
1. If the file system is not locked, then. . .
(a) new messages will be queued.
(b) new messages will be sent to the messages buffer.
(c) the system is functioning normally, and conversely, if the system is
functioning normally, then the file system is not locked.
[...]
(a) Begin by translating the five specifications into propositional
formulas using the four propositional variables [...]I was also pleased to see large deviations, although the lecture notes don’t actually define what a large deviation is.
They do give an example of a Chernoff (exponential) bound for a sum of iid random variables. The bound of course has an exponential form - they just don’t call it a large deviation. So it’s a bit of a missed opportunity, oven that the name is in the chapter title.
These bounds come up all over the place in CS, but especially lately in learning theory.
They do give an example of a Chernoff (exponential) bound for a sum of iid random variables. The bound of course has an exponential form - they just don’t call it a large deviation. So it’s a bit of a missed opportunity, oven that the name is in the chapter title.
These bounds come up all over the place in CS, but especially lately in learning theory.
[deleted]
The Units seem to be independent i.e. could be followed in any order. Can someone knowledgeable confirm this? I know Set theory etc is the basis of many things usually in a formal mathematical setting, hence asking.
Has anyone navigated a career change using OpenCourseware? I have a suspicion that the MOOC era mostly catered towards already-educated, self-starters and hobby learners, moreso than empowering a generation of workers, as was advertised.
Not to knock it. I've been working through quantum computing between work-related fire drills and household commitments, so I should be up to speed in a few decades.
Not to knock it. I've been working through quantum computing between work-related fire drills and household commitments, so I should be up to speed in a few decades.
Having "Mathematics for Computer Science" as a course title rubs me the wrong way, I always believed Computer Science was a specialized subfield of Mathematics.
In principle. But in practice, the industry doesn't need nearly as many mathematicians as it does software engineers, and almost no one is getting into CS out of the love of math. CS coursework reflects that. Here are some important algorithms and data structures, here's how you write Python, good luck at big tech!
My CS program (at Purdue) was from the math department. We didn't even start designing real programs until the 4th semester (and that was in Forth or C).
At that time, if you wanted to do application programming, you took software engineering (OO Pascal and C++) or computer technology (Java) from either tech or engineering schools.
At that time, if you wanted to do application programming, you took software engineering (OO Pascal and C++) or computer technology (Java) from either tech or engineering schools.
You could make an analogous course titled "Mathematics for [subfield of mathematics]" for any subfield of math. It would be a good(ish) title (I have never titled a course), and the content would be nicely focused.
Such courses are generally titled "Intro to".
Such courses are generally titled "Intro to".
I'm going to try formalizing this course in Lean--not sure how hard it is going to be. If anyone is interested in doing the same, please feel free to contribute!
https://github.com/dernett/Lean61200J
https://github.com/dernett/Lean61200J
This sounds very interesting and relevant to the goals of the CSLib initiative that apparently just got started. I don't have a better public link to it now except this LinkedIn post (perhaps there's a Zulip tag):
https://www.linkedin.com/posts/lean-fro_leanlang-cslib-forma...
https://www.linkedin.com/posts/lean-fro_leanlang-cslib-forma...
What will that accomplish?
You can write proofs along with the course, and since they are machine checked you can have confidence that they are correct.
If you don't know, writing a proof in isolation can be difficult, since you may be writing on that isn't actually sound.
If you don't know, writing a proof in isolation can be difficult, since you may be writing on that isn't actually sound.
Learning math is more about the big ideas. Behind each proof is an insight. Formalizing in a computer is like spell checking a document. It helps you catch small mistakes but doesn’t change the content,
I just think this is a distraction unless your goal is to learn lean and not math.
I just think this is a distraction unless your goal is to learn lean and not math.
Hard disagree.
Errors are found in human proofs all the time. And like everything else, going through the process of formalizing to a machine only increases the clarity and accuracy of what you’re doing.
Errors are found in human proofs all the time. And like everything else, going through the process of formalizing to a machine only increases the clarity and accuracy of what you’re doing.
You are correct that mistakes are made all the time - but they tend to be "oh yeah let me fix that right now" mistakes. Or, "oh yeah that's not true in general, but it still works for this case". That's because the experts are thinking about the content of the material - and they are familiar with it enough to tell if an idea has merit or not. Formalism is just a mode of presentation.
Over-emphasis on formalism leads me to conclude you just don't understand the purpose of proofs. You are primarily interested in formal logic - not math.
I would invite you to read a few pages of famous papers - for example Perelman's paper on the Poincaré Conjecture.
Over-emphasis on formalism leads me to conclude you just don't understand the purpose of proofs. You are primarily interested in formal logic - not math.
I would invite you to read a few pages of famous papers - for example Perelman's paper on the Poincaré Conjecture.
A lot of these topics sound interesting, though I think the average software engineer needs approximately none of that. When I first started programming, I was surprised how little mathematics was involved in practice.
Of course, these MIT lectures are aimed at computer scientists, not software engineers, which US universities consider to be quite different.
Of course, these MIT lectures are aimed at computer scientists, not software engineers, which US universities consider to be quite different.
> the average software engineer needs approximately none of that.
Not true. He/She doesn't need to know all of it nor in depth but a conceptual understanding is very much needed to write "correct" (w.r.t. a specification) code. We Humans are natural algorithmic problem solvers and hence can always muddle our way through to a ad-hoc solution for a given problem. Obviously, a lot depends on the intelligence of the individual here. What Mathematics gives you is a structure and concepts to systematize our thinking and rigorously apply it so that problem solving becomes more mechanical. Thus you learn to specify the problem rigorously using mathematical concepts and then use mathematical logic to derive a "Program" satisfying those requirements.
At the very least a knowledge of Set Theory, Logic and Relational Algebra goes a long way towards understanding the mapping from Mathematics to Computer Programming.
The following books are helpful here;
1) Introductory Logic and Sets for Computer Scientists by Nimal Nissanke. A very nice overview and wide coverage of needed basic mathematics.
2) Understanding Formal Methods by Jean-Francois Monin. A fire-hose overview of mathematical models and tools implementing those models.
Not true. He/She doesn't need to know all of it nor in depth but a conceptual understanding is very much needed to write "correct" (w.r.t. a specification) code. We Humans are natural algorithmic problem solvers and hence can always muddle our way through to a ad-hoc solution for a given problem. Obviously, a lot depends on the intelligence of the individual here. What Mathematics gives you is a structure and concepts to systematize our thinking and rigorously apply it so that problem solving becomes more mechanical. Thus you learn to specify the problem rigorously using mathematical concepts and then use mathematical logic to derive a "Program" satisfying those requirements.
At the very least a knowledge of Set Theory, Logic and Relational Algebra goes a long way towards understanding the mapping from Mathematics to Computer Programming.
The following books are helpful here;
1) Introductory Logic and Sets for Computer Scientists by Nimal Nissanke. A very nice overview and wide coverage of needed basic mathematics.
2) Understanding Formal Methods by Jean-Francois Monin. A fire-hose overview of mathematical models and tools implementing those models.
> At the very least a knowledge of Set Theory, Logic and Relational Algebra goes a long way towards understanding the mapping from Mathematics to Computer Programming.
I know all these from university, but I did programming and SQL before that without any issues. Learning these mathematical details seemed really not useful in practice, at least for me.
Of course, coming up with something like SQL in the first place requires a lot of theoretical mathematics (set theory, relational algebra), but as someone who only uses those systems, like 99% of software engineers, understanding the mathematical theory here doesn't seem very helpful.
I know all these from university, but I did programming and SQL before that without any issues. Learning these mathematical details seemed really not useful in practice, at least for me.
Of course, coming up with something like SQL in the first place requires a lot of theoretical mathematics (set theory, relational algebra), but as someone who only uses those systems, like 99% of software engineers, understanding the mathematical theory here doesn't seem very helpful.
I am afraid you have not really understood the mathematical theory and its mapping to programming. Relational Algebra doesn't just mean RDBMS/SQL but is a general algebra where algebraic Operations are defined over mathematical Relations i.e. over a Cartesian Product of one or more Sets.
As a first approximation; a) Type = Set b) Function = subset of Relation (i.e. set of Tuples) obtained from Cartesian Product of {input type set X output type set} c) Logical conditions define new relational sets where its members have a ordering relation d) A Program is a series of functions which prune and transform the tuples from the above cartesian product.
As a first approximation; a) Type = Set b) Function = subset of Relation (i.e. set of Tuples) obtained from Cartesian Product of {input type set X output type set} c) Logical conditions define new relational sets where its members have a ordering relation d) A Program is a series of functions which prune and transform the tuples from the above cartesian product.
Well, I'm familiar with model theory and Church's simple theory of types, but I don't think things like that are useful in practice. Perhaps the concept of currying would be an exception, if I were a Haskell programmer.
I am not sure that you have really understood the topics you have named. All high-level programming languages give you a set of fundamental types and the ability to construct user-defined types. Currying is not an exception but falls under the same model if one considers it as a Relation between "sets of functions". Also by Curry-Howard correspondence you have "formula/proposition = type" and "proof = function". So you have a direct mapping between Sets/Relations/Logic in Mathematics and Types/Logic in a Program.
A Program then becomes a trajectory enforced using predicate logic through a state space obtained from the cartesian product of all the types in the program.
You are using all of the above whether you know it or not when programming in a high-level language. The real value comes when you do it with the knowledge of the mathematics in hand because then it allows you to prove your Program as "Correct" (w.r.t. a specification).
A Program then becomes a trajectory enforced using predicate logic through a state space obtained from the cartesian product of all the types in the program.
You are using all of the above whether you know it or not when programming in a high-level language. The real value comes when you do it with the knowledge of the mathematics in hand because then it allows you to prove your Program as "Correct" (w.r.t. a specification).
> You are using all of the above whether you know it or not when programming in a high-level language.
Exactly. The average programmer doesn't have to know the math behind things like types to use them.
> The real value comes when you do it with the knowledge of the mathematics in hand because then it allows you to prove your Program as "Correct" (w.r.t. a specification).
I don't think the average software engineer does that or would benefit from doing it. I certainly don't.
Exactly. The average programmer doesn't have to know the math behind things like types to use them.
> The real value comes when you do it with the knowledge of the mathematics in hand because then it allows you to prove your Program as "Correct" (w.r.t. a specification).
I don't think the average software engineer does that or would benefit from doing it. I certainly don't.
> I don't think the average software engineer does that or would benefit from doing it. I certainly don't.
Again; you are drawing the wrong conclusions and projecting your own ignorance on others.
To give a simple analogy; anybody can swing a baseball bat at a ball. But that won't make him a notable player. To become a superlative player one needs to understand body dynamics and train scientifically. In a similar vein, anybody can muddle through and come up with a Program. But more often than not, it will be error-prone and bug-ridden not to mention hard to extend and maintain. Given the importance of Software to our modern society this is not what we want. A little bit of knowledge of Mathematics behind Programming gives you orders of magnitude return in software quality which is absolutely worthwhile.
There is nothing to debate/argue here but merely pointing out the application of scientific method to software engineering.
Again; you are drawing the wrong conclusions and projecting your own ignorance on others.
To give a simple analogy; anybody can swing a baseball bat at a ball. But that won't make him a notable player. To become a superlative player one needs to understand body dynamics and train scientifically. In a similar vein, anybody can muddle through and come up with a Program. But more often than not, it will be error-prone and bug-ridden not to mention hard to extend and maintain. Given the importance of Software to our modern society this is not what we want. A little bit of knowledge of Mathematics behind Programming gives you orders of magnitude return in software quality which is absolutely worthwhile.
There is nothing to debate/argue here but merely pointing out the application of scientific method to software engineering.
> Again; you are drawing the wrong conclusions and projecting your own ignorance on others.
I reject your accusation. It's more likely that you are the one who is projecting, namely your ignorance of what the average software engineer is doing.
I reject your accusation. It's more likely that you are the one who is projecting, namely your ignorance of what the average software engineer is doing.
Your opinion has zero basis in facts and betrays some serious ignorance of Scientific Method. No educated person can deny the importance of Mathematics to our technologically advanced society today. Computer Science is a subfield of Mathematics and Computer Programming is the application of principles and concepts studied therein.
As mentioned earlier, the point of studying Mathematics for Computer Science is to make your "average software engineer" better and more productive than he/she was in the past.
Eugene Wigner's classic essay The Unreasonable Effectiveness of Mathematics in the Natural Sciences is very relevant here - https://en.wikipedia.org/wiki/The_Unreasonable_Effectiveness...
As mentioned earlier, the point of studying Mathematics for Computer Science is to make your "average software engineer" better and more productive than he/she was in the past.
Eugene Wigner's classic essay The Unreasonable Effectiveness of Mathematics in the Natural Sciences is very relevant here - https://en.wikipedia.org/wiki/The_Unreasonable_Effectiveness...
> The real value comes when you do it with the knowledge of the mathematics in hand because then it allows you to prove your Program as "Correct" (w.r.t. a specification).
At the risk of nitpicking:
Certainly it's a benefit to structure and understand code such that you can reason about it effectively, but prove goes too far. Almost no real code is proven correct, the ergonomics of formal methods are still far too poor for that.
At the risk of nitpicking:
Certainly it's a benefit to structure and understand code such that you can reason about it effectively, but prove goes too far. Almost no real code is proven correct, the ergonomics of formal methods are still far too poor for that.
It depends; the "proving" can be done at a gross high level function or fine grained at statement level. Thus in the former case one could use Meyer's Design-by-Contract (aka DbC) while in the latter case one might choose to follow a detailed Dijkstra methodology. For both of the above you don't need any special tools (eg. Z/VDM/TLA+/Coq/Lean etc.) but merely the knowledge to learn to think about a Program using Mathematical Concepts. For most "ordinary" software, DbC would be enough while for critical software one might want to go with the whole nine yards using chosen methodologies/tools. Note that usage of the methodologies/tools themselves require a knowledge of the above-mentioned Mathematics.
The point was that a knowledge of the requisite Mathematics gives you a very powerful way of viewing Programs and then you get to choose how to map/implement it using any number of tools and based on the needs of the software.
The point was that a knowledge of the requisite Mathematics gives you a very powerful way of viewing Programs and then you get to choose how to map/implement it using any number of tools and based on the needs of the software.
Design-by-contract typically refers to runtime checking of invariants, which is not the equivalent of formal verification. It should not be referred to as proving correctness. It doesn't do so.
If you want to prove your program's correctness, that's the domain of formal methods, essentially by definition.
If you want to prove your program's correctness, that's the domain of formal methods, essentially by definition.
This argument has been made before and that is why i said it depends and put "proving" within quotes. It is a very narrow and wrong way of looking at Formal Methods (both Specification and Verification) and one of the main reasons the "ordinary" software engineer gets intimidated and overawed by formal methods and afraid to even approach the subject (as my comments to user cubefox shows).
A Formal Method defined broadly is the application of Mathematical Concepts/Models and Logic to the specification/implementation/verification of Computer Programs.
DbC is based on Hoare Logic where you try to "prove" the program intellectually and not necessarily tool driven. This was somewhat mechanized (though still a intellectual pursuit) by Dijkstra's technique of deriving weakest-preconditions. Because the whole process can be quite tedious, laborious and rather complex, automatic theorem prover tools were invented to do the job for you. This has resulted in the unfortunate status quo where Formal Specification/Verification have become identified with the use of tools and not the Mathematics behind them. Proving Correctness need not be fine-grained and absolute but can be good enough done partially at a gross level at specific stages in a program as needed and done either by hand and/or using tools.
DbC can be considered a lightweight Formal Method with aspects of both Specification and Verification included. The Programmer plays the role of "theorem prover" when using DbC. However tools exist to map "Design-by-Contract" constructs to "Verified Design-by-Contract" constructs. See for example https://www.eschertech.com/products/verified_dbc.php. Also Eiffel (quintessential DbC language) can be converted using AutoProof to a form which can be proven by Boogie verifier.
Finally; here is a case study Design by contract formal verification for automotive embedded software robustness (abstract & pdf) - https://hal.sorbonne-universite.fr/ERTS2024/hal-05028176v1
A Formal Method defined broadly is the application of Mathematical Concepts/Models and Logic to the specification/implementation/verification of Computer Programs.
DbC is based on Hoare Logic where you try to "prove" the program intellectually and not necessarily tool driven. This was somewhat mechanized (though still a intellectual pursuit) by Dijkstra's technique of deriving weakest-preconditions. Because the whole process can be quite tedious, laborious and rather complex, automatic theorem prover tools were invented to do the job for you. This has resulted in the unfortunate status quo where Formal Specification/Verification have become identified with the use of tools and not the Mathematics behind them. Proving Correctness need not be fine-grained and absolute but can be good enough done partially at a gross level at specific stages in a program as needed and done either by hand and/or using tools.
DbC can be considered a lightweight Formal Method with aspects of both Specification and Verification included. The Programmer plays the role of "theorem prover" when using DbC. However tools exist to map "Design-by-Contract" constructs to "Verified Design-by-Contract" constructs. See for example https://www.eschertech.com/products/verified_dbc.php. Also Eiffel (quintessential DbC language) can be converted using AutoProof to a form which can be proven by Boogie verifier.
Finally; here is a case study Design by contract formal verification for automotive embedded software robustness (abstract & pdf) - https://hal.sorbonne-universite.fr/ERTS2024/hal-05028176v1
> that is why i said it depends and put "proving" within quotes
I feel I need to insist on this point: a design-by-contract methodology using runtime checking may be an effective means of improving software quality, but it certainly isn't proving the correctness of a program. Using the word 'prove' here is just plain wrong. Putting it in quotation marks doesn't help. It simply does not constitute a proof.
> It is a very narrow and wrong way of looking at Formal Methods (both Specification and Verification) and one of the main reasons the "ordinary" software engineer gets intimidated and overawed by formal methods and afraid to even approach the subject
I'm not insisting on a narrow understanding of formal methods, I'm insisting on proper use of established terminology.
I'm all for practical methodologies that benefit software quality without paying the enormous costs of fully applying formal methods. Design-by-contract with runtime checking seems like a reasonable approach. (Static typing is another. Effect-oriented programming might be another.) My point was just about the proper use of the word proof.
> Proving Correctness need not be fine-grained and absolute but can be good enough done partially at a gross level at specific stages in a program as needed and done either by hand and/or using tools.
A program is proven correct only when the proof is absolute, which implies fine-grained. I'm not opposed to the use of partial proofs or 'good enough' proof-sketches, both of which could be useful in producing high-quality software in the real world, but we should be clear in how we refer to them.
I'm not sure about the idea of proving correctness by hand, though. Manually applying a formal model of a real programming language sounds unworkable. If code is well structured, it should be possible for the programmer to reason about it somewhat precisely, like a proof-sketch, but that's not a proof of correctness.
> The Programmer plays the role of "theorem prover" when using DbC
I don't find this convincing. There's nothing stopping the programmer from failing to notice some error. If you aren't actually doing a proof, just say so. There is no stand-in.
> tools exist to map "Design-by-Contract" constructs to "Verified Design-by-Contract" constructs.
Right, I hinted at this in my previous comment. SPARK Ada does this. In that case, correctness properties of the program are indeed formally proven. That's a quite different methodology to design-by-contract with runtime checking though, to the point that it almost seems unhelpful to refer to them both by the same name. I've had to be careful to explicitly state design-by-contract using runtime checking this whole time.
The deeper distinction here is between testing and formal proof, and design-by-contract can in principle refer to either.
> here is a case study Design by contract formal verification for automotive embedded software robustness
Have only skim-read but seems somewhat similar to this case-study about SPARK: https://www.adacore.com/tokeneer
I feel I need to insist on this point: a design-by-contract methodology using runtime checking may be an effective means of improving software quality, but it certainly isn't proving the correctness of a program. Using the word 'prove' here is just plain wrong. Putting it in quotation marks doesn't help. It simply does not constitute a proof.
> It is a very narrow and wrong way of looking at Formal Methods (both Specification and Verification) and one of the main reasons the "ordinary" software engineer gets intimidated and overawed by formal methods and afraid to even approach the subject
I'm not insisting on a narrow understanding of formal methods, I'm insisting on proper use of established terminology.
I'm all for practical methodologies that benefit software quality without paying the enormous costs of fully applying formal methods. Design-by-contract with runtime checking seems like a reasonable approach. (Static typing is another. Effect-oriented programming might be another.) My point was just about the proper use of the word proof.
> Proving Correctness need not be fine-grained and absolute but can be good enough done partially at a gross level at specific stages in a program as needed and done either by hand and/or using tools.
A program is proven correct only when the proof is absolute, which implies fine-grained. I'm not opposed to the use of partial proofs or 'good enough' proof-sketches, both of which could be useful in producing high-quality software in the real world, but we should be clear in how we refer to them.
I'm not sure about the idea of proving correctness by hand, though. Manually applying a formal model of a real programming language sounds unworkable. If code is well structured, it should be possible for the programmer to reason about it somewhat precisely, like a proof-sketch, but that's not a proof of correctness.
> The Programmer plays the role of "theorem prover" when using DbC
I don't find this convincing. There's nothing stopping the programmer from failing to notice some error. If you aren't actually doing a proof, just say so. There is no stand-in.
> tools exist to map "Design-by-Contract" constructs to "Verified Design-by-Contract" constructs.
Right, I hinted at this in my previous comment. SPARK Ada does this. In that case, correctness properties of the program are indeed formally proven. That's a quite different methodology to design-by-contract with runtime checking though, to the point that it almost seems unhelpful to refer to them both by the same name. I've had to be careful to explicitly state design-by-contract using runtime checking this whole time.
The deeper distinction here is between testing and formal proof, and design-by-contract can in principle refer to either.
> here is a case study Design by contract formal verification for automotive embedded software robustness
Have only skim-read but seems somewhat similar to this case-study about SPARK: https://www.adacore.com/tokeneer
I have used SPARK Ada myself on a couple of security focused products developed by a team of 10-15 developers. For a long time, SPARK has made use of formal methods practical for real-world software. Effective use does require learning and commitment, but that's within reason. By the time the projects were nearly done, I felt that the SPARK findings always turned out to be correct and that any remaining problems could be traced back to poor or missing requirements, not the implementation itself.
There is a nuanced but distinct difference in my use of the word Proof as used in Program as a Proof and Proof in a Mathematical Algebraic System which you have missed. They are isomorphic but not exact (hence my using the phrase it depends and scare-quotes around "proving").
The reason is because Mathematics deals with ideal and abstract objects whereas objects in the real world (eg. a computer program) can only map to aspects of the ideal world and not in its entirety.
To elaborate; a mathematical algebraic system is a set of objects and a set of operations defined on them. Axioms using those objects/operations are then defined and then inference/reasoning rules using these are defined to prove theorems in the system. There are various techniques for constructing proofs (eg. direct, induction, contradiction etc.) but all of them must map back to the domain of definition of the objects in the algebra to be considered valid and sound in the real world.
As an example, the axiom of associativity w.r.t. addition holds absolutely in mathematics when applied to {N, +} where N is the infinite set of ideal integers. But in computing it is not always the case i.e. (a+b)+c =/= a+(b+c) always because a/b/c are constrained/partial finite sets (i.e. int8/int16/int32 etc.) and thus the axiom of associativity will fail when for example, we take boundary values for a/b and a negative value for c (due to overflow/underflow). Thus any proof which uses the axiom of associativity for signed integers in a computer can never be as absolute and general as its counterpart in pure mathematics i.e. everything is Partial. In general, mathematics uses exact Analytical Techniques while computers use approximate Numerical Techniques to solve a problem which is reflected in the nature of their proofs.
Coming to DbC, since it is based on Hoare Logic (i.e. an algebra with axioms/inference rules), a Program is written as a series of Preconditions/Postconditions/Invariants with the Programmer acting as the Proof deriver. Thus if a series of pre/post/inv holds at a certain stage in the program (i.e. proof) the next post will hold (barring external cataclysms). The fact that the proof obligation is discharged dynamically at runtime is immaterial. But we may not want that in certain categories of real world programs since the question of what to do when the proof obligation is not met at runtime becomes a problem; Do we abort/Do we rollback to older known state etc. For a CRUD app we can abort and have the user restart but for a heart pacemaker app we don't want that. In the latter case since it is a closed system with well defined inputs/outputs we can map DbC to VDbC and then prove it through a verifier statically thus guaranteeing invalid states can never arise at runtime. But note that this is merely an incidental distinction due to the needs of the real world but the essential DbC guarantees remain the same.
It should now be clear that when you map concepts from mathematical to computing domain you need to understand how the same names like "Integer", "Set/Type", "Algebra", "Axiom", "Proof" map from one to the other (though not exactly) and how you can leverage their isomorphism to use symbolic Mathematics effectively in Programming while at the same time keeping in mind their differences due to real world computation constraints and limits.
References:
1) Mathematical Proof - https://en.wikipedia.org/wiki/Mathematical_proof
2) Computer-assisted Proof - https://en.wikipedia.org/wiki/Computer-assisted_proof In particular; see the "Philosophical Objections" section.
3) See also the book From Mathematics to Generic Programming by Alexander Stepanov and Daniel Rose to get an idea of how to map between mathematics and programming.
The reason is because Mathematics deals with ideal and abstract objects whereas objects in the real world (eg. a computer program) can only map to aspects of the ideal world and not in its entirety.
To elaborate; a mathematical algebraic system is a set of objects and a set of operations defined on them. Axioms using those objects/operations are then defined and then inference/reasoning rules using these are defined to prove theorems in the system. There are various techniques for constructing proofs (eg. direct, induction, contradiction etc.) but all of them must map back to the domain of definition of the objects in the algebra to be considered valid and sound in the real world.
As an example, the axiom of associativity w.r.t. addition holds absolutely in mathematics when applied to {N, +} where N is the infinite set of ideal integers. But in computing it is not always the case i.e. (a+b)+c =/= a+(b+c) always because a/b/c are constrained/partial finite sets (i.e. int8/int16/int32 etc.) and thus the axiom of associativity will fail when for example, we take boundary values for a/b and a negative value for c (due to overflow/underflow). Thus any proof which uses the axiom of associativity for signed integers in a computer can never be as absolute and general as its counterpart in pure mathematics i.e. everything is Partial. In general, mathematics uses exact Analytical Techniques while computers use approximate Numerical Techniques to solve a problem which is reflected in the nature of their proofs.
Coming to DbC, since it is based on Hoare Logic (i.e. an algebra with axioms/inference rules), a Program is written as a series of Preconditions/Postconditions/Invariants with the Programmer acting as the Proof deriver. Thus if a series of pre/post/inv holds at a certain stage in the program (i.e. proof) the next post will hold (barring external cataclysms). The fact that the proof obligation is discharged dynamically at runtime is immaterial. But we may not want that in certain categories of real world programs since the question of what to do when the proof obligation is not met at runtime becomes a problem; Do we abort/Do we rollback to older known state etc. For a CRUD app we can abort and have the user restart but for a heart pacemaker app we don't want that. In the latter case since it is a closed system with well defined inputs/outputs we can map DbC to VDbC and then prove it through a verifier statically thus guaranteeing invalid states can never arise at runtime. But note that this is merely an incidental distinction due to the needs of the real world but the essential DbC guarantees remain the same.
It should now be clear that when you map concepts from mathematical to computing domain you need to understand how the same names like "Integer", "Set/Type", "Algebra", "Axiom", "Proof" map from one to the other (though not exactly) and how you can leverage their isomorphism to use symbolic Mathematics effectively in Programming while at the same time keeping in mind their differences due to real world computation constraints and limits.
References:
1) Mathematical Proof - https://en.wikipedia.org/wiki/Mathematical_proof
2) Computer-assisted Proof - https://en.wikipedia.org/wiki/Computer-assisted_proof In particular; see the "Philosophical Objections" section.
3) See also the book From Mathematics to Generic Programming by Alexander Stepanov and Daniel Rose to get an idea of how to map between mathematics and programming.
> the average software engineer needs approximately none of that.
This is not really true, especially if you're involved with physics and robotics even just a bit like a do. Without mathematics, you won't understand a thing.
This is not really true, especially if you're involved with physics and robotics even just a bit like a do. Without mathematics, you won't understand a thing.
But the average software engineer probably doesn't do physics simulations or things like that.
I have been a software engineer with and without math knowledge and it’s a different level of contribution and effectiveness.
Which areas of math were the most applicable in practice?
Linear algebra, numerical analysis, and combinatorics are my most commonly used techniques.
But It’s less about the particular field and more about the mindset for thinking about problems. That’s kind of like asking what your most used library functions are.
But It’s less about the particular field and more about the mindset for thinking about problems. That’s kind of like asking what your most used library functions are.
Interesting. I have studied computer science after working as a software engineer for several years, but I didn't become a better software engineer than I was before. And I have zero need for linear algebra, numerical analysis, or combinatorics. May I ask what you are working on? It sounds pretty advanced.
It’s really not advanced. I see more applications than time.
Being able to determine when a mathematical framing is useful and apply it is harder than knowing how to do the math. It requires deeper internalization of the concepts. So your experience is common.
Being able to determine when a mathematical framing is useful and apply it is harder than knowing how to do the math. It requires deeper internalization of the concepts. So your experience is common.
Or it may also be common because there is genuinely little need for higher mathematics for most SWEs.
Need is a strong word. That's why I said effective.
You can often iterate to something that kind of works by adding epicycles. What you're left with is something that fails in rare cases you can't explain, is difficult to change (we don't touch that code), and is slow.
Compare a complex homegrown data store to a relational database like postgresSQL. Both get the job done, but one has significantly more conceptual clarity and reliability.
Being able to come across a hard problem and say, ok this is how to frame it and here is how to go about it, turns months of fiddling into a direct route.
You can often iterate to something that kind of works by adding epicycles. What you're left with is something that fails in rare cases you can't explain, is difficult to change (we don't touch that code), and is slow.
Compare a complex homegrown data store to a relational database like postgresSQL. Both get the job done, but one has significantly more conceptual clarity and reliability.
Being able to come across a hard problem and say, ok this is how to frame it and here is how to go about it, turns months of fiddling into a direct route.
> Compare a complex homegrown data store to a relational database like postgresSQL. Both get the job done, but one has significantly more conceptual clarity and reliability.
That's a good example of what I said. Most software engineers don't develop new database management systems. They just use one. And if they merely use it, they don't need or benefit from nontrivial math. The math is abstracted away behind the intuitive SQL syntax.
That's a good example of what I said. Most software engineers don't develop new database management systems. They just use one. And if they merely use it, they don't need or benefit from nontrivial math. The math is abstracted away behind the intuitive SQL syntax.
So did postgres appear out of thin air? What new SW technologies need to be developed?
If you're not interested in acquiring the kind of competency to develop new software, and are just interested in combining existing software, then you don't need a college degree.
If you're not interested in acquiring the kind of competency to develop new software, and are just interested in combining existing software, then you don't need a college degree.
Just because you don't doesn't mean others don't. Software engineering is nothing without computer science.
The first topic is "Predicates, Sets, and Proofs."
I use predicates and sets quite often in daily programming.
I use predicates and sets quite often in daily programming.
I wish these courses would also provide the answer sheets or tell you where to find them. How am I supposed to check my work and verify my answers otherwise?
I hate to say it but a LLM provides at least some guidance, but you can always try to ask on MathExchange
Where can I find solutions to the problems in the course?
Wait... what? CS is a math degree. Title is like saying mathematics for math.
[deleted]
https://ocw.mit.edu/courses/6-1200j-mathematics-for-computer...
https://www.youtube.com/playlist?list=PLUl4u3cNGP61VNvICqk2H...