Math That Matters: The Case for Probability over Polynomials(anandsanwal.me)
anandsanwal.me
Math That Matters: The Case for Probability over Polynomials
https://anandsanwal.me/math-eduction-more-probability-statistics-less-calculus/
36 comments
Some background might be useful.
In the 1800s, geometry was seen as the best math to train one's mind. Abraham Lincoln, for example, "kept a copy of Euclid in his saddlebag, and studied it late at night by lamplight; he related that he said to himself, "You never can make a lawyer if you do not understand what demonstrate means; and I left my situation in Springfield, went home to my father's house, and stayed there till I could give any proposition in the six books of Euclid at sight"." (quoting https://en.wikipedia.org/wiki/Euclid%27s_Elements ).
This held until around WWII. One way to see how curriculum has changed over the last 75 years is in Sheldon Glashow's autobiography. He graduated from the Bronx High School of Science in 1950. Quoting https://www.nobelprize.org/prizes/physics/1979/glashow/biogr... , "High-school mathematics then terminated with solid geometry."
In the post-war era, the government needed a lot of people trained in the physical sciences and engineering fields where applied skills in calculus was more important than abstract ideas of how to think.
In the last 25 or so years there's been an effort to de-emphasize calculus as the end-goal progression for high school math, and to teach more statistics. AP Statistics wasn't offered until 1997, for example: https://en.wikipedia.org/wiki/AP_Statistics
This essay is part of that modern movement.
That said, I think the author has almost dismissed the classic idea of a mathematical education - to learn logical thinking. The only mention of "proof" is in reference to Gelman's "restructuring mathematical education around understanding uncertainty rather than abstract proofs."
I know nothing of Gelman's writing, but as the Lincoln example shows, abstract proofs are useful in constructing and analyzing legal arguments (and programming, IMO), in way I think should not be so readily set aside.
In the 1800s, geometry was seen as the best math to train one's mind. Abraham Lincoln, for example, "kept a copy of Euclid in his saddlebag, and studied it late at night by lamplight; he related that he said to himself, "You never can make a lawyer if you do not understand what demonstrate means; and I left my situation in Springfield, went home to my father's house, and stayed there till I could give any proposition in the six books of Euclid at sight"." (quoting https://en.wikipedia.org/wiki/Euclid%27s_Elements ).
This held until around WWII. One way to see how curriculum has changed over the last 75 years is in Sheldon Glashow's autobiography. He graduated from the Bronx High School of Science in 1950. Quoting https://www.nobelprize.org/prizes/physics/1979/glashow/biogr... , "High-school mathematics then terminated with solid geometry."
In the post-war era, the government needed a lot of people trained in the physical sciences and engineering fields where applied skills in calculus was more important than abstract ideas of how to think.
In the last 25 or so years there's been an effort to de-emphasize calculus as the end-goal progression for high school math, and to teach more statistics. AP Statistics wasn't offered until 1997, for example: https://en.wikipedia.org/wiki/AP_Statistics
This essay is part of that modern movement.
That said, I think the author has almost dismissed the classic idea of a mathematical education - to learn logical thinking. The only mention of "proof" is in reference to Gelman's "restructuring mathematical education around understanding uncertainty rather than abstract proofs."
I know nothing of Gelman's writing, but as the Lincoln example shows, abstract proofs are useful in constructing and analyzing legal arguments (and programming, IMO), in way I think should not be so readily set aside.
Law seems an odd application of proofs. I've never seen anything in law to which mathematical rigor could apply.
Learning to reason is always good as a brain exercise. But legal "proof" has always struck me as laughably vague.
Learning to reason is always good as a brain exercise. But legal "proof" has always struck me as laughably vague.
As someone with degrees in both law and mathematics I was struck by the similarities in the subjects when I learn law. Everyone was surprised when I told them that... except for my cousin, a mathematician. It didn't surprise him at all.
Law (analytically - practically it is about problem solving, strategy, tactics, understanding your client's problems, clear presentation of ideas, etc. like any other business) is all about logic, reasoning, thinking clearly, separating precedent and antecedent, the logical flow of ideas, etc.
In practice, legal reasoning is more inductive than deductive (not entirely), which means if has a different flavour in practice. It is still just as much about logic.
But don't get distracted by things like the standard of proof etc. That has much more to do with the fact that you are making decisions based on unreliable and imperfect information, as opposed to reasoning about entirely abstract ideas like a mathematician.
Law (analytically - practically it is about problem solving, strategy, tactics, understanding your client's problems, clear presentation of ideas, etc. like any other business) is all about logic, reasoning, thinking clearly, separating precedent and antecedent, the logical flow of ideas, etc.
In practice, legal reasoning is more inductive than deductive (not entirely), which means if has a different flavour in practice. It is still just as much about logic.
But don't get distracted by things like the standard of proof etc. That has much more to do with the fact that you are making decisions based on unreliable and imperfect information, as opposed to reasoning about entirely abstract ideas like a mathematician.
Base rate fallacy is a key part of many erroneous human decisions or judgements. This could easily be taught in high schools:
https://en.wikipedia.org/wiki/Base_rate_fallacy
https://en.wikipedia.org/wiki/Base_rate_fallacy
For a moment there I was excited because I though the article would be about an extension of probability theory to use a polynomial basis instead of real numbers, with an analogy to how Quantum Mechanics is essentially just probability with complex numbers.
Hmm... this gives me an idea.
Hmm... this gives me an idea.
> Quantum Mechanics is essentially just probability with complex numbers
For some more on that topic:
https://arxiv.org/abs/quant-ph/0101012
https://www.scottaaronson.com/democritus/lec9.html
For some more on that topic:
https://arxiv.org/abs/quant-ph/0101012
https://www.scottaaronson.com/democritus/lec9.html
Polynomials had always struck me as begging for the question "when are we ever going to use this"? It eventually makes sense when you get to linear algebra and calculus, but for students not on that track, it seems designed to make them hate math.
You use them at the same time you learn them in physics, no? Also, you sometimes need to learn things on a bit of faith it will eventually be useful. This stuff is so fundamental that it is difficult to motivate it except to say: you will use it so pervasively if you do any technical, scientific, mathematical or computational subject that it will seem like asking "when are we ever going to use this 'algebra' thing?"
I never had much motivation for much of maths presented to me and I never hated it. In fact, I got bored when teachers tried to justify what they were teaching with boring physics examples etc.
I think the sentiment you are talking about has a risk of going too far and making teachers afraid that if they teach something without justifying it to their class that they will lose their attention immediately and justifiably slow. It risks normalising the idea that teachers aren't entitled to the attention and hard work of their classes unless they give the kids a good reason to pay attention: you have to make it relatable, relevant to them, or it is your fault when they don't pay attention.
What ever happened to: you are a child, you are at school to learn, you learn what you are taught and that is that? Obviously sometimes it is good to motivate abstract concepts with concrete examples but if you take it too far you'll end up teaching calculus before you teach algebra because you are trying to justify the algebra! And you'll need to teach differential equations before you teach calculus because you need to motivate calculus. And what is the point of differential equations anyway? Before long you're teaching undergraduate physics to 11 year olds.
I never had much motivation for much of maths presented to me and I never hated it. In fact, I got bored when teachers tried to justify what they were teaching with boring physics examples etc.
I think the sentiment you are talking about has a risk of going too far and making teachers afraid that if they teach something without justifying it to their class that they will lose their attention immediately and justifiably slow. It risks normalising the idea that teachers aren't entitled to the attention and hard work of their classes unless they give the kids a good reason to pay attention: you have to make it relatable, relevant to them, or it is your fault when they don't pay attention.
What ever happened to: you are a child, you are at school to learn, you learn what you are taught and that is that? Obviously sometimes it is good to motivate abstract concepts with concrete examples but if you take it too far you'll end up teaching calculus before you teach algebra because you are trying to justify the algebra! And you'll need to teach differential equations before you teach calculus because you need to motivate calculus. And what is the point of differential equations anyway? Before long you're teaching undergraduate physics to 11 year olds.
Not everybody is going to take physics, either. It's necessary for students on a science track, but only a fraction of students are going to take science.
Even the polynomials we do use are very limited, until much later. We don't factor polynomials; we don't compute residues. They're just formulas with numbers in and numbers out. We may solve for various parts, but nearly always just for a single variable of power 1.
Even the polynomials we do use are very limited, until much later. We don't factor polynomials; we don't compute residues. They're just formulas with numbers in and numbers out. We may solve for various parts, but nearly always just for a single variable of power 1.
You need to take some physics in high school, and IIRC you do enough that you encounter equations of motion such as f=ma and E=(1/2)mv^2 etc.
We do factor polynomials: we do it in year 10/11 at least in NZ, and our maths education is very poor compared to places like Singapore and Finland, as far as I know.
We do factor polynomials: we do it in year 10/11 at least in NZ, and our maths education is very poor compared to places like Singapore and Finland, as far as I know.
The path is polynomials -> matricies -> multivariate statistics. And multivariate statistics are actually the natural home of probability, although it requires too much setup for most people to learn anything about it.
polynomials are generalizations of number. you use them to solve equations just like you use all numbers. polynomials add the notion of an "indeterminate"
in the same way that you use integers to solve equations like 3 + x = 2 (x = -1), or rationals to solve 3x = 2 (x = 2/3). you use polynomials to model equations like x^2 = x + 1 (x = phi). or to perform polynomial division. or to do algebraic geometry (x^2 + y^2 - 1 = 0 => unit circle)
in the same way that you use integers to solve equations like 3 + x = 2 (x = -1), or rationals to solve 3x = 2 (x = 2/3). you use polynomials to model equations like x^2 = x + 1 (x = phi). or to perform polynomial division. or to do algebraic geometry (x^2 + y^2 - 1 = 0 => unit circle)
> One who grasps statistical significance can better interpret health studies ...
Ok, that's where I draw the line. Statistical significance is bullshit. Learning about it is as useful as learning about phlogiston. Its biggest failure is that it gives a false sense of security. It's what lead us to the reproducibility crisis. This should not have a place in an article that purports to emphasize the importance of statistics in the school curriculum.
Ok, that's where I draw the line. Statistical significance is bullshit. Learning about it is as useful as learning about phlogiston. Its biggest failure is that it gives a false sense of security. It's what lead us to the reproducibility crisis. This should not have a place in an article that purports to emphasize the importance of statistics in the school curriculum.
Unfortunately we will not get rid of statistical significance anytime soon. Teaching it and explaining its shortcomings is much better than sweeping it under the rug, and perhaps it will help the next generations of scientists not falling into the same trap.
>Statistical significance is bullshit. Learning about it is as useful as learning about phlogiston.
The term-of-art "statistical significance" isn't bs because we ultimately have to choose what to pay attention to. Removing "statistical significance" from the vocabulary doesn't change the underlying reality about people deciding what to do based on a threshold of a number.
The observed effect of <something> is either:
(1) appearance of cause & effect is actually not there and just random chance or coincidence
(2) a real cause & effect and not random chance
Yes.. but "statistical significance" leads to "p-hacking" and other academic fraud! (https://en.wikipedia.org/wiki/Data_dredging)
That's true but the abuses still doesn't eliminate the need to name the concept of real cause & effect vs random chance. Whatever alternative mental framework one uses to decide which of those scenarios is happening, you will arrive at something that looks like "statistical significance" even if you don't call it that.
As an analogy, even though "averages" in math is misused, it doesn't mean "averages are bs".
The term-of-art "statistical significance" isn't bs because we ultimately have to choose what to pay attention to. Removing "statistical significance" from the vocabulary doesn't change the underlying reality about people deciding what to do based on a threshold of a number.
The observed effect of <something> is either:
(1) appearance of cause & effect is actually not there and just random chance or coincidence
(2) a real cause & effect and not random chance
Yes.. but "statistical significance" leads to "p-hacking" and other academic fraud! (https://en.wikipedia.org/wiki/Data_dredging)
That's true but the abuses still doesn't eliminate the need to name the concept of real cause & effect vs random chance. Whatever alternative mental framework one uses to decide which of those scenarios is happening, you will arrive at something that looks like "statistical significance" even if you don't call it that.
As an analogy, even though "averages" in math is misused, it doesn't mean "averages are bs".
Sorry, but statistical significance has a pretty narrow meaning, and it's wrong.
> Removing "statistical significance" from the vocabulary doesn't change the underlying reality about people deciding what to do based on a threshold of a number.
I'm not arguing to remove it from the vocabulary. I'm arguing that that bloomin' threshold is the source of many problems.
The article also isn't about methodology, it's about the secondary school curriculum. To provide pupils with a false understanding of statistics is just bad. And to teach them why it's bad, is hard.
> Removing "statistical significance" from the vocabulary doesn't change the underlying reality about people deciding what to do based on a threshold of a number.
I'm not arguing to remove it from the vocabulary. I'm arguing that that bloomin' threshold is the source of many problems.
The article also isn't about methodology, it's about the secondary school curriculum. To provide pupils with a false understanding of statistics is just bad. And to teach them why it's bad, is hard.
Any chance you could expand on this, or point me to resources that do? It’s a very interesting take and not one I’m familiar with.
There's a lot of technical stuff, but you could look into Ioanides articles [1], and the book Bernoulli's Falacy by Clayton [2] is a good introduction, also for people less involved in statistics.
[1] https://medium.com/@d.incecushman/the-p-value-problem-how-dr... (this is just an example; I haven't read it)
[2] https://cup.columbia.edu/book/bernoullis-fallacy/97802311999...
[1] https://medium.com/@d.incecushman/the-p-value-problem-how-dr... (this is just an example; I haven't read it)
[2] https://cup.columbia.edu/book/bernoullis-fallacy/97802311999...
Search for "p-hacking", that will give you a nice starter on the topic.
> Statistical significance is bullshit. Learning about it is as useful as learning about phlogiston.
Ok, that's where I draw the line - statistical significance is not "bullshit" - however as you say, leaning on it too hard can cause things to break quite badly. Scientists misusing it do not negate all the medical advances we have made from moving to a significance-based system. It is an absolutely essential tool for people using statistics to understand, but its limitations must be emphasised when taught, and it must be understood that it is a tool, not a conclusion. Also other alternatives should be taught more widely (e.g. Bayesian inference).
Ok, that's where I draw the line - statistical significance is not "bullshit" - however as you say, leaning on it too hard can cause things to break quite badly. Scientists misusing it do not negate all the medical advances we have made from moving to a significance-based system. It is an absolutely essential tool for people using statistics to understand, but its limitations must be emphasised when taught, and it must be understood that it is a tool, not a conclusion. Also other alternatives should be taught more widely (e.g. Bayesian inference).
We would have made better progress if we had skipped NHST. Mind you, nor Fisher, nor Pearson were in favor of it. It is a tool that should be taught after better ways, and then only to understand the past. Like phlogiston.
You seem to be arguing that a grasp of statistical significance helps you interpret health studies.
No no no. You couldn't have gotten this more wrong. The replication crisis comes from soft sciences under-powered statistical training.
I don't even know what you mean by "under-powered statistical training."
But about the harder sciences: when this ball got rolling, I attended a lecture of a statistician, who explained that basically all genetic results preceding were likely wrong, unless they showed something like 6 sigma significance. That's because H0 is so easy to reject when you base H0 and H1 on different measurements. The result is that every theory is true.
But about the harder sciences: when this ball got rolling, I attended a lecture of a statistician, who explained that basically all genetic results preceding were likely wrong, unless they showed something like 6 sigma significance. That's because H0 is so easy to reject when you base H0 and H1 on different measurements. The result is that every theory is true.
Many scientists in non-mathematical fields are wrote taught "how to write a paper" in undergrad. P-values and statistical significance are taught completely devoid of context, essentially just a step in your final analysis. Many scientists perpetrating p-hacking or data dredging thought this was a process of good science, and didn't understand the axioms for which these metrics depend. This is something learning fixes, and ignorance makes worse.
It comes from the fact that soft sciences deal with vast numbers of variables, far too many to control. They actually get a ton of statistics, trying to find a signal in so much noise.
It would be great if human beings were more amenable to rigorous experiment. Failing that, we at least need to understand what these things do and don't actually mean. It's either that or give up trying to study people entirely.
It would be great if human beings were more amenable to rigorous experiment. Failing that, we at least need to understand what these things do and don't actually mean. It's either that or give up trying to study people entirely.
Statistics are not values you collect - but the analysis you perform. In data with many confounding variables or many degrees of freedom, the only way to be honest about what the data shows is through statistics. Statistics is what allows you to filter signal from noise. Statistical significance is a very useful tool, you just have to be honest about what it represents and under what conditions this breaks down.
A statistic is an aggregate number, not an analysis in its own. There may be some linguistic confusion here.
> Statistical significance is a very useful tool
What for? It answers a question in which nobody is interested. What can it do that Bayesian tools can't?
> Statistical significance is a very useful tool
What for? It answers a question in which nobody is interested. What can it do that Bayesian tools can't?
No it doesn’t. Common meme around here but the hard sciences are also rife with unreproducible studies. It’s an incentives problem.
I actually agree - it's part incentives. And it is important for hard sciences to learn these lessons as well.
[deleted]
Interesting idea but math educators put a LOT of thought into what should be taught and how to help students understand. If you don't understand polynomials then you're going to be extremely limited in terms of what kinds of probability problems you can solve. Calculus is also used in statistics very heavily. "I will NEVER need to do that!" you say? Well, how sure can you be, and how many people say that and change their minds? Teachers can't just take everything at face value. They try to educate you up to a level that won't leave you in a bad spot if you want to continue some day. Learning about polynomial factoring never hurt anyone. There are lots of basic problems you can't solve without knowing how to do that stuff. You can self-study less difficult things on your own, as well.
Are polynomials not used in probability?
[deleted]
He was quite surprised, and we discussed what I was doing at length. He asked me what I wanted as a career, and I told him that I already had a video game studio, products in the market, and simply wanted to get better, maybe pursue that new field of 3D computer graphics. (This was '83). My professor completely changed his course and nature of his homework, it was still Calculus 1 and 2, but he was using video game situations and simulations and illustrating the duel nature of solving with statistics or with calculus and when one is the choice over the other. That class, that single professor's insight to modify his course set me on a track that led to a career many dream about, some disbelieve when I relate it, my career has been stellar.