What does the "mean" really mean?(arxiv.org)
arxiv.org
What does the "mean" really mean?
https://arxiv.org/abs/2003.01973
38 comments
In many systems, the mean really does summarize the whole. E.g. kinematics works just fine using the mean point mass for solids (i.e. the centre-of-gravity) in place of the whole.
> In many systems, the mean really does summarize the whole
That's quite true, and the same statement also points to where the arithmetic mean isn't a good choice: where the goal isn't so much to summarize a whole as to give an insight into populations. For instance, if you are trying to figure out the "average" income in a city, the median provides better intuition than the mean.
Furthermore, there are many situations where both the mean and median (and even the mode) fail to provide insight. For instance, the average person has one testicle, whether defined by mean or median (and even the mode is not much help here, with its equally useless value of 0). We should really think of summary stats as something to only be used when a distribution is well understood. In other cases, a simple frequency distribution diagram provides much better insight, and isn't very hard to do.
That's quite true, and the same statement also points to where the arithmetic mean isn't a good choice: where the goal isn't so much to summarize a whole as to give an insight into populations. For instance, if you are trying to figure out the "average" income in a city, the median provides better intuition than the mean.
Furthermore, there are many situations where both the mean and median (and even the mode) fail to provide insight. For instance, the average person has one testicle, whether defined by mean or median (and even the mode is not much help here, with its equally useless value of 0). We should really think of summary stats as something to only be used when a distribution is well understood. In other cases, a simple frequency distribution diagram provides much better insight, and isn't very hard to do.
> For instance, the average person has one testicle, whether defined by mean or median
There are two options:
1) The ratio between women and men is not exactly 1. In this case, the median number of testicles is not 1 (it is either 0, if there are more women than men, or 2 if there are more men than women).
2) The ratio between women and men is exactly 1. This is not true in real life, but let's pretend it is. In this case, any number between 0 and 2 is a median (since any number between 0 and 2 minimizes the sum of absolute deviations), and not just 1.
TL;DR: The median number of testicles is almost surely not 1 (or, at least, not uniquely so).
There are two options:
1) The ratio between women and men is not exactly 1. In this case, the median number of testicles is not 1 (it is either 0, if there are more women than men, or 2 if there are more men than women).
2) The ratio between women and men is exactly 1. This is not true in real life, but let's pretend it is. In this case, any number between 0 and 2 is a median (since any number between 0 and 2 minimizes the sum of absolute deviations), and not just 1.
TL;DR: The median number of testicles is almost surely not 1 (or, at least, not uniquely so).
The median is specifically the mid point of the data set, so it won't be "any number between 0 and 2", it will be one specific number depending on the data set.
If you look at ten people with 2, ten people with 0, and one person with 1, then the median will be 1.
If you make it twelve people with 2, still ten with 0 and one with 1, suddenly the median is 2.
Across an entire population, I would hazard a guess the median would be 0 or 2 unless the difference in numbers of each sex is small enough that people with 0<X<2 testicles might fall right in the middle of the distribution.
If you look at ten people with 2, ten people with 0, and one person with 1, then the median will be 1.
If you make it twelve people with 2, still ten with 0 and one with 1, suddenly the median is 2.
Across an entire population, I would hazard a guess the median would be 0 or 2 unless the difference in numbers of each sex is small enough that people with 0<X<2 testicles might fall right in the middle of the distribution.
> The median is specifically the mid point of the data set, so it won't be "any number between 0 and 2", it will be one specific number depending on the data set.
No. The median is not necessarily unique (just like the mode, and unlike the mean). For more information, you can check here: https://en.wikipedia.org/wiki/Central_tendency#Uniqueness
A median (and not necessarily the median) is any point that minimizes its distance from a set of points under the L1 norm and often you have several points that minimize such distance (such as in the example you provided, assuming perfect 50/50 split between men/women).
> If you look at ten people with 2, ten people with 0, and one person with 1, then the median will be 1.
Correct.
> If you make it twelve people with 2, still ten with 0 and one with 1, suddenly the median is 2.
Sure.
But, again... you are choosing cases where the median of a set of points is unique (odd number of points aligned along a single dimension)... this is the exception, rather than the rule (particularly in higher dimensions than one).
> Across an entire population, I would hazard a guess the median would be 0 or 2 unless the difference in numbers of each sex is small enough that people with 0<X<2 testicles might fall right in the middle of the distribution.
Yes... as I said. The actual median is not 1, but actually either 0 or 2 (because the ratio between men and women is not exactly 50/50). But, again, even in the (rather unlikely) case that the ratio between men and women would be exactly 50/50, the median would still not be uniquely defined as 1 (in such a pathological case, any number between 0 and 2 is a median).
EDIT: after re-reading your comment carefully... I have not considered the subpopulation of people with a single testicle (which is nonzero); so I stand corrected... the median could be 1 (but it is very unlikely that it is the case... it's probably either 0 or 2).
No. The median is not necessarily unique (just like the mode, and unlike the mean). For more information, you can check here: https://en.wikipedia.org/wiki/Central_tendency#Uniqueness
A median (and not necessarily the median) is any point that minimizes its distance from a set of points under the L1 norm and often you have several points that minimize such distance (such as in the example you provided, assuming perfect 50/50 split between men/women).
> If you look at ten people with 2, ten people with 0, and one person with 1, then the median will be 1.
Correct.
> If you make it twelve people with 2, still ten with 0 and one with 1, suddenly the median is 2.
Sure.
But, again... you are choosing cases where the median of a set of points is unique (odd number of points aligned along a single dimension)... this is the exception, rather than the rule (particularly in higher dimensions than one).
> Across an entire population, I would hazard a guess the median would be 0 or 2 unless the difference in numbers of each sex is small enough that people with 0<X<2 testicles might fall right in the middle of the distribution.
Yes... as I said. The actual median is not 1, but actually either 0 or 2 (because the ratio between men and women is not exactly 50/50). But, again, even in the (rather unlikely) case that the ratio between men and women would be exactly 50/50, the median would still not be uniquely defined as 1 (in such a pathological case, any number between 0 and 2 is a median).
EDIT: after re-reading your comment carefully... I have not considered the subpopulation of people with a single testicle (which is nonzero); so I stand corrected... the median could be 1 (but it is very unlikely that it is the case... it's probably either 0 or 2).
It's an approximation, though. You can't really describe a rotating body looking only at its centre of mass. So it does summarise the whole in the sense that we choose to ignore what it cannot describe and live with it, but it's more a matter of convenience than a profound meaning.
No, that's exactly the point. Knowing the center of mass and the moment of inertia is sufficient: You don't know need to know the exact shape, or the variations in density inside the object etc etc.
And notably it can have a validity to it that other averages don’t.
E.g. in this case you can calculate the motion of the center of mass from the forces applied to the object. The same can’t be said for other averages.
E.g. in this case you can calculate the motion of the center of mass from the forces applied to the object. The same can’t be said for other averages.
This is extremely similar to Miguel de Carvalho’s “Mean, what do you mean?” published four years earlier: https://www.maths.ed.ac.uk/~mdecarv/papers/decarvalho2016.pd...
In addition to discussing Kolmogorov’s construction of the mean (first moment), he extends it to higher moments (e.g. variance), and provides some nice examples.
Pity the authors didn’t cite the aforementioned paper. Note that I’m not implying any plagiarism happened; the subject is elementary enough that different people could easily arrive at similar manuscripts independently.
In addition to discussing Kolmogorov’s construction of the mean (first moment), he extends it to higher moments (e.g. variance), and provides some nice examples.
Pity the authors didn’t cite the aforementioned paper. Note that I’m not implying any plagiarism happened; the subject is elementary enough that different people could easily arrive at similar manuscripts independently.
What I got out of this was: Everyone knows that the mean and median are different ways of summarizing a distribution, which have different purposes. But did you know there are many different kinds of means too? The one you think of when you use "mean" is the arithmetic mean. But there are weighted means, geometric means, and harmonic means. The straightforward arithmetic mean isn't always the best way of summarizing the center of a distribution, even though it is conceptually the simplest and the way we've always done it.
> Everyone knows that the mean and median are different ways of summarizing a distribution, which have different purposes. But did you know there are many different kinds of means too?
Sure. More than that, the median is one of those different kinds of means. A mean is a statement about the total aggregated value of several data points. It answers the question "if all of these values were the same value, what would that value be?". And obviously, the answer depends on the details of how you aggregate the individual values.
If your function takes a set of values and returns a value that is at least the minimum of the set and at most the maximum, it's a mean.
Sure. More than that, the median is one of those different kinds of means. A mean is a statement about the total aggregated value of several data points. It answers the question "if all of these values were the same value, what would that value be?". And obviously, the answer depends on the details of how you aggregate the individual values.
If your function takes a set of values and returns a value that is at least the minimum of the set and at most the maximum, it's a mean.
What's with that "associative law" on page 6?
It seems to be asserting that, for all possible k, if the first k input values x_1, ... x_k are replaced by the mean x, then the mean doesn't change.
Sorry what?
(1 2 3 4 5 6) -> arithmetic mean is x = 3.5.
Ok, let k = 3; replace first 3 elements by 3.5:
(3.5 3.5 3.5 4 5 6) -> arithmetic mean is 4.25
Unless I'm totally daft, there has to be a typo there somewhere?
--
Edit, ah, ok. Returning to this, I see where my eyes fooled me. The term on the left goes to x_k. Thus x is a "sub-mean" representative value which is of the first k values. If the first k values are replaced by their "sub-mean", then the "master mean" is undisturbed.
That is very intuitive.
In the case of an arithmetic mean, it's basically a distribution situation: a certain total quantity is distributed into n slots. If we transfer an amount from any slot to any other, the mean does not change. We can choose a subset of the slots, and redistribute their subtotal among them that they have an equal value. That value will be their respective mean. Since we do not add anything to the total, the mean doesn't change.
What isn't so obvious is that this generalizes to the other kinds of means, like geometric.
It seems to be asserting that, for all possible k, if the first k input values x_1, ... x_k are replaced by the mean x, then the mean doesn't change.
Sorry what?
(1 2 3 4 5 6) -> arithmetic mean is x = 3.5.
Ok, let k = 3; replace first 3 elements by 3.5:
(3.5 3.5 3.5 4 5 6) -> arithmetic mean is 4.25
Unless I'm totally daft, there has to be a typo there somewhere?
--
Edit, ah, ok. Returning to this, I see where my eyes fooled me. The term on the left goes to x_k. Thus x is a "sub-mean" representative value which is of the first k values. If the first k values are replaced by their "sub-mean", then the "master mean" is undisturbed.
That is very intuitive.
In the case of an arithmetic mean, it's basically a distribution situation: a certain total quantity is distributed into n slots. If we transfer an amount from any slot to any other, the mean does not change. We can choose a subset of the slots, and redistribute their subtotal among them that they have an equal value. That value will be their respective mean. Since we do not add anything to the total, the mean doesn't change.
What isn't so obvious is that this generalizes to the other kinds of means, like geometric.
> What isn't so obvious is that this generalizes to the other kinds of means, like geometric.
It's obvious if you realize that the geometric mean is just exp(mean(log(values))) and the harmonic mean is just 1/mean(1/values), where "mean" is the arithmetic mean (i.e. they are just the arithmetic mean performed in a transformed space).
It's obvious if you realize that the geometric mean is just exp(mean(log(values))) and the harmonic mean is just 1/mean(1/values), where "mean" is the arithmetic mean (i.e. they are just the arithmetic mean performed in a transformed space).
You replace x_1, ..., x_k with the mean of those k values, not the mean of all the values.
So you would be replacing the first 3 values with 2 in your example, not 3.5
So you would be replacing the first 3 values with 2 in your example, not 3.5
[deleted]
Nice little paper, although maybe a little stilted. It's a good idea to think about these basic quantities a bit more deeply and why you might want to use one measure of central tendency more than another. It's good to have a solid rationale for it, and I thought this was a nice brief overview of some perspectives I wasn't aware of.
For some reason I thought the mean could be thought of as a single number that is most representative of a sample or population in an information loss (algorithmic/kolgomorov complexity, maybe relative entropy?) sense, or maximum likelihood sense (maybe under some distributional constraints?). I might be misremembering that though, and it's difficult to figure out the right search terms to track it down.
For some reason I thought the mean could be thought of as a single number that is most representative of a sample or population in an information loss (algorithmic/kolgomorov complexity, maybe relative entropy?) sense, or maximum likelihood sense (maybe under some distributional constraints?). I might be misremembering that though, and it's difficult to figure out the right search terms to track it down.
Information loss / maximum likelihood only applies in special case distributions. E.g. take a distribution of variable X and the arithmetic mean might be the appropriate average. But then if you look at sqrt(X) then the appropriate average would be RMS.
You can use "maximum entropy" arguments to convince yourself or others that a Gaussian distributional assumption is the best (e.g. if the only thing you know is that your distribution has support over ]-inf,+inf[ and that its variance is bounded, the maximum entropy distribution is a Gaussian) and, in such a case, the sample arithmetic mean will give you the maximum likelihood estimate of the population "location" parameter (and, yes, along with variance, it will contain all required information to perfectly summarize your sample, since these are sufficient statistics, under a Gaussian assumption).
TL;DR: If you can safely assume additiveness/normality, then... yes, mostly. Otherwise, not necessarily.
TL;DR: If you can safely assume additiveness/normality, then... yes, mostly. Otherwise, not necessarily.
I assume not only physicists have the rule: never trust a scalar or statistic unless you can also see the distribution (histogram) behind it.
> average is merely an abstraction which has meaning only within its mathematical set-up.
And probability is just a normalized denumerably additive measure over a sigma algebra.
We've done very well constructing theories on the basis of these definitions, but it is useful to remember that neither has empirical content.
And probability is just a normalized denumerably additive measure over a sigma algebra.
We've done very well constructing theories on the basis of these definitions, but it is useful to remember that neither has empirical content.
Probability began as the study of games of chance, with a lot of empirical content. It wasn't until centuries later that the current abstract mathematical foundation in measure theory (a normalized denumerably additive measure over a sigma algebra) was developed.
And even today probability is taught with a lot of empirical examples. (At least it was in my classes, both undergraduate and graduate.)
And even today probability is taught with a lot of empirical examples. (At least it was in my classes, both undergraduate and graduate.)
> And even today probability is taught with a lot of empirical examples. (At least it was in my classes, both undergraduate and graduate.)
Can you mention some of these examples? FYI, I've taught a bunch of Stats/Econometrics classes at grad and undergrad level.
Can you mention some of these examples? FYI, I've taught a bunch of Stats/Econometrics classes at grad and undergrad level.
My university treated stats and probability as separate disciplines, and I didn't have as many empirical examples in stats classes. We did use data sets in exercises, of course, but the focus was on correctly applying statistical methods, not whether the problem was empirically interesting or the answer was empirically correct.
We had a lot more empirical work in probability.
In introductory probability classes we had, among other things, famous examples like the Chevalier de Méré, birthday, Monty Hall, Bertrand's ballot, and coupon collector's problems; and a lot of calculating probabilities involving coins, dice, cards, and drawing balls from urns in various and increasingly complicated ways.
In graduate probability classes we studied mathematical models of things like Brownian motion, birth-death processes, and queuing systems. Our work on these was theoretical, naturally, but grounded in empirical problems from the physical sciences.
We had a lot more empirical work in probability.
In introductory probability classes we had, among other things, famous examples like the Chevalier de Méré, birthday, Monty Hall, Bertrand's ballot, and coupon collector's problems; and a lot of calculating probabilities involving coins, dice, cards, and drawing balls from urns in various and increasingly complicated ways.
In graduate probability classes we studied mathematical models of things like Brownian motion, birth-death processes, and queuing systems. Our work on these was theoretical, naturally, but grounded in empirical problems from the physical sciences.
Thank you for the examples. That shows that we are talking about different things. In all those examples, we are talking about a way to model them using rules enshrined in the _axiomatic_ foundations of probability. Things like, "given that the probability of X is p and the probability of Y is q and X and Y are independent events, probability of X and Y is p times q" etc.
That statement and the statements about the actual values of probabilities have no empirical content. Statements like "if there are 10 balls in an urn and three of them are red, the probability of pulling out a red ball is 30%" are _assumptions_ and they are not falsifiable. In practice, this simplifying assumption seems to have worked at least in making it easier think about certain phenomena and so we maintain that assumption, but it is an assumption nevertheless.
That statement and the statements about the actual values of probabilities have no empirical content. Statements like "if there are 10 balls in an urn and three of them are red, the probability of pulling out a red ball is 30%" are _assumptions_ and they are not falsifiable. In practice, this simplifying assumption seems to have worked at least in making it easier think about certain phenomena and so we maintain that assumption, but it is an assumption nevertheless.
I disagree. Statements like "if there are 10 [well mixed and otherwise indistinguishable] balls in an urn and three of them are red, the probability of pulling out a red ball is 30%" are absolutely falsifiable. An experiment showing that this statement is false is certainly possible.
You're right that these are mathematical models and therefore simplifications of reality, but they are useful mathematical models only if their predictions are empirically valid.
And if anyone came up with a more empirically useful version of probability, they'd be famous even if mathematicians refused to accept it, because all the scientists, engineers, gamblers, actuaries, etc who rely on the predictions of probability would make it so.
You're right that these are mathematical models and therefore simplifications of reality, but they are useful mathematical models only if their predictions are empirically valid.
And if anyone came up with a more empirically useful version of probability, they'd be famous even if mathematicians refused to accept it, because all the scientists, engineers, gamblers, actuaries, etc who rely on the predictions of probability would make it so.
The most thorough treatment of means and their applications in statistics is the monograph by Corrado Gini.[1] It is also available in a 1970 Russian translation.
[1] Gini, Corrado. (1958) Le Medie. Torino: Unione Tipografico-Editrice Trinese.
[1] Gini, Corrado. (1958) Le Medie. Torino: Unione Tipografico-Editrice Trinese.
[deleted]
Looks like there is an error in the table at page 7. The power generalisation should be raised to the power of 1/α, right?
Is this really meaningful?
I think it's helpful to general number literacy to know about the different types of mean and in which situations they are applicable:
https://en.wikipedia.org/wiki/Geometric_mean#Applications
https://en.wikipedia.org/wiki/Harmonic_mean#Examples
https://en.wikipedia.org/wiki/Geometric_mean#Applications
https://en.wikipedia.org/wiki/Harmonic_mean#Examples
Seems far to philosophical to be useful in my opinion.
2020
Obligatory xkcd: https://xkcd.com/2435/ (Geothmetic Meandian)
The mean is just another magic number
For example - suppose you host some data on 3 cloud providers - GCP, AWS & Azure. You believe, on average, the latency is the same for each provider. However, the variance is different - say 2,3 and 4. So, how much traffic should you route to each provider ?
"What does the mean really mean" is the same as the above question. If you believe average latency is the same, then how should it matter what the variance is ? Just route 1/3 of your traffic to each provider & be done. But that's exactly the wrong answer!
Because the arithmetic mean is not what you want here. You want to make your own mean. You really want a weighted mean, but you can't just pick the weights willy-nilly. The sample mean has its own variance! So if you say you don't give a rat's ass about statistics, you will just route 1/3 traffic to each provider - what you are really doing is picking weights of 1/3, in turn giving your sample mean xbar a sample variance of exactly 1 ( easy to show ).
But if you pick the right weights, the sample variance drops to 12/13, which is smaller than 1. So by picking your own weighted mean, you get to fiddle with the sample variance - you can dial it up or down! The smallest you can get it down to is 12/13, in which case the weights will have to be 6/13 to GCP, 4/13 to AWS & 3/13 to Azure.
Fun stuff.