Statistics have lost their power (2017)(theguardian.com)
theguardian.com
Statistics have lost their power (2017)
https://www.theguardian.com/politics/2017/jan/19/crisis-of-statistics-big-data-democracy
69 comments
I feel like all of this is overcomplicating the issue. Now, at the risk of oversimplifying, I offer this:
If I am not doing well and some journalist or some economist tells me the GDP has grown a lot and everything is great, I am going to feel resentment toward that person. Doubly so if they see my less-than-enthusiasm and start talking about the “common good.” Why? Because the implication is either that there’s something wrong with me (for not keeping up with the Joneses) or that I am an acceptable sacrifice for the benefit of the common good.
Okay, it’s fine if you (the journalist or economist) think I’m an acceptable sacrifice, but don’t expect me to be happy about it, let alone agree with you. Don’t expect me to support the status quo, come Election Day. DO expect me to try to throw a spanner into the gears. DO expect me to support a strongman who says he’ll champion my cause over the cause of those who think I’m the acceptable sacrifice.
If I am not doing well and some journalist or some economist tells me the GDP has grown a lot and everything is great, I am going to feel resentment toward that person. Doubly so if they see my less-than-enthusiasm and start talking about the “common good.” Why? Because the implication is either that there’s something wrong with me (for not keeping up with the Joneses) or that I am an acceptable sacrifice for the benefit of the common good.
Okay, it’s fine if you (the journalist or economist) think I’m an acceptable sacrifice, but don’t expect me to be happy about it, let alone agree with you. Don’t expect me to support the status quo, come Election Day. DO expect me to try to throw a spanner into the gears. DO expect me to support a strongman who says he’ll champion my cause over the cause of those who think I’m the acceptable sacrifice.
[deleted]
Journalism has been one of the worst offenders with regard to the misuse of statistics, not only in "soft" topics, but also in science reporting. Perhaps statistical literacy ought to be normative in that profession the same way that checking sources and using correct grammar (supposedly) is.
Even if statistical literacy was normative wouldn't people's selfishness still cause them to use/abuse statistics to influence those who are less literate in them?
Knowledge can be abused, but lack of knowledge will almost certainly lead to, unintentional, abuse. Especially in the case of statistics.
In many cases, reporters want to be dramatic, but not liars. However, the lack of deeper statistical knowledge often lead to them becoming liars, even though they only wanted to be dramatic.
In many cases, reporters want to be dramatic, but not liars. However, the lack of deeper statistical knowledge often lead to them becoming liars, even though they only wanted to be dramatic.
As usual, the title's description is wrong, and the real news is buried and overlooked in the story:
"People assume that the numbers are manipulated and dislike the elitism of resorting to quantitative evidence."
If people don't believe that something is useful, it doesn't matter what that "something" is, it will start to have a diminishing affect our society. It goes for statistics and liberal democracies just as much as ancient gods and ancient languages of yore.
"People assume that the numbers are manipulated and dislike the elitism of resorting to quantitative evidence."
If people don't believe that something is useful, it doesn't matter what that "something" is, it will start to have a diminishing affect our society. It goes for statistics and liberal democracies just as much as ancient gods and ancient languages of yore.
That's like saying if people don't like military drones, then military drones will start to have less relevance; but some technologies are sufficiently powerful in their own right that it needs only a few people to recognize their power. Then the stress of competition can do the rest.
I doubt statistics or mathematics gained its influence because various cultures were fond of symbolic play. We study these things because we expect concrete payoff, not because of cultural affection.
So is social science lame? Or is Google insight scary? At some point a deliberate posture of mis-devaluation will come to terms with reality.
I doubt statistics or mathematics gained its influence because various cultures were fond of symbolic play. We study these things because we expect concrete payoff, not because of cultural affection.
So is social science lame? Or is Google insight scary? At some point a deliberate posture of mis-devaluation will come to terms with reality.
> That's like saying if people don't like military drones, then military drones will start to have less relevance;
Sure I'll back that up. Thought experiment: if everyone, including all the drone pilots, suddenly magically stand up all at once, walk away, and never talk about it again, then yeah it won't impact humanity.
Is this particular scenario realistic? No. Has mathematic or chemical knowledge been collectively lost before in human history, and been eliminated from society? Yes.
> but some technologies are sufficiently powerful in their own right that it needs only a few people to recognize their power. Then the stress of competition can do the rest.
Sure, those few power can recognize its power, but if it's anything short of immortality, never underestimate the power of every other human being being a Luddite wielding a pitchfork ready to kill over progress, because it violates a cultural norm. Doesn't matter if your statistics can find a cure, societies and collective humans' irrationalities are expert suppressors of technology as both a mean and an end. Look no further than the US stance on nuclear waste reprocessing. The science and power are there. The will of people, is not. Or, look at South Africa and Ukraine giving up nuclear weapons. Power? Yes. Popular will to hold onto this power? No.
> We study these things because we expect concrete payoff, not because of cultural affection.
Exactly, because society values concrete payoff, things like "derivations from first principles" and "following the scientific method". But if society as a whole pressures people to never pursue ideologies along these lines, or turn actively hostile against it (book burning, HDD erasing).
I take this article as an alarming warning. Scientific progress is itself not an end. And it is not an inevitability. I like scientific progress and statistical tools, this article just goes to show how careful -- ethically responsible -- those of us wielding them must be when living in our society. And call out bad actors.
This is why engineering professions have ethics boards. If engineers build bridges and bad actors build bad bridges, people won't trust bridges anymore. And if people don't trust bridges, they don't get used, so more don't get built.
The greatest failure of the "modern" education has precisely been the removal of religious ethics without an equivalent amount of time working to help students develop their sense of personal ethics.
Sure I'll back that up. Thought experiment: if everyone, including all the drone pilots, suddenly magically stand up all at once, walk away, and never talk about it again, then yeah it won't impact humanity.
Is this particular scenario realistic? No. Has mathematic or chemical knowledge been collectively lost before in human history, and been eliminated from society? Yes.
> but some technologies are sufficiently powerful in their own right that it needs only a few people to recognize their power. Then the stress of competition can do the rest.
Sure, those few power can recognize its power, but if it's anything short of immortality, never underestimate the power of every other human being being a Luddite wielding a pitchfork ready to kill over progress, because it violates a cultural norm. Doesn't matter if your statistics can find a cure, societies and collective humans' irrationalities are expert suppressors of technology as both a mean and an end. Look no further than the US stance on nuclear waste reprocessing. The science and power are there. The will of people, is not. Or, look at South Africa and Ukraine giving up nuclear weapons. Power? Yes. Popular will to hold onto this power? No.
> We study these things because we expect concrete payoff, not because of cultural affection.
Exactly, because society values concrete payoff, things like "derivations from first principles" and "following the scientific method". But if society as a whole pressures people to never pursue ideologies along these lines, or turn actively hostile against it (book burning, HDD erasing).
I take this article as an alarming warning. Scientific progress is itself not an end. And it is not an inevitability. I like scientific progress and statistical tools, this article just goes to show how careful -- ethically responsible -- those of us wielding them must be when living in our society. And call out bad actors.
This is why engineering professions have ethics boards. If engineers build bridges and bad actors build bad bridges, people won't trust bridges anymore. And if people don't trust bridges, they don't get used, so more don't get built.
The greatest failure of the "modern" education has precisely been the removal of religious ethics without an equivalent amount of time working to help students develop their sense of personal ethics.
I see this as a good thing. Many 'scientific' studies are manipulated and paid for by special interests. Our existing peer review & journal system hides the actual experimental setup behind paywalls, there is often no repeatability.
The scientific system needs to be opened up or replaced. Distributing information and experimental results no longer requires expensive printing.
People are right not to trust it, scientific journals are now the gatekeepers rather than the gateways they once were.
The scientific system needs to be opened up or replaced. Distributing information and experimental results no longer requires expensive printing.
People are right not to trust it, scientific journals are now the gatekeepers rather than the gateways they once were.
This is an objectively good thing, most statistics don't actually tell most people anything interesting about the world. They don't aid the average untrained guy in the street in gaining a better understanding of the world around them. They are bombarded with opinions that are based on contradictory studies and statistics all day, this is bad in politics and sociology but medical statistics are probably the worst offenders.
Wine is good for you, chocolate is bad for you, cholesterol is good for you, wine is bad for you, red meat good for you, Fluoride bad for you! I could find you a statistically significant study supporting any conclusion I want right this minute. Since that is the case, why should the average Joe believe any of them? This is made even worse by news articles and popsci blogs not even linking to the source papers, so avg Jane can't even investigate further if she wants to.
Wine is good for you, chocolate is bad for you, cholesterol is good for you, wine is bad for you, red meat good for you, Fluoride bad for you! I could find you a statistically significant study supporting any conclusion I want right this minute. Since that is the case, why should the average Joe believe any of them? This is made even worse by news articles and popsci blogs not even linking to the source papers, so avg Jane can't even investigate further if she wants to.
The fact that the statement “most statistics don't actually tell most people anything interesting about the world” relies on two statistical conditions is humorous to me.
Statistics aren’t bad. Extrapolating unwarranted conclusions is bad.
I am not a trained medical professional, and I don't have time to be scrutinizing and reading background literature to gain a full understanding of the context each specific study has within its sub-field. I don't have access to whether these conclusions are generally accepted or are commonly disregarded for some technicality in the methodology that I didn't understand. Me personally, I can't be an expert in everything all the time, and I can't waste my life gaining a full breadth of understanding of every topic. I rely on extrapolations to do this for me, if those extrapolations are commonly faulty, I won't pay any mind to them.
Unfortunately, this is the best option for modeling the world around us that humanity has come up with. Throwing out the only tool we have to evaluate and re-evaluate what we consider to be knowledge is not the best course of action.
Solution is to support proper scientific procedures, support civic organizations carrying out the important tasks of vetting claims, and being an example of someone who values the correct protocols when evaluating data and making conclusions (or not making them).
It's a very slow process, and we're lucky we're in a timeline where the work of various people throughout history has gotten us here. We'll have to work to keep it going further, and literacy in math, statistics, and the scientific process is crucial.
Solution is to support proper scientific procedures, support civic organizations carrying out the important tasks of vetting claims, and being an example of someone who values the correct protocols when evaluating data and making conclusions (or not making them).
It's a very slow process, and we're lucky we're in a timeline where the work of various people throughout history has gotten us here. We'll have to work to keep it going further, and literacy in math, statistics, and the scientific process is crucial.
I disagree here strongly. It is not the best option. Statistics has become the tool of obfuscation and making exaggerated claims.
Compare the two statements: "10 out of 80 people that took the drug lived a month longer but were extremely miserable."
Versus: "A new cancer drug doubles the average survival rates scientists say".
Which one do you understand better? Here I just made up some examples, but as a scientist I'll tell you just about all the papers I read are in the second category, it takes an unreasonable amount of effort to get the first type of infomration out of the publications.
Compare the two statements: "10 out of 80 people that took the drug lived a month longer but were extremely miserable."
Versus: "A new cancer drug doubles the average survival rates scientists say".
Which one do you understand better? Here I just made up some examples, but as a scientist I'll tell you just about all the papers I read are in the second category, it takes an unreasonable amount of effort to get the first type of infomration out of the publications.
I don't understand the chain of reasoning in your claim regarding the 1st and 2nd statements, and statistics in general.
When I use the word statistics in this conversation, I mean the use of math to analyze and compare data. This can be done inappropriately, of course, but the problem doesn't lie with the principles of math (including statistics), but the agent using them inappropriately.
And medical (or biology related) statistics are used most inappropriately, due to various cognitive biases and financial incentives. But the problem is not statistics, and it's still the best tool we have unless we can find a Magic School Bus to shrink us down and go observe the chemical reactions in real time. But I know that stats are abused, so anytime I see a fantastical claim, especially in medicine, I approach it much skepticism.
When I use the word statistics in this conversation, I mean the use of math to analyze and compare data. This can be done inappropriately, of course, but the problem doesn't lie with the principles of math (including statistics), but the agent using them inappropriately.
And medical (or biology related) statistics are used most inappropriately, due to various cognitive biases and financial incentives. But the problem is not statistics, and it's still the best tool we have unless we can find a Magic School Bus to shrink us down and go observe the chemical reactions in real time. But I know that stats are abused, so anytime I see a fantastical claim, especially in medicine, I approach it much skepticism.
There are fields of science that have banned certain statistical measures:
Psychology journal bans P values
https://www.nature.com/news/psychology-journal-bans-p-values...
I happen to agree with the sentiment expressed above that the entire concept of hypothesis testing is fundamentally flawed. This is what you call the "best tool". It is not, it is the tool that is used because it allows for the most fudging of results.
Psychology journal bans P values
https://www.nature.com/news/psychology-journal-bans-p-values...
I happen to agree with the sentiment expressed above that the entire concept of hypothesis testing is fundamentally flawed. This is what you call the "best tool". It is not, it is the tool that is used because it allows for the most fudging of results.
the problem is that in many (most cases?) even trained scientist would have a hard time deciding which statements are statistically valid.
There are endless ways to be unintentionally obscure, to semi-intentionally obfuscate or to just flat out lie. And it very hard to tell these apart - one would need to dedicate more effort than usually worth.
There are endless ways to be unintentionally obscure, to semi-intentionally obfuscate or to just flat out lie. And it very hard to tell these apart - one would need to dedicate more effort than usually worth.
That's why we have organizations like the CDC, FDA, NHTSA, etc. We have the internet, cheap computing and storage of data, so it should all be transparent. Countries verifying claims independently goes a long way too.
And obviously (sweeping) results from health related experiments should obviously be met with skepticism considering the complex nature of the problem.
And obviously (sweeping) results from health related experiments should obviously be met with skepticism considering the complex nature of the problem.
> Extrapolating unwarranted conclusions is bad.
But that's practically all they're used for!
But that's practically all they're used for!
Not in physics or chemistry. It's a bigger problem in biology because we want to be able to make claims but don't have the means to properly design and carry out experiments that aren't subject to many errors.
Sure, but is it significantly bad?
Wouldn’t knowing statistics allow the average Joe to realise that most of these studies are simply not representative?
That a current generation is under-equipped to use quantitative discourse is sad, but it also suggests that what communities need are experts and not evidence. They want to listen to a trustworthy leader and not conflicting data -- because data will conflict.
> but it also suggests that what communities need are experts and not evidence
IMHO it actually suggests we need to improve the average person's educational level
IMHO it actually suggests we need to improve the average person's educational level
I think this is missing the point though, I've taken an upper level Statistics class, most people here with CS degrees have, but that doesn't help me understand whether a sociological study has a faulty component in its methodology. This is compounded by the culture of not reporting full data alongside papers, and the culture of writing papers in terse language.
no, unfortunately, it is not about lack of education
walk into any university, go from one faculty office to another, ask each professor what a p-value is and the overwhelming majority won't give you even a half-decent definition (I did this and can attest to it)
you know why? because p-values are far more complex than people care to admit. Most statisticians cannot properly use it! There are ongoing "ideological wars" over what it actually means. Even the founders Fisher etc argued all the time about them. If you used p-value correctly, you would almost never be able to find anything, not even valid results.
So people have to "bend" the truth a little bit. Nowadays more often than before, the truth gets bent out of shape.
walk into any university, go from one faculty office to another, ask each professor what a p-value is and the overwhelming majority won't give you even a half-decent definition (I did this and can attest to it)
you know why? because p-values are far more complex than people care to admit. Most statisticians cannot properly use it! There are ongoing "ideological wars" over what it actually means. Even the founders Fisher etc argued all the time about them. If you used p-value correctly, you would almost never be able to find anything, not even valid results.
So people have to "bend" the truth a little bit. Nowadays more often than before, the truth gets bent out of shape.
If people were better educated regarding math and statistics, then it would become much less effective to "bend" the truth, especially with verification of experiments.
But yes, society doesn't function well when people are corrupted.
But yes, society doesn't function well when people are corrupted.
you have to bend the truth because it would not work otherwise at all.
Your distribution is not normal - it almost never is, now you have to argue that the method is still robust - there is no proof that is indeed robust as you claim. Then your measurement is a bit biased, see on Tuesdays the measurements do turn out a little bit differently, but most often does not matter, so you skimp over that as well.
Statistics is the wrong approach because if you did everything correctly and stuck to the letter of it you only end up hurting yourself. You'll make fewer of the valid discoveries that you make when you bend it a bit.
This is the fundamentally wrong thing about it! Strictly sticking to the rules will make you weaker. Bending statistics a little leads to more valid results.
Your distribution is not normal - it almost never is, now you have to argue that the method is still robust - there is no proof that is indeed robust as you claim. Then your measurement is a bit biased, see on Tuesdays the measurements do turn out a little bit differently, but most often does not matter, so you skimp over that as well.
Statistics is the wrong approach because if you did everything correctly and stuck to the letter of it you only end up hurting yourself. You'll make fewer of the valid discoveries that you make when you bend it a bit.
This is the fundamentally wrong thing about it! Strictly sticking to the rules will make you weaker. Bending statistics a little leads to more valid results.
I don’t understand. Using statistics doesn’t imply using normal distributions.
I just gave you an example, there are many statistical measures that have that particular requirement.
If it is not normality then something else. Every statistical measure makes assumptions, most of which never quite hold true. That was my point. When someone uses statistics they always fudge things a bit. Many times does not matter, if you were stickler for "precision" it would not work.
If it is not normality then something else. Every statistical measure makes assumptions, most of which never quite hold true. That was my point. When someone uses statistics they always fudge things a bit. Many times does not matter, if you were stickler for "precision" it would not work.
[deleted]
To understand statistics, you have to either know some statistics, or be willing to learn.
It is it's own thing, the art and science of making data and observations into information. Yes newspapers and the regular pundits abuse it all the time, but don't pin that on statistics, pin that on those that abuse it with, or without (usually) knowing better.
Statistics are unavoidable, unavoidable enough that much of our perception seem to make use of the same statistical calculations sciententists use, only without us having the ability to scrutinize its workings.
It is it's own thing, the art and science of making data and observations into information. Yes newspapers and the regular pundits abuse it all the time, but don't pin that on statistics, pin that on those that abuse it with, or without (usually) knowing better.
Statistics are unavoidable, unavoidable enough that much of our perception seem to make use of the same statistical calculations sciententists use, only without us having the ability to scrutinize its workings.
A statement like "most statistics don't actually tell most people anything" demonstrates both a profound lack of understanding of statistics. So why comment?
As the old saying goes: "Better to remain silent and be thought a fool than to speak and to remove all doubt."
As the old saying goes: "Better to remain silent and be thought a fool than to speak and to remove all doubt."
Bet you dollars to donuts none of the underlying studies you complain about pass judgment in the form of x good x now bad. Those are always added by the pop media or implied by the reading.
There's an interesting blog post about something similar:
https://slatestarcodex.com/2014/04/28/the-control-group-is-o...
Basically it boils down to the fact that studies & statistics used are still very, very, prone to human bias.
https://slatestarcodex.com/2014/04/28/the-control-group-is-o...
Basically it boils down to the fact that studies & statistics used are still very, very, prone to human bias.
Which can as well be seen an argument why statistics are necessary, as even with them we are stuck with a lot of bias. Imagine where we would be if we didn't have statistics at all!
The quote, "using statistics like a drunk uses a lamppost, for support and not for illumination" comes to mind.
Data is great for illuminating things, but for arguments, it's usually cherry picked. I do think that writers for the guardian vastly underestimate the intelligence of people who don't agree with them. I generally mistrust statistical arguments because someone forearmed with statistical talking points has usually already signalled their intransigence. Data is nice, but alignment of interests and incentives is better.
It reduces to, "according to criteria and data I have selected, your argument does not bear out." It is a passive appeal to the authority of whoever provided the number, and not meaningful or logical information in itself.
Logical arguments tend not to persuade anyway, but to say someones argument is illogical because it does not resist a coded illogical premise (appeal to authority fallacy), is certainly a way to make them give up on arguing with you.
Data is great for illuminating things, but for arguments, it's usually cherry picked. I do think that writers for the guardian vastly underestimate the intelligence of people who don't agree with them. I generally mistrust statistical arguments because someone forearmed with statistical talking points has usually already signalled their intransigence. Data is nice, but alignment of interests and incentives is better.
It reduces to, "according to criteria and data I have selected, your argument does not bear out." It is a passive appeal to the authority of whoever provided the number, and not meaningful or logical information in itself.
Logical arguments tend not to persuade anyway, but to say someones argument is illogical because it does not resist a coded illogical premise (appeal to authority fallacy), is certainly a way to make them give up on arguing with you.
It is more about declining trust in the source of that information and the institutions that spread it around. People have been given plenty of reasons to distrust the various sources of information over the past decades.
Even if the underlying statistics accurately represent the situation correctly, bad analysis on top of it creates the same results. From last week:
"Paul Krugman and other mainstream trade experts are now admitting that they were wrong about globalization: It hurt American workers far more than they thought it would."
Ref: https://news.ycombinator.com/item?id=21357096
"Paul Krugman and other mainstream trade experts are now admitting that they were wrong about globalization: It hurt American workers far more than they thought it would."
Ref: https://news.ycombinator.com/item?id=21357096
That Foreign Policy article illustrates exactly why the influence of statistics is on the decline. If you read Krugman's original article (https://www.bloomberg.com/opinion/articles/2019-10-10/inequa...), you can see Krugman make several points, all using statistics. One point is that the effect of globalization in the 90s was small, but it had an unexpectedly large impact on manufacturing in the 2000s. The second point is that the overall effect disguised the fact that certain industries were particularly hard hit. He then cites the papers where you can find the details. These papers only exist because of the statistics they do. Krugman is a clear writer, and he is taking the unusual step of admitting he was wrong, so his article should be interesting.
But nobody cares about that, because nobody cares about statistics anymore. They want that dopamine hit of outrage, so they read the FP article, that deliberately misrepresents the argument (notice how "manufacturing" suddenly becomes "workers", despite the fact that most workers are not in manufacturing), and doesn't give any evidence. It just quotes a bunch of people settling scores with Krugman. Even there, it mischaracterizes, for example, Krugman's review of Greider's book to heighten the outrage.
Statistics is boring, and rage is interesting, so the Internet era optimizes for rage.
But nobody cares about that, because nobody cares about statistics anymore. They want that dopamine hit of outrage, so they read the FP article, that deliberately misrepresents the argument (notice how "manufacturing" suddenly becomes "workers", despite the fact that most workers are not in manufacturing), and doesn't give any evidence. It just quotes a bunch of people settling scores with Krugman. Even there, it mischaracterizes, for example, Krugman's review of Greider's book to heighten the outrage.
Statistics is boring, and rage is interesting, so the Internet era optimizes for rage.
This single sentence explains the problem very well:
> The thinktank British Future has studied how best to win arguments in favour of immigration and multiculturalism.
BF didn't set out to figure out whether immigration and multiculturalism is favorable. They only tried to figure out how to use statistics (or anything else) in support of their foregone conclusion.
Maybe the headline should be "Statistics have lost their usefulness for propaganda."
> The thinktank British Future has studied how best to win arguments in favour of immigration and multiculturalism.
BF didn't set out to figure out whether immigration and multiculturalism is favorable. They only tried to figure out how to use statistics (or anything else) in support of their foregone conclusion.
Maybe the headline should be "Statistics have lost their usefulness for propaganda."
I think the solution to this problem is more statistics. In particular, we need a mandatory statistics course at the high school level. The problem is that people are bombarded with "studies" right now claiming to prove stuff and many of them do no such thing. If people had a better sense of what is provable and what isn't, what makes for a good study, how much you should trust the results of a single study etc, I'd hope that they'd be far more informed and effective citizens.
that is the cliche definition of insanity:
"doing the same thing over and over and expecting a different result."
probably the most common mistake students everywhere make. They keep taking more courses - it never helps just makes you more confused.
"doing the same thing over and over and expecting a different result."
probably the most common mistake students everywhere make. They keep taking more courses - it never helps just makes you more confused.
I can't help but laugh. Make the slightest mention of statistics broken down by race, and the same publication that is despairing of how people no longer trust statistics, would be finding innumerable reasons why those statistics are invalid.
Why does that make you laugh? Your take away should not be "well clearly everything someone calls stats should be supported". The article is about how stats are becoming dismissed before examination, not that they shouldn't be scrutinized.
And that all the examples given of uneducated masses distrusting statistics just happen to align with the Guardian's pet causes is of course pure coincidence.
Really, that's the perfect example of why statistics are distrusted - only those statistics that align with someone's agenda get presented. And sometimes, they're not even gathered in the first place: https://www.thelocal.se/20180508/why-sweden-doesnt-keep-stat...
Really, that's the perfect example of why statistics are distrusted - only those statistics that align with someone's agenda get presented. And sometimes, they're not even gathered in the first place: https://www.thelocal.se/20180508/why-sweden-doesnt-keep-stat...
You're describing a separate phenomenon from what the article is about. Poking at statistics with biased/politically motivated effort is not the same thing as dismissing the field of statistics entirely.
The article is not suggesting that statistics should not be scrutinized.
The article is not suggesting that statistics should not be scrutinized.
I guess the point is that, when the statistics support their political bent, they decry people questioning the validity of them, and consider not blindly accepting them as being proof of the idiocy of the reader.
But, give them statistics that go against their pre-ordained narrative, and suddenly it is the reader who doesn't immediately refute the statistics who is the uneducated rube.
But, give them statistics that go against their pre-ordained narrative, and suddenly it is the reader who doesn't immediately refute the statistics who is the uneducated rube.
/cue handwringing op-ed about biased AI algorithms
Hey now, in the Guardian's defense, it's not as if any world government has ever been caught lying and misrepresenting facts to its population in the past.
Too many science projects reporting statistical results that do not hold up in repeat experiments; contradictory media coverage of the same topic repeatedly as a result.
Reduces trust in science, statistics, and the media all at once.
Reduces trust in science, statistics, and the media all at once.
The problem is verifying experiments gets no funding.
I think the bigger problem is that the initial experiments are usually funded by interests with a desired outcome in the study - instead of being funded by grant money or educational funds.
It is a 17th-century concept applied to 21st-century data collection.
Plus it gets misused all the time - even by statisticians that should know better.
Plus it gets misused all the time - even by statisticians that should know better.
At one point does policy targeting a statistical measurement render that statistic useless? I’m looking at you, CPI.
> At one point does policy targeting a statistical measurement render that statistic useless?
When the statistic is a proxy for a quantity of interest, and when policy exploits ways of manipulating that proxy without commensurate change in the underlying quantity of interest. My driving policy of targeting my speedometer does a fine job of controlling my vehicle's speed even though the former is just a potentially-inaccurate measurement.
In the case of the CPI (actually the PCE for the US Fed [1]), the quantity of interest is the "[nominal] cost of living." To the extent the CPI doesn't in fact measure the cost of living, the easy solution is to change to a price index that does measure the cost of living. Since living tends to involve buying and consuming things, a price index, suitably defined, can be a fine measure of this.
To the extent you are concerned about the exact ways in which the CPI is computed, independent measurements [2] generally reproduce the broad trends in US data, which in turn suggests there is no significant manipulation by the BLS.
[1] -- https://en.wikipedia.org/wiki/Personal_consumption_expenditu...
[2] -- http://www.thebillionpricesproject.com/
When the statistic is a proxy for a quantity of interest, and when policy exploits ways of manipulating that proxy without commensurate change in the underlying quantity of interest. My driving policy of targeting my speedometer does a fine job of controlling my vehicle's speed even though the former is just a potentially-inaccurate measurement.
In the case of the CPI (actually the PCE for the US Fed [1]), the quantity of interest is the "[nominal] cost of living." To the extent the CPI doesn't in fact measure the cost of living, the easy solution is to change to a price index that does measure the cost of living. Since living tends to involve buying and consuming things, a price index, suitably defined, can be a fine measure of this.
To the extent you are concerned about the exact ways in which the CPI is computed, independent measurements [2] generally reproduce the broad trends in US data, which in turn suggests there is no significant manipulation by the BLS.
[1] -- https://en.wikipedia.org/wiki/Personal_consumption_expenditu...
[2] -- http://www.thebillionpricesproject.com/
90% of statisticians agree with the conclusion.
The other 90% challenge the sources of this article.
(With apologies you Yogi Berra)
The other 90% challenge the sources of this article.
(With apologies you Yogi Berra)
Describing this entirely as "the declining authority of statistics" is like saying the WTC towers suffered "declining structural integrity" before they collapsed - there are people behind both...
God damnit, you should know better than bringing up WTC towers here. It attracts conspiracy theorist like how moth are drawn to a flame.
freejulian85(4)
It's easy to decry:
> Presented with official estimates of how many immigrants are in the country illegally, a common response is to scoff. Far from increasing support for immigration, British Future found, pointing to its positive effect on GDP can actually make people more hostile to it.
... but the underlying belief appears to be closer to:
> If you live in one of the towns in the Welsh valleys that was once dependent on steel manufacturing or mining for jobs, politicians talking of how “the economy” is “doing well” are likely to breed additional resentment. From that standpoint, the term “GDP” fails to capture anything meaningful or credible.
The first statement makes no sense in and of itself, and it would seemingly suggest a knee-jerk reaction against the very idea of data. However, in the context provided by the second statement we see a different view: a core distrust that "average well-being" (represented by the statistic of choice) in fact represents the common good, suitably defined[1].
As a result, I feel that the article's later musings -- that we're merely collecting the wrong kind of statistics -- misses the core point. If we start reporting on underemployment, then the right circumstances might come along to discredit that indicator, and so on. The article begins to grapple with this issue in its digression into big data (and how there are no national agencies for it in the manner of the ONS), but even there the author deals with a practical problem of said data being more useful in the short term than long term than the core problem:
Ultimately, if society as a whole does not have a shared view of the 'common good', then _no quantity at all_ can measure it. The question of "is this good?" becomes fundamentally unanswerable in a democracy with an identity crisis.
[1] -- That could mean "median well-being," "well-being of people I know / in my region / like me," or "personal, individual well-being of the speaker."