Higher Social Classes Have an Exaggerated Belief That They Are More Capable(apa.org)
apa.org
Higher Social Classes Have an Exaggerated Belief That They Are More Capable
https://www.apa.org/news/press/releases/2019/05/social-class-exaggerated-belief
105 comments
I find this the worst insult of being poor: to be assumed stupid.
But I mean, hey, if they did better on a flashcard game, then an unequal caste system is justified after all!
But I mean, hey, if they did better on a flashcard game, then an unequal caste system is justified after all!
Don’t worry not everyone thinks poor people are stupid. I am poor and know poor people are not stupid. In fact I am so poor I had to learn how to change so many broken things on my car it is now called Theseus Grand Voyager. I am so poor I had to learn to bake bread and feed my family from scratch. I am so poor that instead of buying the wooden building planks they have at Science World that my kids loved to play with I had my dad teach me how to use his table and chop saw and I made my own blocks out of scrap wood. I am rich in knowledge and feel rich to have my necessities for me and my family met. Most of all I am happy with my life. I just feel that I would not have learned nearly as much having money and the ability to pay someone else to do things for me.
mcguire(6)
This reminds me of the studies by Paul Piff:
https://matrix.berkeley.edu/research/are-wealthy-more-narcis...
In particular, he demonstrated that in a rigged game of Monopoly, where one player e.g. starts off with more money than the others, that player will sooner or later demonstrate signs of feeling entitled and, in some cases, even attribute their victory at the end not to their initial advantage but to simply playing better than the other participants:
https://www.ted.com/talks/paul_piff_does_money_make_you_mean
https://matrix.berkeley.edu/research/are-wealthy-more-narcis...
In particular, he demonstrated that in a rigged game of Monopoly, where one player e.g. starts off with more money than the others, that player will sooner or later demonstrate signs of feeling entitled and, in some cases, even attribute their victory at the end not to their initial advantage but to simply playing better than the other participants:
https://www.ted.com/talks/paul_piff_does_money_make_you_mean
I'd again mention the ongoing replication crisis in psychology.
People have specifically attempted to replicate Paul Piff's numerous studies trying to assign various negative characteristics to those of upper social class. The results did find some correlations maintained, such as those between greed and unethical behavior, but Piff's keystone correlation between social class and unethical behavior was completely unable to be replicated.
I do not think people understand how bad of shape psychology is in right now, social psychology in particular. Social psychology replication studies tend to be hitting around 20%. For instance once again the Journal of Personality and Social Psychology comes up. It's interestingly enough the journal that published the study that this link is discussing, as well as the journal that Piff published in. Its current replication rate is 23%. [2]
[1] - https://www.collabra.org/article/10.1525/collabra.166/
[2] - https://en.wikipedia.org/wiki/Replication_crisis#Psychology_...
People have specifically attempted to replicate Paul Piff's numerous studies trying to assign various negative characteristics to those of upper social class. The results did find some correlations maintained, such as those between greed and unethical behavior, but Piff's keystone correlation between social class and unethical behavior was completely unable to be replicated.
I do not think people understand how bad of shape psychology is in right now, social psychology in particular. Social psychology replication studies tend to be hitting around 20%. For instance once again the Journal of Personality and Social Psychology comes up. It's interestingly enough the journal that published the study that this link is discussing, as well as the journal that Piff published in. Its current replication rate is 23%. [2]
[1] - https://www.collabra.org/article/10.1525/collabra.166/
[2] - https://en.wikipedia.org/wiki/Replication_crisis#Psychology_...
> I do not think people understand how bad of shape psychology is in right now, social psychology in particular. Social psychology replication studies tend to be hitting around 20%.
Yes I think we do, at least on HN, because anytime any psychology study or result is mentioned at least two threads popup up citing the replication crisis.
The thing is, it kind of depends on how you define replication, also some of these studies were done decades ago, society has changed since then, possibly affecting the results.
How easy would it be to replicate the bystander apathy studies where people were in a smoke filled room after 9/11, for example?¹
It definitely should be taken into account, but it's much easier to tear down than to give a critical appraisal.
I don't mean to criticize your comment in particular, because it was actually a good appraisal of this particular body of work.
I do think HN has some biases² and the tendency to shout "replication crisis" in a self satisfied manner is one of the things that bug me the most about HN.³
1: https://psycnet.apa.org/record/1969-03938-001
2: No politics, but in human affairs, everything is politics.
3: Especially since most of us on here are from the hard sciences, which ironically are easier because they deal in concrete, measurable phenomena.
Yes I think we do, at least on HN, because anytime any psychology study or result is mentioned at least two threads popup up citing the replication crisis.
The thing is, it kind of depends on how you define replication, also some of these studies were done decades ago, society has changed since then, possibly affecting the results.
How easy would it be to replicate the bystander apathy studies where people were in a smoke filled room after 9/11, for example?¹
It definitely should be taken into account, but it's much easier to tear down than to give a critical appraisal.
I don't mean to criticize your comment in particular, because it was actually a good appraisal of this particular body of work.
I do think HN has some biases² and the tendency to shout "replication crisis" in a self satisfied manner is one of the things that bug me the most about HN.³
1: https://psycnet.apa.org/record/1969-03938-001
2: No politics, but in human affairs, everything is politics.
3: Especially since most of us on here are from the hard sciences, which ironically are easier because they deal in concrete, measurable phenomena.
One other thing to keep in mind about social psychology:
http://www.spsp.org/news-center/blog/stereotype-accuracy-res...
> Stereotype accuracy is one of the largest and most replicable effects in all of social psychology. Richard et al (2003) found that fewer than 5% of all effects in social psychology exceeded r’s of .50. In contrast, nearly all consensual stereotype accuracy correlations and about half of all personal stereotype accuracy correlations exceed .50.
That is, stereotypes are frequently better predictors of human behavior than bona-fide research in social psychology.
http://www.spsp.org/news-center/blog/stereotype-accuracy-res...
> Stereotype accuracy is one of the largest and most replicable effects in all of social psychology. Richard et al (2003) found that fewer than 5% of all effects in social psychology exceeded r’s of .50. In contrast, nearly all consensual stereotype accuracy correlations and about half of all personal stereotype accuracy correlations exceed .50.
That is, stereotypes are frequently better predictors of human behavior than bona-fide research in social psychology.
[deleted]
> The thing is, it kind of depends on how you define replication, also some of these studies were done decades ago, society has changed since then, possibly affecting the results.
If a study doesn't replicate because society has changed, and so the results, then that study cannot be used to infer things about current society. It belongs to historical studies at best, at worst it was garbage from the get-go (I suspect the latter is usually the case; human psychology seems mostly fixed).
Yes, hard sciences are easier; the physical reality keeps you honest. But that only means soft sciences need even stricter standards than hard ones, whereas today the opposite is the case. There's little point in spending all this effort if almost all of it is wasted - we need to accept that psychology is hard, and meaningful results will be rare.
If a study doesn't replicate because society has changed, and so the results, then that study cannot be used to infer things about current society. It belongs to historical studies at best, at worst it was garbage from the get-go (I suspect the latter is usually the case; human psychology seems mostly fixed).
Yes, hard sciences are easier; the physical reality keeps you honest. But that only means soft sciences need even stricter standards than hard ones, whereas today the opposite is the case. There's little point in spending all this effort if almost all of it is wasted - we need to accept that psychology is hard, and meaningful results will be rare.
> If a study doesn't replicate because society has changed, and so the results, then that study cannot be used to infer things about current society.
Not necessarily, again it depends what you mean by replication.
For example in the case of the bystander apathy experiment where they used smoke, do you have to replicate the study exactly as it was done, i.e.: is smoke essential, or would some other sign of threat also do?
This is why I chose that example, if right after 9/11 you tried to replicate that particular study in the US and used smoke, it probably wouldn't replicate, because people would have been recently and strongly primed to react to smoke as a threat.
I think people are over-interpreting the replication failures sometimes.
Not necessarily, again it depends what you mean by replication.
For example in the case of the bystander apathy experiment where they used smoke, do you have to replicate the study exactly as it was done, i.e.: is smoke essential, or would some other sign of threat also do?
This is why I chose that example, if right after 9/11 you tried to replicate that particular study in the US and used smoke, it probably wouldn't replicate, because people would have been recently and strongly primed to react to smoke as a threat.
I think people are over-interpreting the replication failures sometimes.
9/11 was an exceptional circumstance. You could try and replicate that study again, some time after it - e.g. now. If the study still won't replicate, that's evidence for it being bogus. Single replication failure isn't necessarily damning for a study, much less the entire field. But if most studies keep on not replicating, then you have a problem.
>hard sciences
Yes, i have often said (including here on HN) that we should refer to them as the easy sciences, or less incendiary, the concrete sciences.
The question that lingers in my mind about the replication issue is perhaps the empirical process itself is ill suited to psychology. I think the replication crisis might be a crisis of process as much as it is of incentives.
Empiricism seeks repeatable phenomena, same inputs, same outputs. In our psychological, virtual worlds, we have different inputs, different outputs, but are looking for similar processes. (Your example is a good one, in that the underlying process is a common experience but the lack of replicability of the study does not invalidate the process, just the input-ouput.) Maybe we need another model, or another layer, of empiricism in psychology.. I would characterise it as a sort of computational model.
I am reminded of the study where the neurologists try to reverse engineer a computer program by looking at a computer.[0] I can't help but wonder how psychologists would approach the same problem.
[0] cannot find it at the moment, but admittedly do like the analogy in that it is suggestive of the difficulty of understanding a complex system outside of its abstraction layer.
Yes, i have often said (including here on HN) that we should refer to them as the easy sciences, or less incendiary, the concrete sciences.
The question that lingers in my mind about the replication issue is perhaps the empirical process itself is ill suited to psychology. I think the replication crisis might be a crisis of process as much as it is of incentives.
Empiricism seeks repeatable phenomena, same inputs, same outputs. In our psychological, virtual worlds, we have different inputs, different outputs, but are looking for similar processes. (Your example is a good one, in that the underlying process is a common experience but the lack of replicability of the study does not invalidate the process, just the input-ouput.) Maybe we need another model, or another layer, of empiricism in psychology.. I would characterise it as a sort of computational model.
I am reminded of the study where the neurologists try to reverse engineer a computer program by looking at a computer.[0] I can't help but wonder how psychologists would approach the same problem.
[0] cannot find it at the moment, but admittedly do like the analogy in that it is suggestive of the difficulty of understanding a complex system outside of its abstraction layer.
> Yes, i have often said (including here on HN) that we should refer to them as the easy sciences, or less incendiary, the concrete sciences.
They are hard as in unyielding, not hard as in not easy. So hard sciences are hard because their results are easy to replicate.
They are hard as in unyielding, not hard as in not easy. So hard sciences are hard because their results are easy to replicate.
English has many terms with multiple meanings which we use with the express intent to imbue our words with adjacent meaning. A hard decision is difficult but also unyielding. A hard man is unyielding but also strong. A strong decision is controversial but also tough.
These dual meanings are often so ingrained we cannot see the inherent issues with their usage. They are often somewhat Orwellian in their duality. The hard sciences is one of those terms. Hence my joke about the easy sciences.
These dual meanings are often so ingrained we cannot see the inherent issues with their usage. They are often somewhat Orwellian in their duality. The hard sciences is one of those terms. Hence my joke about the easy sciences.
We are just animals, if someone in a low social tier is constantly blocked and fed negative affirmations, while someone from a higher tier is constantly granted access and positive affirmations, one is being trained to feel more confident and capable.
If they dropped me into a town with a bunch of actors who all pretended to be in awe of my intellect, I would begin to think of myself as some kind of genius.
If they dropped me into a town with a bunch of actors who all pretended to be in awe of my intellect, I would begin to think of myself as some kind of genius.
And you might actually become smarter. Ones ability to do some things - including mental tasks - is affected by their beliefs about their ability. That's not to say you can do anything you believe you can, just that belief can influence (not determine) ability.
What you’re describing is the amazing benefit of both blind faith and confidence. There’s jokes about piano teachers teaching kids extremely difficult classical sheets and the kids nailing it. Then someone says “how do you get them to play such difficult music?”
“Stfu, don’t tell them it’s difficult.”
The older I get the more I realize most things are not difficult. I make them more difficult than they are. If you’re in an environment where someone encourages you that “you can nail this on the first or second try”, you probably will. Most of my work is me winging it. Especially when others fail. In things I really don’t have any civilized justifiable reason to even attempt. But I, I don’t know, just do things. Mostly with research. But yea. I started to find out, in the past... 5 years? ... that a lot of silver spoons feel the same way. They just do things. And it works for them because they don’t have the emotion of difficult or easy.
That’s probably the lesson in all of this. Do things. Don’t worry about easy, difficult, credentials or sobriety.
“Stfu, don’t tell them it’s difficult.”
The older I get the more I realize most things are not difficult. I make them more difficult than they are. If you’re in an environment where someone encourages you that “you can nail this on the first or second try”, you probably will. Most of my work is me winging it. Especially when others fail. In things I really don’t have any civilized justifiable reason to even attempt. But I, I don’t know, just do things. Mostly with research. But yea. I started to find out, in the past... 5 years? ... that a lot of silver spoons feel the same way. They just do things. And it works for them because they don’t have the emotion of difficult or easy.
That’s probably the lesson in all of this. Do things. Don’t worry about easy, difficult, credentials or sobriety.
Chess is probably the clearest example of this. Chess is not especially difficulty to become quite strong at - even as an adult. But there is an important catch there. Your improvement is nonlinear and difficult to predict. You can spend 6 months working tirelessly at the game and see no improvement. One extremely typical pattern in it is 'stair stepping.' You don't gradually increase in proportion to your efforts, you instead go up in spikes. You wake up and suddenly you're e.g. 100 points stronger than you were than when you went to bed.
This goes all the way to the top. Magnus Carlsen is currently the world champion, one of the most dominant we've had in a long time, and arguably the strongest player to ever live. And he had phenomenal success. He achieved his grandmaster title at age 13 and was nearly the youngest grandmaster ever. But here's [1] what's interesting. That's a graph of his rating progress - a much more readable table is at the bottom. Carlsen was advancing meteorically rapidly. Looking at his relative rating gains between each period:
---
10/2001: Baseline (2072)
10/2002: +178
10/2003: +200
10/2004: +131
Then, as always, it happens:
10/2005: -11
---
A player meteorically rising and spending immense amounts of energy on the game suddenly manages to achieve nothing but an 11 point loss over more than a year. Imagine if you, as an adult, were in the position and in spite of all your efforts over the past year - you did nothing but lose 11 points. I've coached people in that exact situation. And they tend to rapidly give up. Oh, they just don't have the IQ for it. Maybe they're too old. Maybe they're doing something wrong and start engaging in genuinely wrong study habits. And so on.
These 'plateau' type events happen to literally all chess players. So it leads to the interesting thing that you'll find most chess players are both humble and arrogant. Humble as years of sport against much better players (as is required for improvement) will make you, but arrogant in the sense that they will never say never - even when things look incredibly grim. In effect you end up simultaneously humble and arrogant. I would expect that successful entrepreneurs and successful chess players share many of the exact same characteristics, as it's the same game in both endeavors.
[1] - https://ratings.fide.com/id.phtml?event=1503014
This goes all the way to the top. Magnus Carlsen is currently the world champion, one of the most dominant we've had in a long time, and arguably the strongest player to ever live. And he had phenomenal success. He achieved his grandmaster title at age 13 and was nearly the youngest grandmaster ever. But here's [1] what's interesting. That's a graph of his rating progress - a much more readable table is at the bottom. Carlsen was advancing meteorically rapidly. Looking at his relative rating gains between each period:
---
10/2001: Baseline (2072)
10/2002: +178
10/2003: +200
10/2004: +131
Then, as always, it happens:
10/2005: -11
---
A player meteorically rising and spending immense amounts of energy on the game suddenly manages to achieve nothing but an 11 point loss over more than a year. Imagine if you, as an adult, were in the position and in spite of all your efforts over the past year - you did nothing but lose 11 points. I've coached people in that exact situation. And they tend to rapidly give up. Oh, they just don't have the IQ for it. Maybe they're too old. Maybe they're doing something wrong and start engaging in genuinely wrong study habits. And so on.
These 'plateau' type events happen to literally all chess players. So it leads to the interesting thing that you'll find most chess players are both humble and arrogant. Humble as years of sport against much better players (as is required for improvement) will make you, but arrogant in the sense that they will never say never - even when things look incredibly grim. In effect you end up simultaneously humble and arrogant. I would expect that successful entrepreneurs and successful chess players share many of the exact same characteristics, as it's the same game in both endeavors.
[1] - https://ratings.fide.com/id.phtml?event=1503014
100% agree. The other thing they don't do is worry about failure. Mistakes are a necessary part of learning. Screw up? Fix it and move on. The last month or two I've been deliberately doing things that I'm hesitant to do, and it's really starting to sink in just how important all of this is.
My son often talks of an animal, a dingo iirc, that wasn't socialized normally. Maybe it was orphaned and raised by humans?
Dominance behaviors in the species are hard wired biologically, but submissive behavior was learned behavior, so the animal in question never learned things like how ho back down from a fight. So the animal stood its ground, even when outnumbered and so forth. It simply had no other option, having not ever learned any of the not hard wired behaviors necessary to do something else.
It ended up in charge of the pack.
Dominance behaviors in the species are hard wired biologically, but submissive behavior was learned behavior, so the animal in question never learned things like how ho back down from a fight. So the animal stood its ground, even when outnumbered and so forth. It simply had no other option, having not ever learned any of the not hard wired behaviors necessary to do something else.
It ended up in charge of the pack.
I believe the same thing happened at IBM in the 90s.
The methodology seems highly questionable. From the actual study (https://www.apa.org/pubs/journals/releases/psp-pspi0000187.p...)
"Social class was measured in three ways. To indicate subjective social class, participants saw an image of a ladder that 'represents where people stand in your country.' and selected the rung on which they felt they stood, relative to other people in their country"
"However, when difference scores were used (or the alternative joint-testing procedure), the conclusions were not so straightforward. Here, we found support for Hypothesis 1 only in the case of subjective social class. The difference score approach also showed that education and income were negatively associated with overconfidence, directly contrasting the results generated by the residual score approach."
So it's not so much that people from a higher social class are overconfident but that overconfident people appear to be from a higher social class due to their methodology and downstream assessment confirms their methodological bias. It's quite incredible this received as much attention as it did. Also this sample is limited to Mexican small-business owners applying for loans.
"Social class was measured in three ways. To indicate subjective social class, participants saw an image of a ladder that 'represents where people stand in your country.' and selected the rung on which they felt they stood, relative to other people in their country"
"However, when difference scores were used (or the alternative joint-testing procedure), the conclusions were not so straightforward. Here, we found support for Hypothesis 1 only in the case of subjective social class. The difference score approach also showed that education and income were negatively associated with overconfidence, directly contrasting the results generated by the residual score approach."
So it's not so much that people from a higher social class are overconfident but that overconfident people appear to be from a higher social class due to their methodology and downstream assessment confirms their methodological bias. It's quite incredible this received as much attention as it did. Also this sample is limited to Mexican small-business owners applying for loans.
> It's quite incredible this received as much attention as it did.
Almost any piece of western media tries to paint higher classes negatively, so no, it's the opposite of incredible.
Almost any piece of western media tries to paint higher classes negatively, so no, it's the opposite of incredible.
An exaggerated belief of performance in a flash card game might not indicate that belief is exaggerated in general. Maybe they have a heuristic like "I'm good at things" which gives a pretty accurate estimate of their performance on day-to-day tasks, and flash card games are an edge case that they're not good at estimating.
I don’t know if it’s good heuristic of performance or not. Everyone has to be judged individually in this case. I do think that it is a measure of self-confidence, which may effect performance, and certainly risk taking.
I think about what Guido van Rossum said recently[0]. He’s seen incompetent men’s ideas gain acceptance more often than merited because they are more forceful in how they present them. I’m sure if you asked the incompetent man why his idea was chosen, he’d say it was because it was the best technically, and not because of some lesser reason, like politics.
[0] https://qz.com/1624252/pythons-creator-thinks-it-has-a-diver...
I think about what Guido van Rossum said recently[0]. He’s seen incompetent men’s ideas gain acceptance more often than merited because they are more forceful in how they present them. I’m sure if you asked the incompetent man why his idea was chosen, he’d say it was because it was the best technically, and not because of some lesser reason, like politics.
[0] https://qz.com/1624252/pythons-creator-thinks-it-has-a-diver...
Please. Money lies. We all know it. People with money are deferred to, emotionally coddled, and ALWAYS given the benefit of the doubt. It is literally the foundation of Western culture. Those who have always had it live in a very delusional world that has always lied to them about their abilities. That's how we end up with a " very stable genius" as the "leader" of the western world.
But the noble poor man (or the noble savage if we look to the past) never cheats, lies, steals, etc. And if they do it’s for survival or family or some other sympathetic cause.
Or we’re all humans with normal human traits and egos to rationalize just about anything given the right circumstances.
Or we’re all humans with normal human traits and egos to rationalize just about anything given the right circumstances.
Nobody said they didn't. Point was that people with "old" money who have always been of a higher social class than most believe falsley that they are better than they are, for the previously stated reasons. Yes, they are humans, and that's what happens to humans who are given too much. One of the many horrid things that happens to normal humans.
[deleted]
Yeah, money lies. Overestimating your ability on a memory retention task might not be indicative that money lies, however.
But it might, hence the topic.
There is a replication crisis [1] in the social sciences and most studies are not being replicated. [2]
Here is a study that is not able to replicate the most referenced study on subsaharan and Africa iq [3]. The tests were done on uneducated people, handicapped people, people in remote areas and in poor conditions ie under trees which is not the recommended test procedure. Higher scores were also intentionally dumped in favour of poorer scores for vague reasons with evidence of data massaging.
This kind of 'science' is extremely damaging especially when cherry picked results and sweeping conclusions that these kind of studies do not and cannot support are widely cited outside the scientific context by bigots and racists to construct a narrative that dehumanize others. The nobel laurette James Watson expressed 'gloom about Africa' on the basis of these studies.
There are a lot of well known funds like the pioneer fund [1] and volker fund that have spent tens of decades on 'race' science and that formed a lot of basis for Charles Murray's bell curve. There are not hundreds of heavily funded organizations in Africa and Asia trying to prove others are somehow lower iq or 'inferior'. This effort has been ongoing for over 250 years, first it was brain size and now its iq and evolutionary psychology. What exactly is iq measuring, can we measure something that we don't understand?
[1] https://en.wikipedia.org/wiki/Replication_crisis
[2] https://www.vox.com/science-and-health/2018/8/27/17761466/ps...
[3] http://www.iapsych.com/iqmr/fe/LinkedDocuments/wicherts2010....
[4] https://www.sourcewatch.org/index.php/Pioneer_Fund
Here is a study that is not able to replicate the most referenced study on subsaharan and Africa iq [3]. The tests were done on uneducated people, handicapped people, people in remote areas and in poor conditions ie under trees which is not the recommended test procedure. Higher scores were also intentionally dumped in favour of poorer scores for vague reasons with evidence of data massaging.
This kind of 'science' is extremely damaging especially when cherry picked results and sweeping conclusions that these kind of studies do not and cannot support are widely cited outside the scientific context by bigots and racists to construct a narrative that dehumanize others. The nobel laurette James Watson expressed 'gloom about Africa' on the basis of these studies.
There are a lot of well known funds like the pioneer fund [1] and volker fund that have spent tens of decades on 'race' science and that formed a lot of basis for Charles Murray's bell curve. There are not hundreds of heavily funded organizations in Africa and Asia trying to prove others are somehow lower iq or 'inferior'. This effort has been ongoing for over 250 years, first it was brain size and now its iq and evolutionary psychology. What exactly is iq measuring, can we measure something that we don't understand?
[1] https://en.wikipedia.org/wiki/Replication_crisis
[2] https://www.vox.com/science-and-health/2018/8/27/17761466/ps...
[3] http://www.iapsych.com/iqmr/fe/LinkedDocuments/wicherts2010....
[4] https://www.sourcewatch.org/index.php/Pioneer_Fund
[deleted]
[deleted]
The paper that this article is discussing was published in The Journal of Personality and Social Psychology. That's interesting for two reasons. The first is that it's a rather well regarded journal with a very high impact rate. The other is because it's one of the journals that's driving the ongoing replication crisis plaguing psychology. [1]
In particular the journal this was published in was found to have an overall replication rate of 23%. Another way of putting this is that if you assumed the exact opposite of everything you read that was published in that journal (e.g. - what they claim is statistically significant, is not actually significant) you'd be vastly more informed than a person who took what was published in the journal at face value.
Of course that does not mean that this study is 'false', but you should take it with the same degree of credulity that you'd take something published in a venue when you know 77% of what is published is false. I mention this largely because I think there's a substantial degree of dissonance between the crisis in psychology and people continuing to take studies in psychology at face value. It's illogical.
[1] - https://en.wikipedia.org/wiki/Replication_crisis#Psychology_...
In particular the journal this was published in was found to have an overall replication rate of 23%. Another way of putting this is that if you assumed the exact opposite of everything you read that was published in that journal (e.g. - what they claim is statistically significant, is not actually significant) you'd be vastly more informed than a person who took what was published in the journal at face value.
Of course that does not mean that this study is 'false', but you should take it with the same degree of credulity that you'd take something published in a venue when you know 77% of what is published is false. I mention this largely because I think there's a substantial degree of dissonance between the crisis in psychology and people continuing to take studies in psychology at face value. It's illogical.
[1] - https://en.wikipedia.org/wiki/Replication_crisis#Psychology_...
> if you assumed the exact opposite of everything you read that was published in that journal ... you'd be vastly more informed
I don't think the opposite of not knowing is knowing (the opposite fact) :-)
That said, I agree with your main point on an even wider level. I read such very general headlines and conclusions - and when I read the actual paper there is a very specific and very limited experiment, with very specific people in a very specific context. I'm always left wondering how one gets from such a specific thing to such generalizations, especially when it's about such a complex subject as human behavior where tiny variations can produce radically different outcomes.
---
PS: Animats says it's paywalled, but the PDF on the linked page is accessible and has the data and details? About the N=150,949 part:
> EFL collected the data from small-business owners at the time they were applying for a loan at a microfinance institution (MFI) in Mexico from 2015 to 2017. As part of the loan application process, a prospective borrower had to complete a short psychometric assessment developed by EFL.
That's just what I said, it's very specific.
> The psychometric assessment that EFL developed consists of many modules, one of which is relevant for the purposes of the present investigation: the flashcard game. The flashcard game is a cognitive test on memory and executive functioning. In the flashcard game, participants are presented with an image; after pressing a key, they are shown a second image. Participants are then asked to indicate whether the second image matches the first. The flashcard game is scored based on whether or not the participant correctly identifies a match.
and
> At the end of the flashcard exercise, participants saw a question that asked them to estimate how well they think they did on this particular exercise relative to other applicants.... This question was added to the psychometric application for the purpose of this study
I don't know about others, but I have a hard time seeing such tests justifying the headlines.
I don't think the opposite of not knowing is knowing (the opposite fact) :-)
That said, I agree with your main point on an even wider level. I read such very general headlines and conclusions - and when I read the actual paper there is a very specific and very limited experiment, with very specific people in a very specific context. I'm always left wondering how one gets from such a specific thing to such generalizations, especially when it's about such a complex subject as human behavior where tiny variations can produce radically different outcomes.
---
PS: Animats says it's paywalled, but the PDF on the linked page is accessible and has the data and details? About the N=150,949 part:
> EFL collected the data from small-business owners at the time they were applying for a loan at a microfinance institution (MFI) in Mexico from 2015 to 2017. As part of the loan application process, a prospective borrower had to complete a short psychometric assessment developed by EFL.
That's just what I said, it's very specific.
> The psychometric assessment that EFL developed consists of many modules, one of which is relevant for the purposes of the present investigation: the flashcard game. The flashcard game is a cognitive test on memory and executive functioning. In the flashcard game, participants are presented with an image; after pressing a key, they are shown a second image. Participants are then asked to indicate whether the second image matches the first. The flashcard game is scored based on whether or not the participant correctly identifies a match.
and
> At the end of the flashcard exercise, participants saw a question that asked them to estimate how well they think they did on this particular exercise relative to other applicants.... This question was added to the psychometric application for the purpose of this study
I don't know about others, but I have a hard time seeing such tests justifying the headlines.
I'm always left wondering how one gets from such a specific thing to such generalizations
In this case, I will suggest that someone is simply marrying general "common knowledge" to the study specifics.
Classism is very real. I've become very aware of its effects in recent years.
My mother's mother came from a low level noble family that sold the title. I grew up in a nice suburb, daughter of a retired decorated soldier. I was one of the top people in my high school, arguably one of the top people in my state academically, based on placing 3rd in a statewide competition and being a National Merit Scholarship winner.
In recent years, I've been quite poor and spent several years homeless. My social experience suggests that my recent poverty completely wipes out the value of my academic accomplishments etc in the eyes of a great many people.
It's been a huge problem for trying to get my life back. In the eyes of many people, I'm too poor to be considered competent enough to hire for anything.
In this case, I will suggest that someone is simply marrying general "common knowledge" to the study specifics.
Classism is very real. I've become very aware of its effects in recent years.
My mother's mother came from a low level noble family that sold the title. I grew up in a nice suburb, daughter of a retired decorated soldier. I was one of the top people in my high school, arguably one of the top people in my state academically, based on placing 3rd in a statewide competition and being a National Merit Scholarship winner.
In recent years, I've been quite poor and spent several years homeless. My social experience suggests that my recent poverty completely wipes out the value of my academic accomplishments etc in the eyes of a great many people.
It's been a huge problem for trying to get my life back. In the eyes of many people, I'm too poor to be considered competent enough to hire for anything.
I think few people would disagree with you. But this is really the biggest danger. If you look at the studies that have been let's kindly say 'found to be less than well supported', you'd also find many that you would intuitively agree with the outcome of, or at least want to agree with. Psychology, and social psychology in particular, has a habit of confirming our biases and making us feel good. This, in turn, takes us off our guard for bad science. And by "us", I think this is something that's likely even issue within the field itself. You don't get a 23% replication rate without there being some very severe problems within a field.
Take for contrast, astrology. We all now know it's obviously fake. It even relies upon things we know are wrong, such as assuming planets can magically stop and start going the other way in their orbital trajectory. That assumption was based on an optical illusion driven by the assumption that everything revolved around the Earth. But for hundreds of years astrology was treated as respectable a science as any other. And I think it was for a similar reason. So compelling and entertaining are the correlations for astrology that even today you have millions, if not billions, of people that still partake in it to varying degrees.
The psychology replication crisis is, in my opinion, hurting science as a whole. We should not indulge bad science even if it happens to tell us what we want to hear. It's making a mockery of what science is truly about.
Take for contrast, astrology. We all now know it's obviously fake. It even relies upon things we know are wrong, such as assuming planets can magically stop and start going the other way in their orbital trajectory. That assumption was based on an optical illusion driven by the assumption that everything revolved around the Earth. But for hundreds of years astrology was treated as respectable a science as any other. And I think it was for a similar reason. So compelling and entertaining are the correlations for astrology that even today you have millions, if not billions, of people that still partake in it to varying degrees.
The psychology replication crisis is, in my opinion, hurting science as a whole. We should not indulge bad science even if it happens to tell us what we want to hear. It's making a mockery of what science is truly about.
I think few people would disagree with you. But this is really the biggest danger.
I was merely describing how such happens, not defending the process.
Take for contrast, astrology. We all now know it's obviously fake.
You happen to be speaking to someone who has studied astrology in earnest and believes it has some merit. Astrology is not immune to the rubric "90% of everything is crap." "Sun sign" or "popular" astrology is often quite badly done, but serious astrology is highly mathematical and the practice of marking retrogrades is a perfectly accurate assessment of where those planets appear in relation to the earth because it is a geocentric model, having nothing whatsoever to do with a scientific belief in a geocentric model rather than a solar system.
Astrology is part of the spiritual beliefs of many people. It wouldn't be okay to casually suggest that Christians are nutters for believing Jesus rose from the dead.
I was merely describing how such happens, not defending the process.
Take for contrast, astrology. We all now know it's obviously fake.
You happen to be speaking to someone who has studied astrology in earnest and believes it has some merit. Astrology is not immune to the rubric "90% of everything is crap." "Sun sign" or "popular" astrology is often quite badly done, but serious astrology is highly mathematical and the practice of marking retrogrades is a perfectly accurate assessment of where those planets appear in relation to the earth because it is a geocentric model, having nothing whatsoever to do with a scientific belief in a geocentric model rather than a solar system.
Astrology is part of the spiritual beliefs of many people. It wouldn't be okay to casually suggest that Christians are nutters for believing Jesus rose from the dead.
skookumchuck(1)
Oh I don't want to doubt the result, I think it's probably true. I even posted this on my FB account (which I only use to bookmark and possibly spread "sciency" links I find interesting, not for anything personal) - but then deleted it again. Because I have to admit as much as I personally like the result because it confirms my personal believes, I find the foundation the headline is based on shaky ground given that it's supposed to be science and not opinion. I can't complain about others if I find my own bias influences whether I ignore how results are obtained and just spread the headline hoping (and knowing) that other people (following my news feed) won't read the details.
I'm pretty socially observant and my adult sons are rather aspie. In raising them, I had to really up my game for trying to find data and explanations for social things.
I once had a conversation with my oldest where I tried to say something like "anger is like fire" as a metaphor and he leapt to some conclusion like "so, you rub two people together and they explode?"
Um, no. Forget that metaphor. Let me get back to you.
So we spend a lot of time talking about both social things and how to find actually good information and data because psych studies are so often terribly done and they are inherently hard to do well because most studies are de facto a form of manipulation, which itself pollutes the data.
I once had a conversation with my oldest where I tried to say something like "anger is like fire" as a metaphor and he leapt to some conclusion like "so, you rub two people together and they explode?"
Um, no. Forget that metaphor. Let me get back to you.
So we spend a lot of time talking about both social things and how to find actually good information and data because psych studies are so often terribly done and they are inherently hard to do well because most studies are de facto a form of manipulation, which itself pollutes the data.
skookumchuck(2)
The paper looks more interesting, but it's paywalled.[1] The study involved over 150,000 people, which is impressive. Too many psych studies get subjects from undergrad students, or worse, Mechanical Turk.
Here's the first author's web page.[2] He seems to specialize in this area. Unfortunately, none of his papers listed link to the full text.
[1] https://psycnet.apa.org/doiLanding?doi=10.1037%2Fpspi0000187
[2] https://www.darden.virginia.edu/faculty-research/directory/p...
Here's the first author's web page.[2] He seems to specialize in this area. Unfortunately, none of his papers listed link to the full text.
[1] https://psycnet.apa.org/doiLanding?doi=10.1037%2Fpspi0000187
[2] https://www.darden.virginia.edu/faculty-research/directory/p...
https://sci-hub.tw/#about for bypassing paywalls.
But I would not expect to find any huge red flag. This journal, in spite of having an absolutely abysmal replication rate, is one of the leading journals in psychology. It's not like the 77% of studies that can't be replicated were just obviously broken in some way. It's simply that when people try to do the same thing, the results do not hold. Though, in this case at least a peer comment provided some more insight into the 150k.
But I would not expect to find any huge red flag. This journal, in spite of having an absolutely abysmal replication rate, is one of the leading journals in psychology. It's not like the 77% of studies that can't be replicated were just obviously broken in some way. It's simply that when people try to do the same thing, the results do not hold. Though, in this case at least a peer comment provided some more insight into the 150k.
Keep in mind that the paper does clearly state:
>Furthermore, all three forms of social class were positively associated with actual rank, indicating that those with relatively high social class did objectively better on the flashcard game relative to their lower-class counterparts
This is a poor title and article if you take even a brief moment to view the actual correlations presented in the paper.