Socioeconomic status and the relationship between the SAT and GPA (2009) [pdf](files.eric.ed.gov)
files.eric.ed.gov
Socioeconomic status and the relationship between the SAT and GPA (2009) [pdf]
https://files.eric.ed.gov/fulltext/ED562860.pdf
75 comments
Are standardized tests an accurate measure of intelligence? IMO, they're more a measure of knowledge and perseverance. Having said that, I don't know what is the best measure of intelligence.
There is certainly a genetic component to intelligence, but environmental factors have a big role to play as well.
There is certainly a genetic component to intelligence, but environmental factors have a big role to play as well.
> Are standardized tests an accurate measure of intelligence? IMO, they're more a measure of knowledge and perseverance.
Depends on the standardized test. Some are designed to test knowledge, others are designed to test more fundamental attributes.
ACT is more knowledge, SAT is more fundamental attributes. SAT is still a significant amount of knowledge, but not as focused on it as ACT.
Depends on the standardized test. Some are designed to test knowledge, others are designed to test more fundamental attributes.
ACT is more knowledge, SAT is more fundamental attributes. SAT is still a significant amount of knowledge, but not as focused on it as ACT.
I think the issue is that number 5 is not true, at least not in a direct way.
Look at any family of 3+ siblings. Intelligence tends to vary significantly despite having the same parents.
Look at any family of 3+ siblings. Intelligence tends to vary significantly despite having the same parents.
Anecdata, but yes, I am not the smartest sibling in my family, but one of the wealthier ones.
The "dumbest" one is potentially the wealthiest. The smartest one (hands down) is the least wealthy.
This is besides the point though. Anecdata doesn't get at what happens on average, and that's what his post is about.
The "dumbest" one is potentially the wealthiest. The smartest one (hands down) is the least wealthy.
This is besides the point though. Anecdata doesn't get at what happens on average, and that's what his post is about.
> Anecdata doesn't get at what happens on average, and that's what his post is about.
From what I've read, the inheritability of intelligence has a 0.50 correlation. That's not fantastic.
From what I've read, the inheritability of intelligence has a 0.50 correlation. That's not fantastic.
This is the Simpson paradox in action. It’s possible for smart parents to get less smart kids, or to observe a wide variation in smartness among siblings, but neither of those say anything about whether on average, smarter parents get smarter kids.
Genetics is a grab bag of traits. A whole bunch of marbles mixed around in a bag.
What I am speaking to is the types of marbles and their prevalence in a given bag. Not what is actually chosen in any given drawing.
A better phrasing would be "Kids who draw from bags with higher than average amounts of green marbles, are more likely to end up with green marbles in their bag"
What I am speaking to is the types of marbles and their prevalence in a given bag. Not what is actually chosen in any given drawing.
A better phrasing would be "Kids who draw from bags with higher than average amounts of green marbles, are more likely to end up with green marbles in their bag"
>The emperor is walking around naked, and virtually all of academia is terrified to acknowledge it (namely step 5). Right there is the solution to 50 years of sociological fumbling to avoid saying "Intelligence has a strong genetic component, and we are not all born with equal mental capabilities"
Bien-pensants will tell us to the bitter end "Who are you going to believe, me or your lying eyes?" But they are aware of the truth.
Example: From a 2018 open letter by an eminent geneticist, "How Genetics Is Changing Our Understanding of 'Race'" <https://www.nytimes.com/2018/03/23/opinion/sunday/genetics-r...>:
>I am worried that well-meaning people who deny the possibility of substantial biological differences among human populations are digging themselves into an indefensible position, one that will not survive the onslaught of science. I am also worried that whatever discoveries are made — and we truly have no idea yet what they will be — will be cited as “scientific proof” that racist prejudices and agendas have been correct all along, and that those well-meaning people will not understand the science well enough to push back against these claims.
Bien-pensants will tell us to the bitter end "Who are you going to believe, me or your lying eyes?" But they are aware of the truth.
Example: From a 2018 open letter by an eminent geneticist, "How Genetics Is Changing Our Understanding of 'Race'" <https://www.nytimes.com/2018/03/23/opinion/sunday/genetics-r...>:
>I am worried that well-meaning people who deny the possibility of substantial biological differences among human populations are digging themselves into an indefensible position, one that will not survive the onslaught of science. I am also worried that whatever discoveries are made — and we truly have no idea yet what they will be — will be cited as “scientific proof” that racist prejudices and agendas have been correct all along, and that those well-meaning people will not understand the science well enough to push back against these claims.
I have no interest in bringing race into this because of the wide genetic difference between groups that common people would label as being the same "race". Genetically there is wide diversity even within relatively small geographical areas.
Edit: grammar
Humans and mice have 85% commonality in their DNA.
More to the point, humans have between 99.6 and 99.9 commonality in their genetics. Even more interesting (quoted directly from below) " impossible to draw discrete genetic boundaries around human groups."
Your point is not accurate.
From https://en.wikipedia.org/wiki/Race_and_genetics
> Humans are remarkably genetically similar, sharing approximately 99.6%-99.9% of their genetic code with one another. We nonetheless see wide individual variation in phenotype, which arises from both genetic differences and complex gene-environment interactions. The vast majority of this genetic variation occurs within groups; very little genetic variation differentiates between groups. Crucially, the between-group genetic differences that do exist do not map onto socially recognized categories of race. Furthermore, although human populations show some genetic clustering across geographic space, human genetic variation is "clinal", or continuous. This, in addition to the fact that different traits vary on different clines, makes it impossible to draw discrete genetic boundaries around human groups. Finally, insights from ancient DNA are revealing that no human population is "pure" – all populations represent a long history of migration and mixing.
Humans and mice have 85% commonality in their DNA.
More to the point, humans have between 99.6 and 99.9 commonality in their genetics. Even more interesting (quoted directly from below) " impossible to draw discrete genetic boundaries around human groups."
Your point is not accurate.
From https://en.wikipedia.org/wiki/Race_and_genetics
> Humans are remarkably genetically similar, sharing approximately 99.6%-99.9% of their genetic code with one another. We nonetheless see wide individual variation in phenotype, which arises from both genetic differences and complex gene-environment interactions. The vast majority of this genetic variation occurs within groups; very little genetic variation differentiates between groups. Crucially, the between-group genetic differences that do exist do not map onto socially recognized categories of race. Furthermore, although human populations show some genetic clustering across geographic space, human genetic variation is "clinal", or continuous. This, in addition to the fact that different traits vary on different clines, makes it impossible to draw discrete genetic boundaries around human groups. Finally, insights from ancient DNA are revealing that no human population is "pure" – all populations represent a long history of migration and mixing.
The book "The Bell Curve" basically expressed the same point -- with data-- as yours here. And yeah, you are right. But then everyone is terrified by it, and hence the outcryand the name calling.
Wow, I thought this book had been burned by zealots and its ashes buried long ago. Thank you for resurrecting it.
The problem with this is that it is utterly ahistorical. Like it completely ignores every observation you could make about how wealth was historically allocated.
I mean, I just laid out how you end up with wealth dynasties. And this effect is observable in any population, it's not unique to the US, much less the the West.
If you think wealthy people are particularly smart in Europe, I have a few bridges you can handily take home.
I know a lot of wealthy people, none of them are stupid. They aren't geniuses but none of them had issues in college. I know a lot of poor people from military, a lot of them are stupid and struggled with the very easy material you have to learn in the military.
And this is what statistics tells us, rich people are very rarely stupid, quite a lot of poor people are. The world isn't fair, but stupidity is such a massive drawback that it is basically impossible to get rich if you have it. Stupid people get tricked in deals everywhere etc, there is just no way for them to reach anywhere without extreme amounts of support.
And this is what statistics tells us, rich people are very rarely stupid, quite a lot of poor people are. The world isn't fair, but stupidity is such a massive drawback that it is basically impossible to get rich if you have it. Stupid people get tricked in deals everywhere etc, there is just no way for them to reach anywhere without extreme amounts of support.
My point would be that non-wealthy people in Europe are even less intelligent on the whole.
Ah, but <insert US exceptionalism here>.
The Industrial Revolution is ahistorical.
[deleted]
Except the research doesn't show this at all.
However, having wealth provides tutors, better schooling and teachers which lead to better outcomes from having better mentors, opportunities and education which leads to...
and so on and so on.
However, having wealth provides tutors, better schooling and teachers which lead to better outcomes from having better mentors, opportunities and education which leads to...
and so on and so on.
Right, that is step 6.
If you go over the research, you realize that they all assume the premise "everyone is born with an identical blank slate". Which leads to conclusions (either said outright or implied) like; "If you give enough resources to any given child, they can grow up to be anything."
So we end up with schools that serve low income areas getting enormous amounts of funding to teach admin assistants and tile layers to be heart surgeons. And the data on that end shows unequivocally that it is not working.
If you go over the research, you realize that they all assume the premise "everyone is born with an identical blank slate". Which leads to conclusions (either said outright or implied) like; "If you give enough resources to any given child, they can grow up to be anything."
So we end up with schools that serve low income areas getting enormous amounts of funding to teach admin assistants and tile layers to be heart surgeons. And the data on that end shows unequivocally that it is not working.
> If you go over the research, you realize that they all assume the premise "everyone is born with an identical blank slate".
From my experience this is a large exaggeration to not true at all. For example the Colorado adoption project has been around has been around for a longer time and was well regarded last I checked.
Are there piles of hard science papers out there that are concluding "everyone is born with an identical blank slate" that I am missing?
From my experience this is a large exaggeration to not true at all. For example the Colorado adoption project has been around has been around for a longer time and was well regarded last I checked.
Are there piles of hard science papers out there that are concluding "everyone is born with an identical blank slate" that I am missing?
> And the data on that end shows unequivocally that it is not working.
Of course throwing money at (sometimes corrupt) school boards isn’t going to be effective. There isn’t much 1/20th of a school teacher can do 8am-2pm M-F to counter a bad environment. It takes a village after all.
Not exactly data that’s useful to your hypothesis.
Of course throwing money at (sometimes corrupt) school boards isn’t going to be effective. There isn’t much 1/20th of a school teacher can do 8am-2pm M-F to counter a bad environment. It takes a village after all.
Not exactly data that’s useful to your hypothesis.
It's worth remembering this next time you hear someone say something along the lines of, "I believe in equality of opportunity, not equality of outcome". If people are indeed born with unequal abilities, then equality of opportunity is impossible.
You seem confused. If people are unequal then equality of outcome is impossible and not worth pursuing[1]. The only possible equality is equal opportunity which as you rightly point out isn’t really equal. But it’s the only possibility. Equality of outcome denies free will.
[1] edit to clarify, I mean “not worth pursuing” past the point of equilibrium. We can improve equality of outcome to a point but then will hit a plateau. Pretending that plateau doesn’t exist is one of humanities biggest problems. It’s a costly lie that doesn’t serve anyone but certain professional groups.
[1] edit to clarify, I mean “not worth pursuing” past the point of equilibrium. We can improve equality of outcome to a point but then will hit a plateau. Pretending that plateau doesn’t exist is one of humanities biggest problems. It’s a costly lie that doesn’t serve anyone but certain professional groups.
Here's my simplified way of thinking about equality at times - let's say you've worked hard all your life and have saved $2.5 million dollars - quite a nice sum. I just looked and Bill Gates stands at $128 billion. So 51,200 times more money. Do you think Bill Gates worked 51,200 times harder? Is he 51,200 times smarter? Better? Some level of inequality just seems inherently unjustifiable. I have no specific quarrel with Gates; insert any other filthy rich person and imagine it.
“Some level of inequality just seems inherently unjustifiable.“
Define justifiable. I would agree with “unfair.” Life is not fair. Is life “justifiable” to you? In nature, there are predators and prey, is that “justifiable”?
I like to simplify my thinking too, but your view seems inherently suppose almost superhuman discernment, that you (or other humans individually or collectively) can judge what is justified on a case by case basis without any concrete principals or criteria, and without bias! Doesn’t seem simple to me at all! How to distinguish the bill gateses from the other guys is a problem as old as civilization. There is more law and tax code written on this than you can ever hope to capture in your thinking. Ignoring policy and centuries of human knowledge isn’t “simple” as much as it is ignorance.
Define justifiable. I would agree with “unfair.” Life is not fair. Is life “justifiable” to you? In nature, there are predators and prey, is that “justifiable”?
I like to simplify my thinking too, but your view seems inherently suppose almost superhuman discernment, that you (or other humans individually or collectively) can judge what is justified on a case by case basis without any concrete principals or criteria, and without bias! Doesn’t seem simple to me at all! How to distinguish the bill gateses from the other guys is a problem as old as civilization. There is more law and tax code written on this than you can ever hope to capture in your thinking. Ignoring policy and centuries of human knowledge isn’t “simple” as much as it is ignorance.
Bill Gates has probably had 51200 times more impact on the world than me or other more average people. Do you disagree about that? Maybe I could have had that level of impact given the same circumstances, but probably I wouldn't, most people with his circumstances didn't. And most likely I would have stopped working the moment I had enough money, I never would have gone on to push for more and more, expand for ever greater impact, I'd be happy as a millionaire like most other people would and stop growing.
People who continue working to expand their impact even when they don't need to are rare.
People who continue working to expand their impact even when they don't need to are rare.
I think Dennis Ritchie and Linus Torvalds have had a roughly similar impact upon the world with the benefit that I also consider it a positive impact.
Expanding your area of impact, when it is no longer your technical domain, is to put it mildly, dangerous and often counterproductive.
That's not what equality of opportunity is referring to. It's about removing barriers and discrimination that would lead someone with equal or greater abilities to do worse than someone else with the same or lower abilities.
The phrase, "I believe in equality of opportunity, not equality of outcome" explicitly acknowledges that equality of opportunity will not yield equality of outcome. In part, because of the unequal distribution of talent that you point out.
The phrase, "I believe in equality of opportunity, not equality of outcome" explicitly acknowledges that equality of opportunity will not yield equality of outcome. In part, because of the unequal distribution of talent that you point out.
The use of equality of opportunity there is about making sure a kid has the chance to perform as well as they can regardless of where they happen to be born and the socioeconomic status of their parents because today where you're born has a lot to do with how good of a school you get into unless your parents are well off enough to go private or have the time and energy to get you into a magnet/charter program that can bypass the geographic destiny of property and income tax levels.
Equality of opportunity means in a strict sense external factors, in an even stricter sense things that can be changed by policy. Otherwise, it would be asking for all people to be born with the same mental processing potential. Which is absurd, which I guess is your point. But it isn't that absurd if you define opportunity as "things society can actually do for the individual".
[deleted]
Relatedly, UT Austin found the SAT highly predictive and decided to reinstate the requirement: https://news.utexas.edu/2024/03/11/ut-austin-reinstates-stan...
> Those who opted in had a median SAT score of 1420, compared with a median of 1160 among those who did not.
> The higher standardized scores translated on average to better collegiate academic performance. Of 9,217 first-year students enrolled in 2023, those who opted in had an estimated average GPA of 0.86 grade points higher during their first fall semester, controlling for a wide range of factors, including high school class rank and GPA.
> Those who opted in had a median SAT score of 1420, compared with a median of 1160 among those who did not.
> The higher standardized scores translated on average to better collegiate academic performance. Of 9,217 first-year students enrolled in 2023, those who opted in had an estimated average GPA of 0.86 grade points higher during their first fall semester, controlling for a wide range of factors, including high school class rank and GPA.
These kind of things are making me conservative on knowledge.
>10-15 years ago 'standardized testing doesnt matter at all'
>You are an idiot if you think it matters
>Oops THEY were wrong. You are still an idiot and ahole for disagreeing with me 10 years ago.
Obviously there is a golden mean here, but I just remember how radical some of these teachers were.
>10-15 years ago 'standardized testing doesnt matter at all'
>You are an idiot if you think it matters
>Oops THEY were wrong. You are still an idiot and ahole for disagreeing with me 10 years ago.
Obviously there is a golden mean here, but I just remember how radical some of these teachers were.
[deleted]
A golden mean between what? Truth and falsehoods. I hate citing 1984 in any matter such as this one, but there's a word in it for that - "doublethink".
A golden mean between "standardized tests don't matter at all" (false) and "standardized tests are the one right way to measure students' abilities" (also false).
They arent binary. Some tests wont matter at all, some will, and many will be in an unverifiable grey area,
The people who were saying tests didn't matter were the people that thought other things mattered and had other agendas to push...
But, is first-semester GPA predictive of anything? I'd rather see this result at graduation. Does the higher SAT predict higher graduation rate, better grad school or job placement, or anything else? Note, I'm not arguing it won't be - I don't know. I just know more than a few people who bombed their first semester (most with SATs well above average), myself included, who ended up with decent academic performance once they got over the "shock" of being an adult away from home.
I did terribly in first year, scraped through second year and did really well in third and fourth years - to the point of getting a first and a class prize. I don't think it was anything to do with "shock being away from home" - but actually realising that some things require hard work.
Oh, for me, it was definitely "no parents? Party!" That got old pretty quickly, just not quickly enough to save my first semester grades.
Happily the grade of degree awarded was based something like 75% on final year exams and project and 25% on 3rd year and you just had to survive the first two years.
Edit: I had started partying long before going to university so had already got that out of my system... :-)
Edit: I had started partying long before going to university so had already got that out of my system... :-)
> I just know more than a few people who bombed their first semester (most with SATs well above average), myself included, who ended up with decent academic performance once they got over the "shock" of being an adult away from home.
Individual students might underperform because they're not prepared for living outside the home, but that factor would cancel out when you're comparing two populations with thousands of students. Note that, according to the article, the 0.86 GPA point difference--which is huge, like the difference between top of the class and below median--was after adjusting for high school GPA. So the analysis should factor out differences resulting from some kids simply being more "put together" as freshman than other kids.
Individual students might underperform because they're not prepared for living outside the home, but that factor would cancel out when you're comparing two populations with thousands of students. Note that, according to the article, the 0.86 GPA point difference--which is huge, like the difference between top of the class and below median--was after adjusting for high school GPA. So the analysis should factor out differences resulting from some kids simply being more "put together" as freshman than other kids.
0.86 GPA points higher during their first semester. Is that really such a massive informational advantage?
For me, I got much, much better SAT scores than my grades would suggest, because my grades were depressed significantly due to ADHD, (forgetfulness, never remembering when tests were going to happen, going off on tangents when studying) even though I actually knew the material. So the standardized tests were a chance to prove that I wasn’t a total dummy.
And you know, I think that’s kind of the point of a standardized test- grades definitely vary based on how involved your parents are, the particular teachers you have, whether you have adhd, and yes socioeconomic status- tests in principal are blind to all that stuff.
And you know, I think that’s kind of the point of a standardized test- grades definitely vary based on how involved your parents are, the particular teachers you have, whether you have adhd, and yes socioeconomic status- tests in principal are blind to all that stuff.
I can get almost all the questions correct on the SAT test without any time pressure but when I do it when the clock is ticking, I find myself unable to reason my way to the correct answer nowhere near as fast. As a result, I have to move on and skip 1/2 of the verbal and math questions because I don't have enough time. And I don't draw blank when under pressure so there must be something else going on.
I have tried to pretend that I'm an LLM and trained myself on hundreds of SAT questions on YouTUbe where people solve them. I've found that my reasoning speed under time pressure is a direct function of the amount data I've trained myself on.
Is there a quicker way to this or do people like me, with average IQ or Chat-GPT 3.5 if you will, have to train on thousands of math and verbal questions because we don't have the Chat-GPT4 like the gifted people where they ara able to be creative and don't need to rely only on past knowledge to solve novel problems?
I have tried to pretend that I'm an LLM and trained myself on hundreds of SAT questions on YouTUbe where people solve them. I've found that my reasoning speed under time pressure is a direct function of the amount data I've trained myself on.
Is there a quicker way to this or do people like me, with average IQ or Chat-GPT 3.5 if you will, have to train on thousands of math and verbal questions because we don't have the Chat-GPT4 like the gifted people where they ara able to be creative and don't need to rely only on past knowledge to solve novel problems?
Yes, there are ways to speed up your time. When I took the SAT (2011), the main skill I developed was to quickly categorize the kind of question. Since there are only a few different kinds of questions, you could train yourself to do this formulaically/algorithmically. The type of question tells you how to eliminate a couple of the answer choices, and then you apply another question-type-specific algorithm to figure out the answer from the remaining options.
The Kaplan book (or similar) went over this. You learn to avoid “solving the problem the hard way” as much as possible.
However, the new digital SAT will require new strategies because its computer assisted, more like the LSAT/GRE (?) is.
However, the new digital SAT will require new strategies because its computer assisted, more like the LSAT/GRE (?) is.
> I've found that my reasoning speed under time pressure is a direct function of the amount data I've trained myself on.
That is the most pretentious way I have ever heard of saying "I did better when I studied and practiced more"
I'm actually in awe.
That is the most pretentious way I have ever heard of saying "I did better when I studied and practiced more"
I'm actually in awe.
The NYTimes had an good interactive story on this same topic this week:
https://www.nytimes.com/interactive/2024/03/09/upshot/affirm...
https://www.nytimes.com/interactive/2024/03/09/upshot/affirm...
I would hypothesize that SES probably correlates very heavily with grades at the very bottom where you have things like abuse, abject poverty, children supporting their own parents, lack of access to quality nutrition, violence inside and outside the home, etc and even a genius would be swimming against the current. But middle class and above, I would expect the curve to flatten out. In my experience super wealthy people are more lucky than clever- it doesn’t take much brains to make more money when you’re already rich.
So, the goal here should be to identify smart kids in terrible situations, while not allowing rich kids who aren’t actually that talented to game the system. That’s the goal. And of course there will also be genuinely talented middle class kids and you don’t want to ignore them either.
So, the goal here should be to identify smart kids in terrible situations, while not allowing rich kids who aren’t actually that talented to game the system. That’s the goal. And of course there will also be genuinely talented middle class kids and you don’t want to ignore them either.
i.e. socioeconomic status is correlated with SAT performance because socioeconomic status is also correlated with stronger basic academic skills.
Yeah, the article ends with:
> In conclusion, our work focuses on the predictive power of admissions tests and shows that this power is not an artifact of SES. There is a substantial SES–test relationship, though it is important to note that test scores reflect far more than SES. About a quarter of the variance in test scores is shared with SES, and thus there is large variability in test scores at any given level of SES. Thus, claims that tests are merely proxies for SES are unfounded
I've never understood the model of the world that people who don't agree with this. If you believe that
a) genetics exist, and traits such as intelligence are therefore hereditary; and b) socioeconomic success is correlated to intelligence
Then the inevitable conclusion is that wealthy people will be more likely to have higher SAT scores. Of course this is obviously just a correlation; lottery winners are rich but with no correlation to intelligence, children of two intelligent parents tend to regress to the mean, two average parents can produce intelligent children, families might be poor simply through bad luck or some particular emotional/mental block they have, and a million other factors. But nevertheless we'd expect test scores to be our best available wealth-blind predictor of college success and later success. Certainly it's hard to imagine children with money-poor - or maybe mildly abusive - parents having a shot without test scores. They'd be less likely to have a deep extracurricular involvement or anything else requiring parental involvement.
> In conclusion, our work focuses on the predictive power of admissions tests and shows that this power is not an artifact of SES. There is a substantial SES–test relationship, though it is important to note that test scores reflect far more than SES. About a quarter of the variance in test scores is shared with SES, and thus there is large variability in test scores at any given level of SES. Thus, claims that tests are merely proxies for SES are unfounded
I've never understood the model of the world that people who don't agree with this. If you believe that
a) genetics exist, and traits such as intelligence are therefore hereditary; and b) socioeconomic success is correlated to intelligence
Then the inevitable conclusion is that wealthy people will be more likely to have higher SAT scores. Of course this is obviously just a correlation; lottery winners are rich but with no correlation to intelligence, children of two intelligent parents tend to regress to the mean, two average parents can produce intelligent children, families might be poor simply through bad luck or some particular emotional/mental block they have, and a million other factors. But nevertheless we'd expect test scores to be our best available wealth-blind predictor of college success and later success. Certainly it's hard to imagine children with money-poor - or maybe mildly abusive - parents having a shot without test scores. They'd be less likely to have a deep extracurricular involvement or anything else requiring parental involvement.
> Yeah, the article ends with:
> > In conclusion, our work focuses on the predictive power of admissions tests and shows that this power is not an artifact of SES. There is a substantial SES–test relationship, though it is important to note that test scores reflect far more than SES. About a quarter of the variance in test scores is shared with SES, and thus there is large variability in test scores at any given level of SES. Thus, claims that tests are merely proxies for SES are unfounded
> I've never understood the model of the world that people who don't agree with this. If you believe that
> a) genetics exist, and traits such as intelligence are therefore hereditary;
> and
> b) socioeconomic success is correlated to intelligence
That is only true up to a certain level though. The correlation plateaus and even decreases above 1%
https://www.sciencedaily.com/releases/2023/02/230208125113.h...
> > In conclusion, our work focuses on the predictive power of admissions tests and shows that this power is not an artifact of SES. There is a substantial SES–test relationship, though it is important to note that test scores reflect far more than SES. About a quarter of the variance in test scores is shared with SES, and thus there is large variability in test scores at any given level of SES. Thus, claims that tests are merely proxies for SES are unfounded
> I've never understood the model of the world that people who don't agree with this. If you believe that
> a) genetics exist, and traits such as intelligence are therefore hereditary;
> and
> b) socioeconomic success is correlated to intelligence
That is only true up to a certain level though. The correlation plateaus and even decreases above 1%
https://www.sciencedaily.com/releases/2023/02/230208125113.h...
Top 1% is very high though, correlation up to top 1% is very good.
I've never understood the model of the world that people who don't agree with this.
My take is people question the tests because wealthy/powerful people have a history of "gaming the system" in order to retain that wealth/power. It's not unreasonable to ask "is the test predictive of anything other than the parent's ability to hire a tutor?" While a test is, in theory, easy for a poor student to pass through independent studies/library, etc, reality gets in the way - that student needs transport to the library, time to study (many work), etc.
But, based on this research, it seems the answer is "Yes, the test is actually predictive of something other than parent's SES."
My take is people question the tests because wealthy/powerful people have a history of "gaming the system" in order to retain that wealth/power. It's not unreasonable to ask "is the test predictive of anything other than the parent's ability to hire a tutor?" While a test is, in theory, easy for a poor student to pass through independent studies/library, etc, reality gets in the way - that student needs transport to the library, time to study (many work), etc.
But, based on this research, it seems the answer is "Yes, the test is actually predictive of something other than parent's SES."
A more common term is "Victim Mentality". It is the world view that all injustices or inequality is due to an oppressor group victimizing the oppressed class.
I think that the strength of point b) is the point that a lot of people question. I think for people like me who question the SES - intelligence link, the point of testing is to provide any opportunity for bright people who have lower SES to have a shot, which they would be otherwise denied due to their status.
If anything, testing is one of the things that can help create that link between SES and intelligence.
If anything, testing is one of the things that can help create that link between SES and intelligence.
Right, it's only a correlation, not a guarantee. And people who don't end up part of the correlation have maybe their best shot of becoming successful (assuming they pursue a career requiring a degree) via test scores. Anything else likely has a much stronger correlation with SES or might be very noisy: extracurriculars require money and parental involvement, high school grades can be worthless depending on the school, filtering by school rules out a lot of poor people, etc.
I think the OP has the arrow of causation pointing the other way from what you suggest: that is to say that higher intelligence leads to higher SES because knowledge work tends to be higher SES and those with higher intelligence are more likely to be able to find employment in knowledge work professions.
That is probably predicated on the existence of at least some social mobility though.
That is probably predicated on the existence of at least some social mobility though.
Regression to the mean is a real thing though. The reason why is this: every gene in your body is also in the bodies of millions of other people. Your genes are not unique. It’s the combination that’s unique. And the combination is diluted 50% with each generation. So Einstein was a prodigy and a genius, but his great grandchildren are pretty ordinary. That’s just the way it goes.
It's ironic to conclude this, without skepticism, knowing that it's a study sponsored by College Board which shows that the tests which College Board administers are definitely efficacious.
Sure; but do you disagree with the method or the conclusions?
Their study makes a link between test scores and first-semester performance. Even if we accept that link is real, that doesn't imply first-semester performance is indicative of academic success over the typical 4+ year undergraduate timeline.
I'd like to see independent corroboration of their study. Plus a longer term follow-up tracking these students through graduation or withdrawal.
I'd like to see independent corroboration of their study. Plus a longer term follow-up tracking these students through graduation or withdrawal.
[deleted]
Wouldn't socioeconomic status correlate just as strongly or even more with high school AP grades and extracurriculars?
It seems that these would require more discretionary time, as well as discretionary money, when comoared to the SAT.
It seems that these would require more discretionary time, as well as discretionary money, when comoared to the SAT.
So, I know the authors of this report personally. I sincerely have the highest respect for them, they do solid work; if I see something by them I take it seriously because I know I will disagree at my peril.
However, I also feel like findings like this get distorted and manipulated, and used to refute strawmen versions of arguments out there against use of standardized testing. Some of the recent arguments being used to reinstate standardized testing as required seem irresponsible to me, even though I don't necessarily see required standardized testing as undefensible per se (my own opinions about all of this are complex and don't fall neatly into either "side", so take this all as coming from someone who is frustrated with how things get oversimplified on both sides, not as advocating for a particular position).
Here's some things to keep in mind:
1. This is all about predicting freshman GPA, which itself is questionable as a criterion. If your sole question is "how will this individual do in our college their first year", it's reasonable. But if you are asking "how well would this individual master the skills necessary to succeed in X, Y, or Z role" it's entirely different. Freshman GPA predicts graduating GPA less well than you might think, and both of those are poor proxies for "real world" behavior. They're not unrelated to "real world behavior", but the relationship is low enough that this is the entire point of social justice advocates.
A lot of people would say neither SAT nor freshman GPA are relevant, that that's the point.
2. A correlation of 0.47 or 0.44 is something to pay attention to, but it's also far from perfect. If you want to remind yourself of how much noise is in that correlation look here:
http://www.analytictech.com/mb313/correl4.jpg
That's a ton of noise at the individual level, which is the level we care about when we're talking about admissions decisions.
The problem with a lot of this isn't with the SAT, it's that what happens is this 0.45 r inevitably gets turned into a blind metric that ignores all that real variation because it puts too much pressure on the school to take people who have all kinds of other evidence of competency, because it makes their numbers look bad.
Let me put it this way: if I was selling you a bathroom scale, and showed you that scatterplot of my scale's numbers and actual weight, would you buy it? You shouldn't, but we're making consequential life decisions based on that level of noise.
The Dartmouth report conveniently ignored that noise, when that noise is about 75% of the problem.
3. Relatedly, if you look at the individual level, things are vastly different when you start taking these kinds of population-level trends and using them to make individual decisions around a threshold. That is, it's very easy to say "0.46 correlation is pretty substantial, we should require SATs", but that 0.46 correlation is across the entire range. It includes people whose SAT scores are very very low. It's not only in the range of where decisions tend to be made, and to make use of that test as a metric for decision-making, you have to assume that what's in the college board's SES index is a perfect summary of what might be said for every applicant to your college. Maybe someone is upper class but their parents were both killed in a plane crash. Maybe they are middle class but come from an abusive family. Maybe they are applying as a much older candidate, or much younger candidate, and the meaning of the scores is really different.
In a sane world where we can have nice things, the admissions committee would look at this and make exceptions. This is the idea of having test scores optional. But when you make them required, and are required to post them, then there's pressure to take the highest scores regardless of all the other information.
4. About the other information: in Table 2, they show that HS GPA is actually less correlated with SES than SAT score. Other papers with similarly large amounts of data have shown that HS GPA is actually slightly more correlated with freshman GPA than SAT score. So is the SAT necessary? Probably not. We can argue about grade inflation etc., but the numbers are the numbers, and if it's working as an alternative that is less correlated with SES, reflects a longer sample of school behavior than just a few hours on one day, why wouldn't you prefer that?
5. Note their Figure 1b. They spend a lot of time talking about this model, and in some ways it's the focus of their paper, evaluating it against the alternative. But related to my first point, the model in Figure 1b says nothing about the extent to which a college, or anyone else for that matter, might want to select on some process represented by Figure 1b. What does it matter if test score predicts, say, some other test score composite, and both are influenced by SES?
In widely used intelligence tests, there were questions about things like 19th century European literature and classical music (I think they're still there, but I can't remember offhand; they might have been made optional now). On the one hand, yes, 19th century European literature and classical music is great, knowledge of it probably reflects some memory ability etc, but are questions about that really how you want to evaluate someone's skillset? Maybe it is, maybe it isn't, but I can tell you if you're not knowledgable about that you would not get credit on those questions.
In many ways, for many, the SAT and college freshman GPA are kind of similar. I don't feel that way, but this is the classic "book smarts" versus "real smarts" issue that always has come up since the beginning of humanity probably. Showing book smarts predicts other book smarts just isn't important in some paradigms.
Again, I'm not anti-standardized testing. I think it's useful. But I also think the way it's been used in the past, and continues to be used, is in fact broken. It's not even necessarily a problem with the tests, it's a problem with the way they get used. But to paraphrase a famous educational psychologist, if you have a thing that people tend to misuse, every single time, and there are good alternatives, maybe there is something about that thing that's a problem when you put it the hands of people, and the thing either shouldn't be used, or there should be some rules put in place with teeth to prevent it from being misused.
However, I also feel like findings like this get distorted and manipulated, and used to refute strawmen versions of arguments out there against use of standardized testing. Some of the recent arguments being used to reinstate standardized testing as required seem irresponsible to me, even though I don't necessarily see required standardized testing as undefensible per se (my own opinions about all of this are complex and don't fall neatly into either "side", so take this all as coming from someone who is frustrated with how things get oversimplified on both sides, not as advocating for a particular position).
Here's some things to keep in mind:
1. This is all about predicting freshman GPA, which itself is questionable as a criterion. If your sole question is "how will this individual do in our college their first year", it's reasonable. But if you are asking "how well would this individual master the skills necessary to succeed in X, Y, or Z role" it's entirely different. Freshman GPA predicts graduating GPA less well than you might think, and both of those are poor proxies for "real world" behavior. They're not unrelated to "real world behavior", but the relationship is low enough that this is the entire point of social justice advocates.
A lot of people would say neither SAT nor freshman GPA are relevant, that that's the point.
2. A correlation of 0.47 or 0.44 is something to pay attention to, but it's also far from perfect. If you want to remind yourself of how much noise is in that correlation look here:
http://www.analytictech.com/mb313/correl4.jpg
That's a ton of noise at the individual level, which is the level we care about when we're talking about admissions decisions.
The problem with a lot of this isn't with the SAT, it's that what happens is this 0.45 r inevitably gets turned into a blind metric that ignores all that real variation because it puts too much pressure on the school to take people who have all kinds of other evidence of competency, because it makes their numbers look bad.
Let me put it this way: if I was selling you a bathroom scale, and showed you that scatterplot of my scale's numbers and actual weight, would you buy it? You shouldn't, but we're making consequential life decisions based on that level of noise.
The Dartmouth report conveniently ignored that noise, when that noise is about 75% of the problem.
3. Relatedly, if you look at the individual level, things are vastly different when you start taking these kinds of population-level trends and using them to make individual decisions around a threshold. That is, it's very easy to say "0.46 correlation is pretty substantial, we should require SATs", but that 0.46 correlation is across the entire range. It includes people whose SAT scores are very very low. It's not only in the range of where decisions tend to be made, and to make use of that test as a metric for decision-making, you have to assume that what's in the college board's SES index is a perfect summary of what might be said for every applicant to your college. Maybe someone is upper class but their parents were both killed in a plane crash. Maybe they are middle class but come from an abusive family. Maybe they are applying as a much older candidate, or much younger candidate, and the meaning of the scores is really different.
In a sane world where we can have nice things, the admissions committee would look at this and make exceptions. This is the idea of having test scores optional. But when you make them required, and are required to post them, then there's pressure to take the highest scores regardless of all the other information.
4. About the other information: in Table 2, they show that HS GPA is actually less correlated with SES than SAT score. Other papers with similarly large amounts of data have shown that HS GPA is actually slightly more correlated with freshman GPA than SAT score. So is the SAT necessary? Probably not. We can argue about grade inflation etc., but the numbers are the numbers, and if it's working as an alternative that is less correlated with SES, reflects a longer sample of school behavior than just a few hours on one day, why wouldn't you prefer that?
5. Note their Figure 1b. They spend a lot of time talking about this model, and in some ways it's the focus of their paper, evaluating it against the alternative. But related to my first point, the model in Figure 1b says nothing about the extent to which a college, or anyone else for that matter, might want to select on some process represented by Figure 1b. What does it matter if test score predicts, say, some other test score composite, and both are influenced by SES?
In widely used intelligence tests, there were questions about things like 19th century European literature and classical music (I think they're still there, but I can't remember offhand; they might have been made optional now). On the one hand, yes, 19th century European literature and classical music is great, knowledge of it probably reflects some memory ability etc, but are questions about that really how you want to evaluate someone's skillset? Maybe it is, maybe it isn't, but I can tell you if you're not knowledgable about that you would not get credit on those questions.
In many ways, for many, the SAT and college freshman GPA are kind of similar. I don't feel that way, but this is the classic "book smarts" versus "real smarts" issue that always has come up since the beginning of humanity probably. Showing book smarts predicts other book smarts just isn't important in some paradigms.
Again, I'm not anti-standardized testing. I think it's useful. But I also think the way it's been used in the past, and continues to be used, is in fact broken. It's not even necessarily a problem with the tests, it's a problem with the way they get used. But to paraphrase a famous educational psychologist, if you have a thing that people tend to misuse, every single time, and there are good alternatives, maybe there is something about that thing that's a problem when you put it the hands of people, and the thing either shouldn't be used, or there should be some rules put in place with teeth to prevent it from being misused.
2.) Intelligent people get higher paying jobs.
3.) Intelligent people live in wealthier areas.
4.) Intelligent people marry other intelligent people.
5.) Intelligent people birth intelligent children.
6.) Intelligent people use their intelligence to augment their kid's intelligence.
7.) Intelligent kids go to step 1.)
The emperor is walking around naked, and virtually all of academia is terrified to acknowledge it (namely step 5). Right there is the solution to 50 years of sociological fumbling to avoid saying "Intelligence has a strong genetic component, and we are not all born with equal mental capabilities"
Once academia can acknowledge this, then they can finally work on real solutions to close the disproportionate economic gaps it creates.
(Yes, I am aware of your anecdote that counters this. It's a broad generalization to counter the mainstream broad generalization.)