Scores for adults are dropping on tests of basic skills(nataliewexler.substack.com)
nataliewexler.substack.com
Scores for adults are dropping on tests of basic skills
https://nataliewexler.substack.com/p/scores-for-adults-are-dropping-on
119 comments
This is very interesting. The study it's reporting on has all the numerical data here, which speaks volumes: https://www.oecd.org/en/publications/2024/12/do-adults-have-...
The US was never built to produce a cohesive nation on any level outside of commerce and federal military. This is particularly visible today in terms of education.
There's an attitude here of "I'm gonna do the best for my family; fuck all other people". Such a culture isn't going to be globally competitive. The money machines have dried up and there's no obvious frontier remaining....
There's an attitude here of "I'm gonna do the best for my family; fuck all other people". Such a culture isn't going to be globally competitive. The money machines have dried up and there's no obvious frontier remaining....
The US used to be globally competitive. I don't know what changed.
I do get the strong impression that there was a period of perhaps a century or so, from the mid 19th through late 20th, where we were at least a little less likely to be fighting against each other. Those were rather prosperous times.
Since then we've still produced some very big things, but it seems less widely beneficial. And we do seem increasingly to be at each other's throats. I don't know if they're related.
I do get the strong impression that there was a period of perhaps a century or so, from the mid 19th through late 20th, where we were at least a little less likely to be fighting against each other. Those were rather prosperous times.
Since then we've still produced some very big things, but it seems less widely beneficial. And we do seem increasingly to be at each other's throats. I don't know if they're related.
Less social cohesion and less meritocracy
The American Civil War was in the 1860s and there's been some pretty incredible periods of unrest between now and then. We are reaching some historic levels of wealth inequality though.
That's a bizarre claim considering that the US economy is currently outperforming most other developed countries.
[deleted]
When did I mention the economy?
> Such a culture isn't going to be globally competitive
Globally competitive in what sense, then?
Globally competitive in what sense, then?
Well, the account was made 46 days ago. People can talk about rules of the site and so on. But, we live in a different world now than when the rules were made. Hybrid warfare and online interactions are the easiest way for US adversaries to spread FUD and sow dissent.
> "I'm gonna do the best for my family; fuck all other people"
If I was going to rank countries best described by this statement the US would come in very close to the bottom.
If I was going to rank countries best described by this statement the US would come in very close to the bottom.
As in you think the US is actually very collectivist or as in you think the US doesn't care about families at all, only individuals?
Americans are extremely generous and charitable by many metrics like this. [1]
Not to mention soft generosity like tipping culture.
[1] https://www.axios.com/2022/03/09/america-charitable-giving-s...
Not to mention soft generosity like tipping culture.
[1] https://www.axios.com/2022/03/09/america-charitable-giving-s...
Found this but the ranking is from 2016.
https://en.wikipedia.org/wiki/List_of_countries_by_charitabl...
https://en.wikipedia.org/wiki/List_of_countries_by_charitabl...
That's a horrible metric since charity is used as a tax dodging mechanism in the US, while other countries (non OECD ones) do not have this mechanism.
You will spend less money just paying taxes than donating money are writing off your top end. For instance if I have $100 and the government wants 40% of it then I could give them $40. Or I could donate, lets say $15 and now I owe 40% on $85 which is $34. Add the $15 I donated and it's $49.
Plus there are limits on how much you can deduct and it's less for assets than for cash. Lastly we have a thing called Alternitive Minimum Tax what is a catchall for anyone that thinks they can pay less by adjusting their income down. You need to calculate both taxes and you pay the higher one.
Plus there are limits on how much you can deduct and it's less for assets than for cash. Lastly we have a thing called Alternitive Minimum Tax what is a catchall for anyone that thinks they can pay less by adjusting their income down. You need to calculate both taxes and you pay the higher one.
> Not to mention soft generosity like tipping culture.
barfs
barfs
This is an interesting article (list-icle? It is pretty short and aggregate list like).
It is from 2022 and mainly focuses on pandemic times, so I'd be curious to see what these trends look like compared to pre/post pandemic. I'm sure I could do more digging and find the answers to this, but still I appreciate this, it is interesting.
One thing I think I would push back on though is the remark of 'soft generosity like tipping culture' which isn't mentioned at all in the list. I also would not consider US tipping culture generosity, but more of a social contract/obligation. Someone that does not tip anywhere, except for where they received great service, isn't normally seen as a non-generous person, they're seen as not abiding by the social contract that you have to tip.
It is from 2022 and mainly focuses on pandemic times, so I'd be curious to see what these trends look like compared to pre/post pandemic. I'm sure I could do more digging and find the answers to this, but still I appreciate this, it is interesting.
One thing I think I would push back on though is the remark of 'soft generosity like tipping culture' which isn't mentioned at all in the list. I also would not consider US tipping culture generosity, but more of a social contract/obligation. Someone that does not tip anywhere, except for where they received great service, isn't normally seen as a non-generous person, they're seen as not abiding by the social contract that you have to tip.
You don’t have to tip. Nothing will happen to you if you don’t. Some people don’t or tip very little. Many tip well above expectations. Most people don’t mind the practice.
But it’s not just tipping at a restaurant. We tip our garbage men, helpers, mailmen, doormen, maids, nanny’s, etc at Christmas. You don’t have to do this. But many of us do because we are raised to be generous if we are fortunate enough to be.
But it’s not just tipping at a restaurant. We tip our garbage men, helpers, mailmen, doormen, maids, nanny’s, etc at Christmas. You don’t have to do this. But many of us do because we are raised to be generous if we are fortunate enough to be.
Yeah, generous like “I’ll pay you if I want to” instead of generous like “I pay you because you work for me”
Have you ever been outside the US?
Yes, there’s definitely a lot of nice people there (most, even), but there’s also a whole bunch more that make it feel unsafe to just walk the streets.
Yes, there’s definitely a lot of nice people there (most, even), but there’s also a whole bunch more that make it feel unsafe to just walk the streets.
> Americans are extremely generous
A country with one of the highest income inequalities in the western world!
https://en.wikipedia.org/wiki/List_of_countries_by_income_in...
Being generous is voting for healthcare and social welfare programmes for all, regardless of income.
Also, the charitable giving figures you quote are skewed as the US offers tax breaks to donate to "charities" which can be financially beneficial to the giver (charity oversight in the USA is notoriously lax).
> Not to mention soft generosity like tipping culture.
Don't take the piss!
A country with one of the highest income inequalities in the western world!
https://en.wikipedia.org/wiki/List_of_countries_by_income_in...
Being generous is voting for healthcare and social welfare programmes for all, regardless of income.
Also, the charitable giving figures you quote are skewed as the US offers tax breaks to donate to "charities" which can be financially beneficial to the giver (charity oversight in the USA is notoriously lax).
> Not to mention soft generosity like tipping culture.
Don't take the piss!
> Being generous is voting for healthcare and social welfare programmes for all, regardless of income.
There are various methods of rationing healthcare. I'm not sure choosing the government to arbitrate rationing is any more "generous". Regardless, American's that can't afford to subsidize their own healthcare (poor, disabled, child, elderly, etc) have many options from the government to assist them such as Medicare, Medicaid, a variety of state programs, etc. I also wouldn't use the NHS as any model of egalitarian or generosity in healthcare as any Brit that can afford it has private insurance on top of it since the NHS leaves a lot to be desired.
> Also, the charitable giving figures you quote are skewed as the US offers tax breaks to donate to "charities" which can be financially beneficial to the giver (charity oversight in the USA is notoriously lax).
You can deduct from your taxable burden certain charitable donations, but it isn't anywhere near a complete write-off. You'd save considerably more by just paying taxes instead of giving to charity. And this is a great policy as it allows citizens to fund things that would otherwise be well out of scope of government.
> Don't take the piss!
Waitstaff is what many people think of in this context. But waitstaff in the USA earns considerably more than in the UK or EU due to the generosity of tipping culture. I've never met a waiter that wanted a higher hourly wage to replace their tips as they can and do make a lot of money with tips.
But Americans tip far more than waiters. During Christmas we give tips to our garbage collectors, mail carriers, extra to maids and other helpers, tip to teachers and school staff, doormen if you live in an apartment with them. People give greatly to organizations they support and to their churches.
I've spent a lot of time in the UK and although a lovely place in many regards, I'm not sure there's a more stingy, tight culture anywhere else on Earth. Everything is precise and transactional and anything regarding helping someone is measured and on the lean side. In large part it's because Brits are very poor compared to Americans but it's also a culture thing too.
There are various methods of rationing healthcare. I'm not sure choosing the government to arbitrate rationing is any more "generous". Regardless, American's that can't afford to subsidize their own healthcare (poor, disabled, child, elderly, etc) have many options from the government to assist them such as Medicare, Medicaid, a variety of state programs, etc. I also wouldn't use the NHS as any model of egalitarian or generosity in healthcare as any Brit that can afford it has private insurance on top of it since the NHS leaves a lot to be desired.
> Also, the charitable giving figures you quote are skewed as the US offers tax breaks to donate to "charities" which can be financially beneficial to the giver (charity oversight in the USA is notoriously lax).
You can deduct from your taxable burden certain charitable donations, but it isn't anywhere near a complete write-off. You'd save considerably more by just paying taxes instead of giving to charity. And this is a great policy as it allows citizens to fund things that would otherwise be well out of scope of government.
> Don't take the piss!
Waitstaff is what many people think of in this context. But waitstaff in the USA earns considerably more than in the UK or EU due to the generosity of tipping culture. I've never met a waiter that wanted a higher hourly wage to replace their tips as they can and do make a lot of money with tips.
But Americans tip far more than waiters. During Christmas we give tips to our garbage collectors, mail carriers, extra to maids and other helpers, tip to teachers and school staff, doormen if you live in an apartment with them. People give greatly to organizations they support and to their churches.
I've spent a lot of time in the UK and although a lovely place in many regards, I'm not sure there's a more stingy, tight culture anywhere else on Earth. Everything is precise and transactional and anything regarding helping someone is measured and on the lean side. In large part it's because Brits are very poor compared to Americans but it's also a culture thing too.
It’s literally the world’s oldest existing democracy and has the world’s most dynamic economy and diverse population and isn’t at civil war. What isn’t cohesive or globally competitive about that?
Americans are also among the most generous and charitable people according to a number of easily Googleable metrics. What is “fuck other people” about that?
Nowhere is perfect and America is far from it but at least 25% of the world (and likely higher) would immediately move there if they could.
Americans are also among the most generous and charitable people according to a number of easily Googleable metrics. What is “fuck other people” about that?
Nowhere is perfect and America is far from it but at least 25% of the world (and likely higher) would immediately move there if they could.
[deleted]
It's only the worlds oldest democracy if you don't consider black people and women to be people.
What does that have to do with the us being a democracy? You don't have to have universal suffrage to be a democracy.
The people who don’t get to vote might disagree.
In any case, the UK parliament has existed since 1705, which predates the American revolution.
In any case, the UK parliament has existed since 1705, which predates the American revolution.
I disagree? As does the word democracy I'd say.
By that logic, I don't think a democracy has ever existed. No country anywhere has or as far as I know, has ever had, true universal suffrage. Do you have an example of a democracy under your definition?
Yeah, Sweden. Every citizen above the age of majority can vote. The only way to lose your right to vote it to not be a citizen. I'm sure there are other countries as well. The US is not one of them however as certain felons can't vote, making up a total of 2,47% of the voting population in 2016 according to wikipedia.
Unless you wanna argue that age of majority is the same as disallowing black people and women, which I'd disagree strongly with. You don't grow out of being black or a woman.
Unless you wanna argue that age of majority is the same as disallowing black people and women, which I'd disagree strongly with. You don't grow out of being black or a woman.
Yes, the age limit is the most common (universal?) restriction on universal suffrage.
I don't understand what the difference between restricting the vote based on age and race or gender. Universal suffrage must be universal, right? It's not clear what growing out of anything has to do with the concept.
Otherwise it seems to just be "universal amongst the groups I think should have it".
I don't understand what the difference between restricting the vote based on age and race or gender. Universal suffrage must be universal, right? It's not clear what growing out of anything has to do with the concept.
Otherwise it seems to just be "universal amongst the groups I think should have it".
So, in general, it seems like we're okay with discriminating people based on essential mutable characteristics and non-essential characteristics, but not based on essential immutable characteristics.
Illnesses are non-essential characteristics. A human doesn't necessarily have an illness. We're okay with discriminating towards people based on this. You can't become a pilot if you have severe problems with your vision and you can't operate heavy machinery if you get regular seizures.
Age is essential, a human must have an age, but it's mutable. A human must also have a height and weight, but they're also mutable. We're generally okay with discriminating based on all of these. You can't go on certain rides if you're too tall or too short, you can't vote if you're too young, you can't ride horses if you're too heavy and you can't give blood if you're too light.
Race, sex and sexuality are essential and immutable. In the west people seem to be converging on it not being okay to discriminate based on essential immutable characteristics. I agree with this and I see race, sex, sexuality as far more intimately tied to the essence of a human being than their age, height, weight or list of illnesses.
If you disagree with these distinctions and groupings, that's fine. But then I don't think there's much of a point in us continuing as it'll simply take too long, heh.
So, having read all of the above, my first objection to myself is "So I think we should be okay with taking a person's right to vote based on their height? It falls in the same category as age.". To that objection I'd say no, we should not. Note that in all the above examples of discrimination the basis for the discrimination is that the discriminated group can't do the activity safely/competently. We're usually not okay with baseless discrimination. You can't disallow disabled people from buying your cake, you can't disallow short people from eating at your restaurant and you can't disallow young people from operating a camera. These would all be baseless and the characteristics being discriminated don't relate to the actions in any meaningful way.
In this same sense, young people can't vote safely/competently and voting is important enough for us to desire safe/competent voting. A young persons brain simply is not developed enough to be able to reason in some ways an adult can. A young person is also far too attached to their parents. The parents will have such a huge amount of influence that giving children the right to vote is the same as giving their parent another vote.
Now, this leads to another conclusion which I'm not entirely comfortable with. If we're okay with not giving a person the right to vote based on age-caused lack of reasoning ability, shouldn't we also be taking away the ability to vote from adults who are deemed to have the reasoning ability of someone below the age of majority? I'm not sure to be honest. I suppose we're just legislating for the average though and the average person above the age of majority has a greater reasoning ability than the average person below the age of majority. So even people well below the average, below even the young people average, are treated as though they're average.
This says nothing about what the age of majority should be, mind you. 18? 12? 45? Who knows. But I find it reasonable that a age limit should exist and that it has enough basis to still call it a democracy.
Thank you for attending my TED talk :)
Illnesses are non-essential characteristics. A human doesn't necessarily have an illness. We're okay with discriminating towards people based on this. You can't become a pilot if you have severe problems with your vision and you can't operate heavy machinery if you get regular seizures.
Age is essential, a human must have an age, but it's mutable. A human must also have a height and weight, but they're also mutable. We're generally okay with discriminating based on all of these. You can't go on certain rides if you're too tall or too short, you can't vote if you're too young, you can't ride horses if you're too heavy and you can't give blood if you're too light.
Race, sex and sexuality are essential and immutable. In the west people seem to be converging on it not being okay to discriminate based on essential immutable characteristics. I agree with this and I see race, sex, sexuality as far more intimately tied to the essence of a human being than their age, height, weight or list of illnesses.
If you disagree with these distinctions and groupings, that's fine. But then I don't think there's much of a point in us continuing as it'll simply take too long, heh.
So, having read all of the above, my first objection to myself is "So I think we should be okay with taking a person's right to vote based on their height? It falls in the same category as age.". To that objection I'd say no, we should not. Note that in all the above examples of discrimination the basis for the discrimination is that the discriminated group can't do the activity safely/competently. We're usually not okay with baseless discrimination. You can't disallow disabled people from buying your cake, you can't disallow short people from eating at your restaurant and you can't disallow young people from operating a camera. These would all be baseless and the characteristics being discriminated don't relate to the actions in any meaningful way.
In this same sense, young people can't vote safely/competently and voting is important enough for us to desire safe/competent voting. A young persons brain simply is not developed enough to be able to reason in some ways an adult can. A young person is also far too attached to their parents. The parents will have such a huge amount of influence that giving children the right to vote is the same as giving their parent another vote.
Now, this leads to another conclusion which I'm not entirely comfortable with. If we're okay with not giving a person the right to vote based on age-caused lack of reasoning ability, shouldn't we also be taking away the ability to vote from adults who are deemed to have the reasoning ability of someone below the age of majority? I'm not sure to be honest. I suppose we're just legislating for the average though and the average person above the age of majority has a greater reasoning ability than the average person below the age of majority. So even people well below the average, below even the young people average, are treated as though they're average.
This says nothing about what the age of majority should be, mind you. 18? 12? 45? Who knows. But I find it reasonable that a age limit should exist and that it has enough basis to still call it a democracy.
Thank you for attending my TED talk :)
Another question: are you suggesting that the United States isn't a democracy today?
If so, your definition just doesn't seem realistic or useful.
If so, your definition just doesn't seem realistic or useful.
It is a very flawed democracy but it's definitely attempting it. I find them disallowing felons from voting and the way the electoral college can, and has, allowed presidents to be elected while having less of the popular vote than the loser to knock it down many pegs on the democracy ladder. Hillary got 2.1% more of the popular vote in 2016 but lost.
Defining "democracy" would take us two the rest of our lives but we'd never arrive at losing while ahead by 2.1% in a popular vote as the conclusion.
Defining "democracy" would take us two the rest of our lives but we'd never arrive at losing while ahead by 2.1% in a popular vote as the conclusion.
[deleted]
Sources?
That doesn't really explain why there's a particularly notable drop in older adults after 2017, does it?
[deleted]
> On the test’s 500-point scale, American adults scored 258 on average in literacy, 13 points lower than in 2017, and 249 in numeracy, 6 points lower.
> One literacy question involves a passage describing how bread and crackers become stale, with the last sentence reading “Crackers seem soft when their moisture level reaches about 9 percent.” The sample “low difficulty” question asks, “At what moisture level do crackers become soft?”
> The question may seem straightforward, but it doesn’t use the kind of sentence structure that appears in oral language. People are more likely to say something like, “When do crackers get soft?” Lack of familiarity with complex syntax can be a huge barrier to comprehension.
I wonder if there is research about whether understanding of complex non-conversational syntax requires practice - or is it like "riding a bike" where an adult who had exposure to this kind of syntax during their education is able to retain this skill without practice?
One pair of major shifts in the recent decade in the US have been a decline in trust in print and televised journalistic sources (who would be more likely to use complex syntax in their presentation), and an increased prevalence of targeted video content that has been effectively optimized to be as concise as possible. Is exposure to complex syntax declining as a result?
> One literacy question involves a passage describing how bread and crackers become stale, with the last sentence reading “Crackers seem soft when their moisture level reaches about 9 percent.” The sample “low difficulty” question asks, “At what moisture level do crackers become soft?”
> The question may seem straightforward, but it doesn’t use the kind of sentence structure that appears in oral language. People are more likely to say something like, “When do crackers get soft?” Lack of familiarity with complex syntax can be a huge barrier to comprehension.
I wonder if there is research about whether understanding of complex non-conversational syntax requires practice - or is it like "riding a bike" where an adult who had exposure to this kind of syntax during their education is able to retain this skill without practice?
One pair of major shifts in the recent decade in the US have been a decline in trust in print and televised journalistic sources (who would be more likely to use complex syntax in their presentation), and an increased prevalence of targeted video content that has been effectively optimized to be as concise as possible. Is exposure to complex syntax declining as a result?
One thing to consider here is that the article pulled this question out, but did not actually state how many people actually got this question right or wrong.
It is pretty common for these kinds of tests to include a bunch of questions along various scales, including ones that researchers expect "everyone" to get.
This author might have a story they are trying to tell, and if they had information about this question like "oh well everyone got this question right", then they would not be able to illustrate what they mention afterwards:
> So can unfamiliar words or concepts. It’s been estimated that comprehension starts to break down when as little as 2 percent of the words in a passage are unfamiliar. The bread-and-crackers text uses a number of words and phrases that aren’t likely to crop up in ordinary conversation, like “exposed to the elements,” “crystallized,” and “retrogradation”
This seems fairly straightforward, right? Like some people will not be able to fill in the blank of the meaning of certain words, especially people who are not as good at "test taking" skills (solving the puzzle of understanding all the answers are in the text, process of elimination and other tricks that people who ace this stuff are using without even thinking).
It is pretty common for these kinds of tests to include a bunch of questions along various scales, including ones that researchers expect "everyone" to get.
This author might have a story they are trying to tell, and if they had information about this question like "oh well everyone got this question right", then they would not be able to illustrate what they mention afterwards:
> So can unfamiliar words or concepts. It’s been estimated that comprehension starts to break down when as little as 2 percent of the words in a passage are unfamiliar. The bread-and-crackers text uses a number of words and phrases that aren’t likely to crop up in ordinary conversation, like “exposed to the elements,” “crystallized,” and “retrogradation”
This seems fairly straightforward, right? Like some people will not be able to fill in the blank of the meaning of certain words, especially people who are not as good at "test taking" skills (solving the puzzle of understanding all the answers are in the text, process of elimination and other tricks that people who ace this stuff are using without even thinking).
I don’t think these things are different between the 2017 and 2023 test, so it all feels silly to argue about. Even if complex sentences are harder for people to understand, that’s no excuse for them to get worse at it.
Yeah, I totally agree with the presumption that the measure is measuring something, and that there is cause for discussion there.
I just feel very weird about how the article takes that sample question and sort of priming people with "oh everyone is dumb" takes, when there's an (IMO) likely reality of that question in particular having a 98% hit rate.
I just feel very weird about how the article takes that sample question and sort of priming people with "oh everyone is dumb" takes, when there's an (IMO) likely reality of that question in particular having a 98% hit rate.
I'm sorry, but "At what moisture level do crackers become soft" isn't complex syntax. I would simply blame lack of basic literacy taught in schools, and the 'whole word' method. I would recommend the 'Sold a Story' podcast series[1] for more information.
[1] https://features.apmreports.org/sold-a-story/
[1] https://features.apmreports.org/sold-a-story/
That wouldn't explain why the biggest decreases are in older adults, who have been out of school for many decades.
I would say “at what moisture level do crackers become soft” is very general. What is a “moisture level?” They should be specific.
Though I guess half of the point is they want people to tease it out.
Though I guess half of the point is they want people to tease it out.
When it comes to evaluating literacy you take everything in the text to be fact and assume it makes sense. It shouldn't require any external knowledge or synthesis beyond understanding the language itself.
I don't like these tests very much because they live in a somewhat uncanny valley for me for the reason you're getting at. They're almost questions you would encounter in real life but play by different rules than people assume. Like no the question really is that easy, but you have to be told that because otherwise you'll assume you're missing something.
I don't like these tests very much because they live in a somewhat uncanny valley for me for the reason you're getting at. They're almost questions you would encounter in real life but play by different rules than people assume. Like no the question really is that easy, but you have to be told that because otherwise you'll assume you're missing something.
There are many people who can understand "What level does X happen at?", but do not understand "At what level does X happen?"
I would posit that perhaps this is exactly what this article is about?
I mean, we're not talking about complex grammar.
I mean, we're not talking about complex grammar.
> “Crackers seem soft when their moisture level reaches about 9 percent.”
> “At what moisture level do crackers become soft?”
Are they sure the problem is with "complex syntax"? The statement is that crackers >seem< soft at >about< 9%; yet the question is asking what level do they >become< soft. I had to constantly remind myself on standardize test to remember to use the most general/basic understanding of this type of problem since they did not give the level it actually becomes soft and its standardize testing so needed to remember, "It's not that deep", as the kids say.
> “At what moisture level do crackers become soft?”
Are they sure the problem is with "complex syntax"? The statement is that crackers >seem< soft at >about< 9%; yet the question is asking what level do they >become< soft. I had to constantly remind myself on standardize test to remember to use the most general/basic understanding of this type of problem since they did not give the level it actually becomes soft and its standardize testing so needed to remember, "It's not that deep", as the kids say.
Had the exact same thought. It's simply not clear that the statement and the question relate to the same thing. And this is the kind of thing that matters to other portions of the test, such as what a set of rules allows or prohibits.
I never eat crackers and I didn't even know that softness is what defines their staleness until now.
You may enjoy reading about why Jaffa cakes are not biscuits. A landmark UK case.
> “Crackers seem soft when their moisture level reaches about 9 percent.” The sample “low difficulty” question asks, “At what moisture level do crackers become soft?”
Are the crackers soft when they seem soft ? To put it another way: is a woman beautiful when she seems beautiful ? (English is not my mother tongue)
Are the crackers soft when they seem soft ? To put it another way: is a woman beautiful when she seems beautiful ? (English is not my mother tongue)
Here's a list of demo tests in case you're interested: https://www.oecd.org/en/about/programmes/piaac/education-and...
They're amazingly janky. I'll work on my literacy if you do better on your front end, OECD.
They're amazingly janky. I'll work on my literacy if you do better on your front end, OECD.
Yeah, I just tried taking the test too. I'm shocked that anybody could successfully complete it. It's dreadfully unclear how you're even supposed to answer the questions.
I did a bunch of the demo questions. Got them all correct. Most were only Level 1 or level 2 so didn't seem too hard.
Are you sitting down?
Are you sitting down?
The ideas are good, but wow, the implementation is borderline criminal.
Those feel like the majority of online survey systems, I'm afraid. They match the state of the industry.
> You must use Firefox – most current version or one version older than most current. The test does not function properly in other browsers
Truly. I wonder if they are trying to maintain UX continuity with 1997, forcing them to maintain this jankfest forever.
It seemed fine enough to me, if a bit dated (I am using the latest version of Firefox though).
You think it's bad now? Wait until LLMs become ubiquitous.
If the ability to critically reason is something that must be deliberately practiced, then the siren song of instantly reaching for an LLM the moment (THE SECOND) you have a question—without at least taking some almost meditative time to ponder and spin the problem around in your own mind—represents a potentially catastrophic decline in basic reasoning.
Furthermore let's not fall into the trap of believing that LLMs are just another tool like a calculator. If anything an LLM is more like a universal calculator where you can simply say, "Here's a math problem, please solve it for me."
If the ability to critically reason is something that must be deliberately practiced, then the siren song of instantly reaching for an LLM the moment (THE SECOND) you have a question—without at least taking some almost meditative time to ponder and spin the problem around in your own mind—represents a potentially catastrophic decline in basic reasoning.
Furthermore let's not fall into the trap of believing that LLMs are just another tool like a calculator. If anything an LLM is more like a universal calculator where you can simply say, "Here's a math problem, please solve it for me."
Right. People may be unable to do everyday grocery store type math because they're dependent on calculators which they don't have the wherewithal to pull out in casual situations.
Maybe that's not catastrophic, but offloading general thinking to machines, imagine where we're going to be a generation from now when that skill has declined across the population.
Maybe that's not catastrophic, but offloading general thinking to machines, imagine where we're going to be a generation from now when that skill has declined across the population.
Maybe, maybe not.
I have to imagine that if thinking skill becomes less common, it will become more valuable, thus incentivizing people to develop it, no?
I have to imagine that if thinking skill becomes less common, it will become more valuable, thus incentivizing people to develop it, no?
It becomes less common because it's less valuable. If LLM can reason, there is little value in human reasoning (beyond particular situations).
Given that there's known neurological complications from Covid infection, it's possible it's a contributing factor to the drop in test scores.
I wonder about this regularly and think back to discussions on this topic back in 2020-2021. Are there more recent studies or research on this topic you'd recommend?
You'd have to be on disability to not understand the sort of questions like this. Those disability numbers, especially from covid, should be available.
“Crackers seem soft when their moisture level reaches about 9 percent.” The sample “low difficulty” question asks, “At what moisture level do crackers become soft?”
“Crackers seem soft when their moisture level reaches about 9 percent.” The sample “low difficulty” question asks, “At what moisture level do crackers become soft?”
Crackers also seem soft when their moisture level reaches about 5 percent.
What moisture level do crackers become soft?
I'm sorry, 5 percent isn't the correct answer; they actually get soft earlier than then.
This seems like one of those questions like "what's the difference between an abstract class and an interface in Java" where you're looking what the expected answer is as opposed to the actual answer. (w.r.t. Java; the answer they're looking for is that you can define a default method in an abstract class. It doesn't matter how long ago you could do that in an interface ...)
What moisture level do crackers become soft?
I'm sorry, 5 percent isn't the correct answer; they actually get soft earlier than then.
This seems like one of those questions like "what's the difference between an abstract class and an interface in Java" where you're looking what the expected answer is as opposed to the actual answer. (w.r.t. Java; the answer they're looking for is that you can define a default method in an abstract class. It doesn't matter how long ago you could do that in an interface ...)
I guess you’d fail the reading comprehension part then. All the required info is in the question. You can’t just pull 5% from nowhere. There is an assertion that it’s 9%, the task is to identify and repeat the asserted value, not make up your own.
But the text says "seems", not that it actually "becomes" soft at 9%. Depending on your view of language, whether the answer is there changes.
“Seems” here being “you experience it as soft” since soft isn’t an absolute thing. Since there are no other mentions of percentages that are soft there isn’t really anything else to mix it up with.
That line of thinking breaks down in the real world though.
Its like when some scientist reports they killed 70% of cancer cells in the lab and a news report comes out with scientists develop treatment for 70% of cancer.
You cannot just take "scientist reports they killed 70% of cancer cells in the lab" and then follow-up with a question of "what percent of cancer did the scientist cure?" and then use the only percentage you saw ("70%") as the answer. The information given wasn't not sufficient to answer the question asked; in a test scenario, sure make an assumption but in the real world just get clarification ...
Its like when some scientist reports they killed 70% of cancer cells in the lab and a news report comes out with scientists develop treatment for 70% of cancer.
You cannot just take "scientist reports they killed 70% of cancer cells in the lab" and then follow-up with a question of "what percent of cancer did the scientist cure?" and then use the only percentage you saw ("70%") as the answer. The information given wasn't not sufficient to answer the question asked; in a test scenario, sure make an assumption but in the real world just get clarification ...
> All the required info is in the question.
I mean it's not though. The question is about "is soft" not "seems soft".
These slight of hands are pretty crucial to actually reading in the real world (or conversely, deceiving people).
It's like if you read a press release about how X is 120% better than Y. Judging from the press release, X is clearly better than Y but then if you actually do your own research you find for your workload X is actually 10% worse than Y.
I mean it's not though. The question is about "is soft" not "seems soft".
These slight of hands are pretty crucial to actually reading in the real world (or conversely, deceiving people).
It's like if you read a press release about how X is 120% better than Y. Judging from the press release, X is clearly better than Y but then if you actually do your own research you find for your workload X is actually 10% worse than Y.
"Reaches" ought to imply the lower threshold for softness. Of course, softness itself is not a crisp predicate.
What you are missing is the part about following directions. The literacy tests will instruct you to use the provided information to answer the questions. It doesn't matter if the information is incorrect. In fact, it may be beneficial to have incorrect information to eliminate people using preexisting knowledge vs comprehending the provided text.
Ok so why did the scores change?
COVID is global, obviously. The article specifically calls out Americans' scores dropping.
The article also pointed out that similar things are apparent in many other countries. Finland was highlighted as a place that avoided the trend -- a country that also had relatively low covid impact compared to the US.
I mean, not really; John's Hopkins puts Finland's all time test positivity per capita at ~26%, to the United States' ~30%. A bit better, but not enough to explain differences like this. Its not vaccination either: Finland only leads the US by ~0.23%.
One area where Finland had low impact is in deaths: 0.16% per capita to the US's 0.33%. But, if the theory is that COVID infections themselves caused a globally observable effect like this, that stat should cause the problem to hit Finland more, not less, given they had more COVID survivors than the US.
I think the better explanation might be COVID adjacent: COVID seems to have some direct neurological impact, but the lingering secondary cultural, economic, and psychological effects of the pandemic & lockdown feel a lot more likely to explain this. Additionally, I'd put money on social media also being higher on the sources list than direct COVID neurological impact.
One area where Finland had low impact is in deaths: 0.16% per capita to the US's 0.33%. But, if the theory is that COVID infections themselves caused a globally observable effect like this, that stat should cause the problem to hit Finland more, not less, given they had more COVID survivors than the US.
I think the better explanation might be COVID adjacent: COVID seems to have some direct neurological impact, but the lingering secondary cultural, economic, and psychological effects of the pandemic & lockdown feel a lot more likely to explain this. Additionally, I'd put money on social media also being higher on the sources list than direct COVID neurological impact.
Just eyeballing the tables[0][1], that certainly seems plausible.
[0] https://www.worldometers.info/coronavirus/ [1] https://www.oecd.org/en/publications/2024/12/do-adults-have-...
[0] https://www.worldometers.info/coronavirus/ [1] https://www.oecd.org/en/publications/2024/12/do-adults-have-...
[dupe]
Earlier on the OECD report: https://news.ycombinator.com/item?id=42376014
Earlier on the OECD report: https://news.ycombinator.com/item?id=42376014
absolutely to no ones surprise
Unless I am misreading the survey design, they are randomly sampling the 16-65 age group. There is a significant trend of lower scores with age, even within a birth cohort. Under those conditions, it's not clear that the weighting is taking care of that bias. Also their USA results have near the worst non-response bias of any country, and come with cautionary notes about that.
Title should probably say American adults.
Poland, Lithuania, New Zealand, Hungary, Korea, and Slovak Republic all declined more, and that wouldn't all fit in the title.
Reading the article and the linked same report for 2023, the cross-country comparisons seem useless. Looking at the questions given, they're greatly influenced by ones cultural background. Changes in scores for individual countries over time may be worth something, but even then there's big question marks.
Social media and mobile devices destroyed the ability to pay attention to anything for longer than 5 seconds. Of course that’s going to be reflected in test results that require close reading or anything more than surface level thinking.
I didn’t see anything in the article about how it determined “average”. America had ~3 million illegal immigrants enter the country in 2023 alone. That’s almost 1% population growth from people who can’t speak English. I don’t see Finland dealing with this on such a massive scale.
That number is very wrong, so are you lying or are you mistaking border encounters for successful entries that result in growing the population?
Mass migration, legal or not, is changing the face of education in western countries. Australia gained 5% in the past 2 years, the size of another Adelaide. All stats will be distorted
3M may be a bit high but there has been an enormous increase in the illegal alien population over the past few years. As with any criminal activity it's difficult to get accurate numbers.
https://www.newsweek.com/illegal-immigration-costs-us-billio...
As to whether that could impact scores on this particular test, that seems highly speculative. We don't have any data to support such a claim.
https://www.newsweek.com/illegal-immigration-costs-us-billio...
As to whether that could impact scores on this particular test, that seems highly speculative. We don't have any data to support such a claim.
I hear a lot of people speculate this, but frankly we don't need a million people repeating this theory. We need a few scientists looking into it, and the rest repeating whatever the scientists conclude.
I have seen LLMs with more agency than this comment.
Finding a group of people that don’t use either isn’t going to be easy. Especially if you want a broad selection of age groups.
> Tackling gender gaps to boost growth
The test sample material couldn’t represent better its ODCE origin. Everything’s a growth lever.
The test sample material couldn’t represent better its ODCE origin. Everything’s a growth lever.
The biggest declines in literacy are among older adults. So did older Americans forget how to read? Of course not. America has imported an unbelievable amount of human capital including a lot of middle and older aged people that were never educated in their origin countries, etc.
There wasn't a single chapter devoted to actionable items. As much as I like to read 199 pages pf reports, their value goes down rapidly once there is no sense of finding solutions.
This is a good 'pointing-fingers' i-told-you-so kind of bureaucracy. They should know better
This is a good 'pointing-fingers' i-told-you-so kind of bureaucracy. They should know better
> This is a good 'pointing-fingers' i-told-you-so kind of bureaucracy.
Maybe there's just nothing actionable for the readers? That's certainly most situations.
Maybe there's just nothing actionable for the readers? That's certainly most situations.
Reports like this play into a very loud narrative that adults (in the United States) are getting stupider, but I really don't think that's true, and I also don't think that it is supported by these results, for at least two reasons:
1) What the results actually show most starkly is that the 55-65 cohort today had reduced performance compared to the 55-65 cohort in 2017 (who of course are 62-72 today).
> it was older adults whose scores declined the most. Those aged 55 to 65 went down eight 8 points in numeracy...In literacy, the oldest adults declined by a full 16 points
If you are 72, you were born in 1952 and spent your primary and secondary educational years (during which the tested skills are mostly taught) in relatively placid, calm, prosperous times (especially for white people).
The cohort tested in 2024, by contrast, went to school during the Vietnam War, the assassinations of JFK/RFK/MLK, the civil rights movement, etc. Perhaps they had more important things to do than learn the nuances of grammar and sentence structure? If they were tested on empathy, social skills, nuances of political statements, etc., do we expect the trend to stay the same?
2) The PIAAC methodology doesn't appear to account (unless that accounting is mentioned somewhere I'm not seeing) for the diversification of media that has occurred in the intervening years, and especially in the most affected cohorts. More people are consuming their most intellectually stimulating content via audio (in audiobooks or podcasts) or video (via lectures, etc) than ever before.
It seems plausible that our brains have become better wired and adapted for critical thinking in response to audiovisual stimuli rather than written words. I'd love to see how a conversational or audiovisual-based analysis differs from 2017 to today. My hunch is that we'd be seeing improvements, not declines.
1) What the results actually show most starkly is that the 55-65 cohort today had reduced performance compared to the 55-65 cohort in 2017 (who of course are 62-72 today).
> it was older adults whose scores declined the most. Those aged 55 to 65 went down eight 8 points in numeracy...In literacy, the oldest adults declined by a full 16 points
If you are 72, you were born in 1952 and spent your primary and secondary educational years (during which the tested skills are mostly taught) in relatively placid, calm, prosperous times (especially for white people).
The cohort tested in 2024, by contrast, went to school during the Vietnam War, the assassinations of JFK/RFK/MLK, the civil rights movement, etc. Perhaps they had more important things to do than learn the nuances of grammar and sentence structure? If they were tested on empathy, social skills, nuances of political statements, etc., do we expect the trend to stay the same?
2) The PIAAC methodology doesn't appear to account (unless that accounting is mentioned somewhere I'm not seeing) for the diversification of media that has occurred in the intervening years, and especially in the most affected cohorts. More people are consuming their most intellectually stimulating content via audio (in audiobooks or podcasts) or video (via lectures, etc) than ever before.
It seems plausible that our brains have become better wired and adapted for critical thinking in response to audiovisual stimuli rather than written words. I'd love to see how a conversational or audiovisual-based analysis differs from 2017 to today. My hunch is that we'd be seeing improvements, not declines.
This comment should be at the top. There's a lot of nuance here that's being lost in the original article's title/conclusion.
> Unlike tests given to students, these tests attempt to evaluate real-life skills
Bummer that we spend so much time learning fantasy world skills in school.
Bummer that we spend so much time learning fantasy world skills in school.
[deleted]
These tests cannot account for the complexity of the information economy today. They are asking for math proficiency for operations that are not necessarily useful outside of a classroom, they are asking for reading competency that has nothing to do with how most people engage with text today (online, on their phones, in a audio-visual format). This is a useless study.
Edit: nevermind, I took the test. It does address those things, but the implementation is terrible.
Edit: nevermind, I took the test. It does address those things, but the implementation is terrible.