Cognitive reflection, intelligence, and cognitive abilities: A meta-analysis(sciencedirect.com)
sciencedirect.com
Cognitive reflection, intelligence, and cognitive abilities: A meta-analysis
https://www.sciencedirect.com/science/article/pii/S0160289621000982
53 comments
> Basically the idea that you think before you act
They talk about thinking before you judge, not thinking before you act. ADHD people often have a hard time thinking before they act but many of them are still really smart and think a lot before judging.
Judging without thinking is a much bigger problem than doing without thinking, but most people still do it.
They talk about thinking before you judge, not thinking before you act. ADHD people often have a hard time thinking before they act but many of them are still really smart and think a lot before judging.
Judging without thinking is a much bigger problem than doing without thinking, but most people still do it.
That’s a really interesting and important distinction. Thanks for clarifying. Suddenly I understand ignorance and stupidity in a whole new light.
Doing a meta-analysis on something easily quantifiable is sketchy enough, doing a meta-analysis on something as vague and hard to measure as "cognitive intelligence" is.. well, it's sociology's territory. Not to say it isn't of value, many discoveries were made on a lot less methodologically strict grounds and this kind of "conversation" does create the stimulus for further, more specific research, but you ought to read this more like a an investigative journalistic piece with a lot of opinion rather than hard science.
It’s impulsive if it’s blindly obvious.
Someone of intelligence would find that so and have no need to reflect or even think about it.
Someone of intelligence would find that so and have no need to reflect or even think about it.
oh the word games in this thread.
This is an old concept:
System 1 and System 2 (The linked article but also everyone?)
book: Thinking fast and slow (Daniel Kahneman)
analogy: Elephant and Rider (Haidt)
analogy: Charioteer and Horse (stoics)
An impulse is not blindly obvious (e.g., unconscious bias or unquestioned assumptions), and an intelligent person may or may not have correct biases / impulses in all situations. However, a person who can sit with their impulses, override them, and try to see through them through reflection may in fact have something that is synonymous with intelligence to a degree (thesis of TFA).
I believe the implication is that "much of intelligence is reflection" and that "continuous reflection yields better intuition on the reflected domain". Not "a person born with high IQ has better instincts".
This is an old concept:
System 1 and System 2 (The linked article but also everyone?)
book: Thinking fast and slow (Daniel Kahneman)
analogy: Elephant and Rider (Haidt)
analogy: Charioteer and Horse (stoics)
An impulse is not blindly obvious (e.g., unconscious bias or unquestioned assumptions), and an intelligent person may or may not have correct biases / impulses in all situations. However, a person who can sit with their impulses, override them, and try to see through them through reflection may in fact have something that is synonymous with intelligence to a degree (thesis of TFA).
I believe the implication is that "much of intelligence is reflection" and that "continuous reflection yields better intuition on the reflected domain". Not "a person born with high IQ has better instincts".
Just to be sure, "TFA" is Daniel Kahneman's "Thinking, Fast and Slow", I assume ...
Ah no, it was the article we were all discussing here. The one the HN thread is about.
Understood. :)
Man, I'm shocked at how often this has to be reiterated here...
"TFA" means The F[ine]* Article (or insert another 'F' word if you wish), which I first heard on Slashdot, which was perhaps the OG "news aggregator" site back in the 90's and early 2000s, to refer back to the original article which we were all discussing.
This was a back-formation from RTFA, which would have meant, "Read The F[ine] Article", an admonition / accusation not uncommon on Slashdot which is explicitly discouraged here.
RTFA, in turn, was a modification of RTFM, familiar to many of the IT crowd, as an admonition / recommendation often proffered in response to user questions: Read the F[ine] Manual.
Although RTFM was obviously fairly toxic, and RTFA was of the same spirit, TFA by itself simply filled a useful role: a way to refer back specifically to the article we were all (supposedly) discussing.
* There are other F-words sometimes used here, of course
"TFA" means The F[ine]* Article (or insert another 'F' word if you wish), which I first heard on Slashdot, which was perhaps the OG "news aggregator" site back in the 90's and early 2000s, to refer back to the original article which we were all discussing.
This was a back-formation from RTFA, which would have meant, "Read The F[ine] Article", an admonition / accusation not uncommon on Slashdot which is explicitly discouraged here.
RTFA, in turn, was a modification of RTFM, familiar to many of the IT crowd, as an admonition / recommendation often proffered in response to user questions: Read the F[ine] Manual.
Although RTFM was obviously fairly toxic, and RTFA was of the same spirit, TFA by itself simply filled a useful role: a way to refer back specifically to the article we were all (supposedly) discussing.
* There are other F-words sometimes used here, of course
IDK if you just wrote that all on the spot but well done, you need a blog! I knew most of those facts individually but never tied it all together into a narrative, or made the connection with Read the Freakin Manual.
To only slightly elongate this tangent: I’m giving the Reddit crowd cultural victory on this one with “OP” (for Original Post) rather than “TFA”/“TLA”. I’m curious what is typically used on platforms like Instagram and TikTok, but I’m guessing no one here knows lol
To only slightly elongate this tangent: I’m giving the Reddit crowd cultural victory on this one with “OP” (for Original Post) rather than “TFA”/“TLA”. I’m curious what is typically used on platforms like Instagram and TikTok, but I’m guessing no one here knows lol
Everyone learned it somewhere!
Ah. Slashdot. Those were the days.-
Had not made the connection between "TFA" here and that. Much less "RTFM" ...
... which is a classic by now.-
Had not made the connection between "TFA" here and that. Much less "RTFM" ...
... which is a classic by now.-
The OP article cites Kahneman & Frederick quite a bit in the intro.
Our knowledge of a situation is almost always imperfect, and can most of the time benefit from longer, deeper, less blond analysis.
Where impulses and reflexes rule is time-sensitive activities, like most sports, playing musical instruments, or driving. Whoever can deploy a better repertoire of thoughtless, instant reactions wins. It's basically precomputed and cached thought.
Where impulses and reflexes rule is time-sensitive activities, like most sports, playing musical instruments, or driving. Whoever can deploy a better repertoire of thoughtless, instant reactions wins. It's basically precomputed and cached thought.
username135(1)
It is junk science--when a skill is truly mastered, it is done entirely without thinking bout it. Flow. Conscious, reflective thought is very slow and is typically only invoked when all else fails.
When you are programming--really sucked into it, the state of flow---you are not thinking about how you are programming. You are not aware that you have feet, inside of socks, or even that you are sitting down. You are not even aware you are thinking about it at all--you actually disappear altogether, all that really exists is this mind-meld of you, the problem you are trying to solve, and the program you are solving it with.
If, in the middle of this state, you stop and start questioning "am I programing this the right way? Am I sure this is the best way to do it? Should I be making this more general, or generic? All you get is analysis paralysis.
Its a trap even, because bike shedding feels like you are getting things done, or being smart, foresighted, etc etc. You are not; if you know how to solve the problem within spec and budget, doing anything else is just procrastinating.
When you are programming--really sucked into it, the state of flow---you are not thinking about how you are programming. You are not aware that you have feet, inside of socks, or even that you are sitting down. You are not even aware you are thinking about it at all--you actually disappear altogether, all that really exists is this mind-meld of you, the problem you are trying to solve, and the program you are solving it with.
If, in the middle of this state, you stop and start questioning "am I programing this the right way? Am I sure this is the best way to do it? Should I be making this more general, or generic? All you get is analysis paralysis.
Its a trap even, because bike shedding feels like you are getting things done, or being smart, foresighted, etc etc. You are not; if you know how to solve the problem within spec and budget, doing anything else is just procrastinating.
I disagree with this. Outstanding programming (outstanding performance at almost anything) involves a flow state where you are doing-the-thing almost unconsciously, but that frees your mind up for meta-cognition reflecting on the bigger picture and concurrently refining the performance of the task. To be an outstanding athlete, you also need to be your own best coach.
I don't think this is right at all, a lot of intellectual forms of 'flow' necessarily involve incessant reflection, often through writing and re-writing, but also through conversation with self and others. Keep in mind that Mihály Csíkszentmihályi, who pioneered the concept of flow, thought of the manipulation of symbols - including in science, philosophy, conversation - as one of its principal domains.
[deleted]
Hierarchical model for the win. Consistent with Jeff Hawkins research as well.
So video games and quick stimuli make you smarter? :)
I feel its the complete opposite. Playing video games for hours makes me generally incapable of deep thinking. Btw the same happens after hours-long coding sessions.
You're exercising your cognition. It's no surprise that the acute effects include exhaustion. The long term effects are you have stronger cognition.
no, its something different. I found the Kahneman concept of the two systems to be really helpful in explaining - it feels like gaming pushes all cognition into the twitchy system 1 brain, and puts system 2 to sleep. Whereas reading arduous technical text wears system 2 down.
Two completely different types of incapacitations.
Two completely different types of incapacitations.
[deleted]
The study found that your ability to reflect on "your first impulsive response" in order to refine it or discard it, is closely related to your general intelligence ability and numerical abilities. It refutes the idea that cognitive reflection is a separate skill independent of general intelligence.
This is usually what happens when you try to tease out another skill or separate "intelligence" from the general factor, it ends up being correlated in the end.
Well yeah, defects are clustered so if someone is likely to be damaged in one way they're likely to be damaged in another.
Super interesting maxim, thanks for sharing! Of course that’s part of it, but given the spatially distributed nature of the relevant networks (ie we could maybe constrain some of these Piagetian abilities to one hemisphere or lobe of the cortex, at best) I think there’s more at play than adjacency.
I’m personally confident that the four faculties of the brain are each involved in reinforcing the others, and I think that provides a compelling causal mechanism correlating the development of one with increased performance among the others. For example, consider the Metaconscious gradually tweaking a baby’s Unconscious visual networks to improve their sight, and the increase in information input eventually bubbles up to develop the Metaconscious in turn.
I’m personally confident that the four faculties of the brain are each involved in reinforcing the others, and I think that provides a compelling causal mechanism correlating the development of one with increased performance among the others. For example, consider the Metaconscious gradually tweaking a baby’s Unconscious visual networks to improve their sight, and the increase in information input eventually bubbles up to develop the Metaconscious in turn.
I guess, by some measure it should be a separate skill. It appears to be a measurable skill, apart from AGI, so there's that.-
PS. It would appear to be a great, reflexive, quality for an AI to have: the ability to monitor its outputs, and improve on responses, in a "metacognitive" way.-
PS. It would appear to be a great, reflexive, quality for an AI to have: the ability to monitor its outputs, and improve on responses, in a "metacognitive" way.-
Just because something is distinct does not mean it is independent.
> ability to reflect on "your first impulsive response"
This is why it's harder to be brave if you're smart.
This is why it's harder to be brave if you're smart.
Isn't the other way around? Take a fireman, he's running into a burning building, whereas I'd think the first impulse would be to run away, hence he's reflecting on his first impulse to be brave.
We all three of us understood this differently.
I heard the old "Thinking fast and slow" dichotomy all over: You have snap impulses, and you have the ability to override them if you recognize them. Also "elephant and rider" from Haidt's books.
A fireman spends time training his "elephant" (impulses) to be helpful in fire situations, first by overcoming his impulse for fear in the face of fire.
This takes time.
A person who is predisposed to contemplation might appear smarter on tests or be smarter in actuality (see TFA), but that doesn't have much to do with someone who has faced and conquered their fearful impulses enough that they are dampened.
I heard the old "Thinking fast and slow" dichotomy all over: You have snap impulses, and you have the ability to override them if you recognize them. Also "elephant and rider" from Haidt's books.
A fireman spends time training his "elephant" (impulses) to be helpful in fire situations, first by overcoming his impulse for fear in the face of fire.
This takes time.
A person who is predisposed to contemplation might appear smarter on tests or be smarter in actuality (see TFA), but that doesn't have much to do with someone who has faced and conquered their fearful impulses enough that they are dampened.
It's kind of both ways. Your impulse is controlled by your amygdala. Fight or flight, where "fight" in this instance represents running into a building to save others.
However, being able to reflect on your first impulse means you will sit back and assess the situation: "Can I get in?", "What are my chances?", "Am I protected enough?", "Will the structure collapse soon?", etc.
So your first instinct may be to run in, but you hesitate because you want to do that critical thinking first.
However, being able to reflect on your first impulse means you will sit back and assess the situation: "Can I get in?", "What are my chances?", "Am I protected enough?", "Will the structure collapse soon?", etc.
So your first instinct may be to run in, but you hesitate because you want to do that critical thinking first.
I think that's training, which is the opposite of reflection - you train behaviors exactly so you don't dwell on them at the moment.
Citation needed.
It seems to me like it is yet another study which refutes the idea that we're able to scientifically measure "general intelligence" (or "cognitive intelligence," using the author's term). Instead of anything which can be called intelligence, these tests keep measuring the same wrong thing. This thing is clearly amenable to short-term improvement with practice, and possibly assistance from a trained psychologist. And I don't think it has much to do with the actual problems human brains have to solve. The fact that human brains can be trained to solve IQ test problems or math problems is interesting. But surely it's a tiny slice of what human brains can do.
No, it's another datapoint for IQ. General intelligence was discovered empirically, not hypothesized and then searched-for.
The basic observation was that people who score highly on one intelligence-related test will generally score highly on every other intelligence-related test. The "dimensions" of intelligence, while hypothetically separable and distinct, don't actually appear to be so in practice. Smart people are smart at most things (in general), dumb people are dumb at most things (in general).
This only gets shaky when one uses it to make moral arguments about people's value or worth, so I'll just explicitly say it: being smart is not a moral victory or very likely even something you earned at all. Smart and dumb people have equal claim to a dignified life here on earth.
The basic observation was that people who score highly on one intelligence-related test will generally score highly on every other intelligence-related test. The "dimensions" of intelligence, while hypothetically separable and distinct, don't actually appear to be so in practice. Smart people are smart at most things (in general), dumb people are dumb at most things (in general).
This only gets shaky when one uses it to make moral arguments about people's value or worth, so I'll just explicitly say it: being smart is not a moral victory or very likely even something you earned at all. Smart and dumb people have equal claim to a dignified life here on earth.
This is just a distortion, and shows you haven't actually engaged with the scientific criticism of IQ tests:
> General intelligence was discovered empirically, not hypothesized and then searched-for.
General intelligence has been hypothesized - and assumed to exist - by almost all peoples since prehistory. The g factor is what you are referring to, and that was discovered "empirically." What was not at all empirical was the decision to call the g factor a measure of general intelligence. That is a hypothesis which seems flatly wrong, and I have yet to see an argument for its validity that isn't circular or specious. My point is that written cognitive tests don't measure intelligence, they measure something much more shallow. The fact that written cognitive tests have a correlating g factor does not mean that the g factor itself is any less shallow.
> General intelligence was discovered empirically, not hypothesized and then searched-for.
General intelligence has been hypothesized - and assumed to exist - by almost all peoples since prehistory. The g factor is what you are referring to, and that was discovered "empirically." What was not at all empirical was the decision to call the g factor a measure of general intelligence. That is a hypothesis which seems flatly wrong, and I have yet to see an argument for its validity that isn't circular or specious. My point is that written cognitive tests don't measure intelligence, they measure something much more shallow. The fact that written cognitive tests have a correlating g factor does not mean that the g factor itself is any less shallow.
From the first line of the Wiki article: "The g factor (also known as general intelligence,...)"
I was referring to g, not to the concept of general intelligence, which very obviously has pretty much always existed. Apologies for the ambiguity.
In any case, I don't think it really matters how precisely or reliably we can actually measure g directly, which is really what your sticking point is. Every important decision we need to make around the idea of general intelligence has to do with 1) whether it exists, 2) how well it correlates to positive outcomes in life under specific social and economic regimes, and 3) how heritable it is. All of these seem pretty darn obvious from various experiments not relying on measuring g directly.
Why do you think it matters whether we can measure g? What decisions are there to be made with that information?
I was referring to g, not to the concept of general intelligence, which very obviously has pretty much always existed. Apologies for the ambiguity.
In any case, I don't think it really matters how precisely or reliably we can actually measure g directly, which is really what your sticking point is. Every important decision we need to make around the idea of general intelligence has to do with 1) whether it exists, 2) how well it correlates to positive outcomes in life under specific social and economic regimes, and 3) how heritable it is. All of these seem pretty darn obvious from various experiments not relying on measuring g directly.
Why do you think it matters whether we can measure g? What decisions are there to be made with that information?
> The "dimensions" of intelligence, while hypothetically separable and distinct, don't actually appear to be so in practice.
Circularity. People have normally distributed IQ test results because the numbers are sent through a process that makes them normally distributed.
If this was true then the actual raw test results wouldn't come in different categories. (And there wouldn't be "autistic savants". And being a grandmaster chess player would transfer to other fields instead of coming entirely from practicing since age 5.)
Circularity. People have normally distributed IQ test results because the numbers are sent through a process that makes them normally distributed.
If this was true then the actual raw test results wouldn't come in different categories. (And there wouldn't be "autistic savants". And being a grandmaster chess player would transfer to other fields instead of coming entirely from practicing since age 5.)
1. I wasn’t making any claim to the distribution of IQ test results across the population.
2. You can have different categories and still have correlations between them
3. Presence of outliers (savants) doesn’t discredit the general correlation
4. No one is claiming skills cannot be learned
2. You can have different categories and still have correlations between them
3. Presence of outliers (savants) doesn’t discredit the general correlation
4. No one is claiming skills cannot be learned
> Presence of outliers (savants) doesn’t discredit the general correlation
You're now claiming a "correlation", but a correlation is meaningless so we don't need to credit it in the first place - since, you know, correlation is not causation. It is evidence against a useful causation.
(If the causation is "low IQ causes poor skills except when it doesn't" this isn't useful since it's a tautology.)
> No one is claiming skills cannot be learned
You were claiming that smart people are better at everything before this, but it's well known that they're not better at this even though it's a purely mental skill.
Strictly speaking you were claiming that people better at some test are better at every test, but presumably you meant to generalize there, since if they just had high "test taking skills" nobody would care. Also a likely circularity lurking in the definition of "test"…
You're now claiming a "correlation", but a correlation is meaningless so we don't need to credit it in the first place - since, you know, correlation is not causation. It is evidence against a useful causation.
(If the causation is "low IQ causes poor skills except when it doesn't" this isn't useful since it's a tautology.)
> No one is claiming skills cannot be learned
You were claiming that smart people are better at everything before this, but it's well known that they're not better at this even though it's a purely mental skill.
Strictly speaking you were claiming that people better at some test are better at every test, but presumably you meant to generalize there, since if they just had high "test taking skills" nobody would care. Also a likely circularity lurking in the definition of "test"…
I didn’t claim anything other than a correlation, and no, I don’t see why causation is important but correlation is not. What decision hinges on this?
You’re speaking as if there’s no correlation between IQ and performance in chess. This is not true. https://www.sciencedaily.com/releases/2016/09/160913124722.h...
“Strictly speaking,” no, I didn’t say people better at some test are better at every test.
You’re speaking as if there’s no correlation between IQ and performance in chess. This is not true. https://www.sciencedaily.com/releases/2016/09/160913124722.h...
“Strictly speaking,” no, I didn’t say people better at some test are better at every test.
Yes, intelligence is more a multiplier of effects, rather than something useful in and of itself.
You still need knowledge, skill, training, etc. Intelligence will allow you to acquire all of that more quickly and make better use of it, but intelligence without knowledge is useless.
And just like taller people aren't better than shorter people, more intelligent people aren't better than less intelligent people. Nothing can make you a good person except your choices to do good things.
You still need knowledge, skill, training, etc. Intelligence will allow you to acquire all of that more quickly and make better use of it, but intelligence without knowledge is useless.
And just like taller people aren't better than shorter people, more intelligent people aren't better than less intelligent people. Nothing can make you a good person except your choices to do good things.
> more intelligent people aren't better than less intelligent people.
I completely agree with this, and I wish it was something that was a bit more emphasized.
However, I do think there lies the dangers of intelligence research. In seems that so much of the world is catered towards the idea that more intelligence = better person. I suppose the notion is that more intelligence people can produce greater value, thus they are more important/beneficial to society?
In society, we have "gifted" programs, "elite" universities, "knowledge" workers, etc.. I'm probably rambling and wording this all poorly, but I guess what I am trying to allude to is that society definitely ranks people's value by intelligence in overt and subtle ways.
Don't you think most parents would rather have a child with average empathy with gifted intelligence than a child that with gifted empathy and average intelligence?
(Tangent below)
I was required to take a legitimate IQ test (WAIS-IV) for the purposes of diagnosing/confirming a disability when I was in college. I'm not kidding when I say this, but I wish I was able to remain ignorant of the composite and subtest scores. Sure, it told me somethings about myself, but at the same time, I felt like it was some kind of report describing what domains my aspirations and limitations in life should be confined to.
To me, it seems like intelligence is not important to those that have a lot of it, and it's significant to those that lack it (whether they realize it or not).
I completely agree with this, and I wish it was something that was a bit more emphasized.
However, I do think there lies the dangers of intelligence research. In seems that so much of the world is catered towards the idea that more intelligence = better person. I suppose the notion is that more intelligence people can produce greater value, thus they are more important/beneficial to society?
In society, we have "gifted" programs, "elite" universities, "knowledge" workers, etc.. I'm probably rambling and wording this all poorly, but I guess what I am trying to allude to is that society definitely ranks people's value by intelligence in overt and subtle ways.
Don't you think most parents would rather have a child with average empathy with gifted intelligence than a child that with gifted empathy and average intelligence?
(Tangent below)
I was required to take a legitimate IQ test (WAIS-IV) for the purposes of diagnosing/confirming a disability when I was in college. I'm not kidding when I say this, but I wish I was able to remain ignorant of the composite and subtest scores. Sure, it told me somethings about myself, but at the same time, I felt like it was some kind of report describing what domains my aspirations and limitations in life should be confined to.
To me, it seems like intelligence is not important to those that have a lot of it, and it's significant to those that lack it (whether they realize it or not).
Weirdly, I'd say exceptional skill/talent winds up fostering empathy rather than hindering it.
When you have exceptional talent, you quickly learn not everyone is like you and that you need to understand their perspectives. Whereas if you're within the average range, you think everyone should be in the average range, so you can wind up overly frustrated with both those below it and those above it.
People mistakenly believe that a proficiency in one area means a deficiency in others, like we're all given a set of stat points at birth we have to allocate. When the truth is some people are just generally proficient. There are people with higher intelligence and greater empathy. It's not a choice. And ironically, empathy is one of those things that is actually positively correlated with intelligence. Which kind of tracks as a component of intelligence is pattern recognition and empathy is a form of pattern recognition.
When you have exceptional talent, you quickly learn not everyone is like you and that you need to understand their perspectives. Whereas if you're within the average range, you think everyone should be in the average range, so you can wind up overly frustrated with both those below it and those above it.
People mistakenly believe that a proficiency in one area means a deficiency in others, like we're all given a set of stat points at birth we have to allocate. When the truth is some people are just generally proficient. There are people with higher intelligence and greater empathy. It's not a choice. And ironically, empathy is one of those things that is actually positively correlated with intelligence. Which kind of tracks as a component of intelligence is pattern recognition and empathy is a form of pattern recognition.
If you have the time, I am curious what you think about this post, "Three-Toed Sloth."
http://bactra.org/weblog/523.html
It is obviously just one person's blog post, but I did find it interesting and compelling. However, I must admit that some of the areas of discussion surpass my level of knowledge of statistics and psychometrics.
I am just curious what your opinions are since you seem much more versed in this topic than I am.
http://bactra.org/weblog/523.html
It is obviously just one person's blog post, but I did find it interesting and compelling. However, I must admit that some of the areas of discussion surpass my level of knowledge of statistics and psychometrics.
I am just curious what your opinions are since you seem much more versed in this topic than I am.
To be clear I’m nowhere close to an expert. But my reaction to that blog post is that I’m not quite sure what it’s rebutting? It seems to argue (successfully, I guess) that g isn’t some singular source of intellect that’s powering all other dimensions of intellect. I am not sure if other people believe this to be the case, but I certainly don’t, as that seems contrary to the structure of the brain itself. There is no “smartness” lobe that’s supporting the “smartness” of all the other things the brain is doing and smart people have a bigger/better smartness lobe than dumb people. That’s just not what the brain looks like.
What this blog post does say, which I believe to be true, is that intelligence on one dimension is highly correlated with intelligence on other dimensions. Not because they’re necessarily all driven by the “same source,” which seems like an implementation detail with, AFAICT, pretty much zero bearing on anything of practical concern?
What this blog post does say, which I believe to be true, is that intelligence on one dimension is highly correlated with intelligence on other dimensions. Not because they’re necessarily all driven by the “same source,” which seems like an implementation detail with, AFAICT, pretty much zero bearing on anything of practical concern?
[deleted]
Basically the idea that you think before you act. Makes sense that this would enhance all other skills. Is this junk science? I’d rather see actual test plus data with control variables. Meta analysis seems to have the conclusion already in the introduction. We think this matters here’s a bunch of evidence we found, therefore it matters.