Why "Freakonomics" failed to transform economics(economist.com)
economist.com
Why "Freakonomics" failed to transform economics
https://www.economist.com/finance-and-economics/2024/03/21/why-freakonomics-failed-to-transform-economics
93 comments
I think that, much like with Thinking, Fast and Slow, anyone recommending it needs to add a caveat that some of the results didn't hold up.
Ideally, there were would be revised editions with mistakes corrected.
Ideally, there were would be revised editions with mistakes corrected.
The trouble with doing that for Freakonomics is that the work on abortions reducing crime, which has been proven wrong, is the first chapter and the centerpiece of the book. It's the thing that they use to exemplify the "freakonomics" approach in the rest of the book.
Yeah, it would be a major revision.
do you have links to the papers disproving that? That result is kind of the basis for the moral argument for allowing abortion for consensual fetuses for me. Without that result the cost/benefit looks terrible because the deflationary death spiral of the below replacement birthrate it helps cause is really, really bad.
Per Wikipedia [0], the Freakonomics analysis holds up. Levitt addressed the disagreement on a podcast as well [1].
[0]: https://en.m.wikipedia.org/wiki/Legalized_abortion_and_crime...
[1]: https://freakonomics.com/podcast/abortion-and-crime-revisite...
[0]: https://en.m.wikipedia.org/wiki/Legalized_abortion_and_crime...
[1]: https://freakonomics.com/podcast/abortion-and-crime-revisite...
I'm not even an armchair economist but when I read the book many years ago, I could still tell that some of the stories and conclusions were anecdotal at best. It was still a fun read and fun to think about. The best thing about it besides entertainment value was don't take what you read at face value, even if (maybe especially) it's popular.
> was don't take what you read at face value, even if (maybe especially) it's popular.
Its a popular book that people take very seriously
Its a popular book that people take very seriously
Hard not to read this as, "because some old guard economists don't like fun things."
I am not overly fond of some of the ideas in freakonomics, but I can't deny the fun of looking at things and trying to tease out explanations. If anything, I would want to laud getting people to even try replication at a good thing. I fear if we only enjoy the flawless replications, we set a bit too high of a bar.
I am not overly fond of some of the ideas in freakonomics, but I can't deny the fun of looking at things and trying to tease out explanations. If anything, I would want to laud getting people to even try replication at a good thing. I fear if we only enjoy the flawless replications, we set a bit too high of a bar.
Freakonomics is actually covered in some college econ textbooks (usually as an aside that reinforces a core concept). It's hard to say which one as I've gone through like 7 so far in the program I'm doing. Most of the textbooks I've read have been VERY "cool" and approachable with a decent amount of humor from the authors.
“A recent study by economists at the Federal Reserve found that less than half of the published papers they examined could be replicated, even when given help from the original authors. Mr Levitt’s counterintuitive results have fallen out of fashion and economists in general have become more sceptical.”
Didn’t realize replication was so bad in Econ too.
Didn’t realize replication was so bad in Econ too.
Depends on which part of econ. Micro and econometrics fare pretty well (there was even an econ pseudo-Nobel awarded to Vernon Smith for figuring out how to do reproducible experiments in micro), but macro is the darkest of swamps.
If the purpose of half of economics study was to justify political decisions, rather than to learn facts about the world, that replication rate would be what you'd expect.
> If the purpose of half of economics study was to justify political decisions
I mean, let’s be honest, is it _not_ actually this? Maybe economics should just give up the charade and accept its role as a political tool and not something with a scientific base so comically bad, even the softest of other sciences look positively mathematical in comparison.
I mean, let’s be honest, is it _not_ actually this? Maybe economics should just give up the charade and accept its role as a political tool and not something with a scientific base so comically bad, even the softest of other sciences look positively mathematical in comparison.
There are lots of good reasons why you can't admit that it's a con, most of which are some variation on "it's good to lie."
If you want to use propaganda but you tell people your propaganda is propaganda, it won't work.
Which specific social sciences do you believe are doing better?
None of them I would guess.
Math is math and the hard sciences like physics, chemistry, biology,and engineering are all good assuming somebody isn't mining p-values or something like that.
The social sciences require a ton of data and simulating something like the economy with trillions of variables and bounded rationality just isn't possible. If you wanted to simulate an agent like myself you'd likely need millions of lines of probabilistic code and then you'd need millions of other agents running on a supercomputer the size of Rhode Island.
Math is math and the hard sciences like physics, chemistry, biology,and engineering are all good assuming somebody isn't mining p-values or something like that.
The social sciences require a ton of data and simulating something like the economy with trillions of variables and bounded rationality just isn't possible. If you wanted to simulate an agent like myself you'd likely need millions of lines of probabilistic code and then you'd need millions of other agents running on a supercomputer the size of Rhode Island.
Biomed is already too complex to simulate fully, which is why simulating is just, not really a thing that is done to find new knowledge. Heck, some quantum interactions are already (so far) computationally intractable, which is why quantum chemistry exists as a field at all, it's far from trivial. That doesn't mean the best practices developed for medicine won't help in the social sciences as well.
The issue with degrees of freedom can be mostly pinned down with preregistration, among other measures. It won't solve the problem of having more type I errors at any specific error rate the more science you do, but it does give assurances the target type I error rate is accurate.
The issue with degrees of freedom can be mostly pinned down with preregistration, among other measures. It won't solve the problem of having more type I errors at any specific error rate the more science you do, but it does give assurances the target type I error rate is accurate.
Econ is a notorious "nothing and everything topic" (not being fair here, but economists also talk about this). It doesn't really fit anywhere. It's not very practical to study scientifically.
I don't want to insult Sociologists here, but economics can be very much like sociology for money issues.
It's intuitive that replication would be terrible in economics (not taking a shot at you for not "realizing"; we don't think about every subject in the world).
What you really want to do is take one society, run it for 5-200 years. Then go back in time, take the same society, and try again with one small change. In practice, you might need to repeat many times because you would expect chance to change such a complicated system (butterfly effect).
"In practice"... listen to me, I'm being absurd. This is clearly impossible. Of course there's ways to try anyway, but they're all known to be "bad science".
I don't want to insult Sociologists here, but economics can be very much like sociology for money issues.
It's intuitive that replication would be terrible in economics (not taking a shot at you for not "realizing"; we don't think about every subject in the world).
What you really want to do is take one society, run it for 5-200 years. Then go back in time, take the same society, and try again with one small change. In practice, you might need to repeat many times because you would expect chance to change such a complicated system (butterfly effect).
"In practice"... listen to me, I'm being absurd. This is clearly impossible. Of course there's ways to try anyway, but they're all known to be "bad science".
It's bad. But the field has moved towards causal inference methods anyhow, and away from the "cutenomics" Freakonomics championed.
I mean half went be bad but like 10-20% would be the expected if it matched the other sciences
[deleted]
I used to love the podcast in particular. But then it sort of went in this annoying lifestyle direction with very rare involvement from Levitt and had nothing to do with it's original subject matter.
The podcast was good for awhile, but I think they mined all the novel ideas and just turned into a stop on the book promotion tour so many authors take.
Levitt has his own podcast, "people I mostly admire."
Interview format. I really enjoy his questions and perspectives. He asks things I don't really hear from other interviewers.
Interview format. I really enjoy his questions and perspectives. He asks things I don't really hear from other interviewers.
https://archive.is/Wlmsf
Great to see all the subscribers to The Economist commenting in this thread so far!
Great to see all the subscribers to The Economist commenting in this thread so far!
Freakonomics is fun and clever.
A more subtle and standard economics book is probably more correct.
But then it would be boring and I wouldn't have read it.
A more subtle and standard economics book is probably more correct.
But then it would be boring and I wouldn't have read it.
It's fun and clever, but is it true?!?! [or if you think it doesn't matter whether it's true and replicable, it's great salon chatter material, but that wouldn't be science, it would be something above Malcolm Gladwell-grade material.]
It's more from the "interesting-if-true" school of clickbait. It popularized the "amazing counterintuitive results!" school of airport books along with Ariely, Kahneman.
(Conversely, it did have a few benefits: teaching the layman about experiment design, correlation-does-not-imply-causation, instrumental variables.)
(The most plausible-sounding counterintuitive which Freakonomics radio program (on NPR) reported was the claim that participating in the FIFA World Cup tended to reduce overall internal corruption in a country, because if the sports ministry embezzled the money or didn't hire good staff and use it effectively (or at the most basic level, pay players' wages, facilities, travel, jerseys), the country's national team would tend to get beaten embarrassingly by their peers, and the outraged populace would demand reform, not just in sports but other sectors. However, better check if that was replicable or not; it should be, because international tournaments, World Cup player strikes or scandals plus Transparency.org rankings give us "natural experiments" every 2-4 years.)
It's more from the "interesting-if-true" school of clickbait. It popularized the "amazing counterintuitive results!" school of airport books along with Ariely, Kahneman.
(Conversely, it did have a few benefits: teaching the layman about experiment design, correlation-does-not-imply-causation, instrumental variables.)
(The most plausible-sounding counterintuitive which Freakonomics radio program (on NPR) reported was the claim that participating in the FIFA World Cup tended to reduce overall internal corruption in a country, because if the sports ministry embezzled the money or didn't hire good staff and use it effectively (or at the most basic level, pay players' wages, facilities, travel, jerseys), the country's national team would tend to get beaten embarrassingly by their peers, and the outraged populace would demand reform, not just in sports but other sectors. However, better check if that was replicable or not; it should be, because international tournaments, World Cup player strikes or scandals plus Transparency.org rankings give us "natural experiments" every 2-4 years.)
I found the episode, but although it cited some sources, can't see an actual publication with that finding, but other sources plus some plausible conjecture by people who are fairly informed:
Freakonomics Radio Episode #338 "How to Catch World Cup Fever" Jun 13, 2018 https://freakonomics.com/podcast/how-to-catch-world-cup-feve...
[5m00s-onwards]" the World Cup isn’t just a gargantuan sporting event; it’s a microcosm of human foibles and (yep) economic theory brought to life."
Click on 'Read Full transcript'. In the audio, jump to 11m29s, where they talk to Stefan Szymanski, professor of economics at the University of Michigan and co-author of Soccernomics.
Freakonomics Radio Episode #338 "How to Catch World Cup Fever" Jun 13, 2018 https://freakonomics.com/podcast/how-to-catch-world-cup-feve...
[5m00s-onwards]" the World Cup isn’t just a gargantuan sporting event; it’s a microcosm of human foibles and (yep) economic theory brought to life."
Click on 'Read Full transcript'. In the audio, jump to 11m29s, where they talk to Stefan Szymanski, professor of economics at the University of Michigan and co-author of Soccernomics.
In the podcast: 14m28s: comparing historical GDP data and [national-team] soccer goals
Also book: Stefan Szymanski, "Football economics and policy" (2010)
https://www.google.com/books/edition/Football_Economics_and_...
Also book: Stefan Szymanski, "Football economics and policy" (2010)
https://www.google.com/books/edition/Football_Economics_and_...
Without reading the article admittedly, but probably because it’s social science which intrinsically suffers from the replication crisis. Fun book - but replicable?
Ding Ding. With the added story Levitt told on Jon Hartley's podcast about the extremes Heckman went to, to disparage Levitt - translated into the terse Economist house style (which doesn't do justice to the utter toxicity of Chicago Economics...).
If Books Could Kill (a podcast dunking on crappy airport books from the 2000s) did their first episode on Freakonomics:
https://podcasts.apple.com/us/podcast/freakonomics/id1651876...
https://podcasts.apple.com/us/podcast/freakonomics/id1651876...
Great first episode from a great series of podcasts. Have to say though that although it's usually totally justified, the relentless cynicism Mike and Peter employ when dispatching these books unfortunately begins to grate after a while. I still fully recommend listening until that happens though.
Your Favourite Band Sucks is popular despite most people only listening to one or two episodes before they're sick of the format.
No doubt there’s some gems in there, but also some horrendously bad calls. They have an episode deriding the ridiculous lab leak theory for COVID… which is now one of the official explanations according to a US Senate report.
That's a glaring red flag that they're overly conventional thinkers. Independent-minded people were generally either proponents of the lab leak theory all along or were not going to ridicule it as "obviously ridiculous". Likewise, I'd be suspicious of somebody who very strongly argued in favor of "masks don't work" in March 2020 when that was the official position of the US CDC.
People whose opinions I'd consider worth listening to are the ones who don't automatically change their position whenever their tribe's party line changes.
People whose opinions I'd consider worth listening to are the ones who don't automatically change their position whenever their tribe's party line changes.
The CDC never had any official position that masks don't work. In March of 2020, the CDC official position was that people not working in healthcare or caring for the at-risk do not need to wear masks. in the same announcement they said
"Facemasks may be in short supply and they should be saved for caregivers"
in other words, masks work, but we need to save them for where they will do the most good.
however, the Trump admin's messaging was an unholy pile of hot burning garbage, so one could be forgiven for thinking the official recommendations were that masks don't work.
"Facemasks may be in short supply and they should be saved for caregivers"
in other words, masks work, but we need to save them for where they will do the most good.
however, the Trump admin's messaging was an unholy pile of hot burning garbage, so one could be forgiven for thinking the official recommendations were that masks don't work.
I guess strictly speaking is was the official position of the US government, but not the CDC:
https://www.latimes.com/science/story/2021-07-27/timeline-cd...
[deleted]
It wasn’t just, or even primarily, the Trump administration.
It was hordes of social media personalities trying to sound Very Smart and establishing themselves as authorities during a time of crisis.
It was hordes of social media personalities trying to sound Very Smart and establishing themselves as authorities during a time of crisis.
I would argue the complete opposite. One of Michael Hobbes' persistant cri de couers is that the media is overwhelmingly only capable of covering the meta-conversation around the reactions to various claims and seem totally incapable and incurious about digging into the meat of the claims themselves, despite it being methodologically trivial to do so.
One of their most recent episodes was on the Claudine Gay plagiarism scandal where they systematically dug into every one of the 54 cases of plagiarism and demonstrated just how much of it was bad faith right wing bullshit that didn't stand up to even the slightest intellectual scrutiny. But the larger point they were making was not about the right, for which they expected intellectual dishonesty, but the complicity of the left & mainstream media in only ever talking about the scandal in terms of who is saying what about it and why and not on the meat of the facts on the ground.
They took the exact same approach to the lab leak episode where they said let's ignore all of the media meta-narrative and look at just the plain facts on the ground and they found that, like most things, it was a sandcastle built of air that collapsed when you even looked at it funny.
One of their most recent episodes was on the Claudine Gay plagiarism scandal where they systematically dug into every one of the 54 cases of plagiarism and demonstrated just how much of it was bad faith right wing bullshit that didn't stand up to even the slightest intellectual scrutiny. But the larger point they were making was not about the right, for which they expected intellectual dishonesty, but the complicity of the left & mainstream media in only ever talking about the scandal in terms of who is saying what about it and why and not on the meat of the facts on the ground.
They took the exact same approach to the lab leak episode where they said let's ignore all of the media meta-narrative and look at just the plain facts on the ground and they found that, like most things, it was a sandcastle built of air that collapsed when you even looked at it funny.
It had been the official position of the CDC for decades that masks are not effective for stopping a respiratory virus. They did an about face with covid for unknown reasons.
Do you have a link to official publications saying this?
Are we talking about the October 2022 report by Senate Republicans or the April 2023 report from, again, Senate Republicans? Because as I understand it, the official political explanation according to Republicans of a lot of things doesn't exactly comport with reality, so I'm not sure why you believe we should trust them here, given the track record. Unless the idea was to push a particular point of view.
A report from senate Republicans, who are in the minority.
Thanks for posting this, these two are hilarious, I was laughing out loud the entire time.
Thank you for suggesting this podcast. Sounds very interesting. Will check out other episodes as well.
istultus(7)
[deleted]
I appreciated the attempts by Steven Levitt and Stephen Dubner to communicate with layman non-economists like me. I think The Economist article was, as always, too harsh and nitpicky. I think Freakonomics holds a special place in the "intellectual" and "rationalists" community. I cannot verify the flaws in techniques and conclusion as stated in this article but I would still recommend reading Freakonomics first over the dry economics textbooks or MIT OCW courseware.