An Elaborate Academic-Journal Hoax(chronicle.com)
chronicle.com
An Elaborate Academic-Journal Hoax
https://www.chronicle.com/article/Sokal-Squared-Is-Huge/244714
155 comments
Also "not rigorous research" != "effort would be better spent looking at more-substantive problems."
Rigor and importance are orthogonal concepts. Someone could do research of great rigor on a completely unimportant problem and people do bad research all the time on important ones.
Rigor and importance are orthogonal concepts. Someone could do research of great rigor on a completely unimportant problem and people do bad research all the time on important ones.
>> The whole point here is that you have these "conspicuously ideological" fields
or subfields, which are not doing real scholarship, not interested in truth,
but which rather are pushing a specific position and attempting to label as
evil and shout down anyone who disagrees;
Yeah, look- it's not that simple. In academia, we have to allow discussion and debate to be carried out freely, but obviously it doesn't make sense to dictate the subject of a debate and call it "free"! And the only people who are qualified to choose the subject of the debate are academics themselves- because nobody else has the expertise to make that decision.
So if academics choose to debate identity politics and "Theory"- well, that's what they get to debate. If the debate is done in a rotten manner that is not conducive to good scholarship, now that's another matter, but if the question is if those fields should exist, the answer is yes, and the reason is that they do (it's a tautology, OK?).
It's a feature, not a bug. It's a bit like in a democracy you must allow everyone their political opinion, even those people who are against democracy itself? In academia, we must allow all academics the ability to investigate the subjects of their interest, else we don't have academia, we just have a closed club of people who agree with each other- so, just like the fields targeted by the hoax.
Yeah, look- it's not that simple. In academia, we have to allow discussion and debate to be carried out freely, but obviously it doesn't make sense to dictate the subject of a debate and call it "free"! And the only people who are qualified to choose the subject of the debate are academics themselves- because nobody else has the expertise to make that decision.
So if academics choose to debate identity politics and "Theory"- well, that's what they get to debate. If the debate is done in a rotten manner that is not conducive to good scholarship, now that's another matter, but if the question is if those fields should exist, the answer is yes, and the reason is that they do (it's a tautology, OK?).
It's a feature, not a bug. It's a bit like in a democracy you must allow everyone their political opinion, even those people who are against democracy itself? In academia, we must allow all academics the ability to investigate the subjects of their interest, else we don't have academia, we just have a closed club of people who agree with each other- so, just like the fields targeted by the hoax.
Huh? I'm not claiming that these topics should be off-limits; I'm saying that the actual existing departments presently studying them are rotten. In short I'm not sure we're disagreeing here.
Edit: If you like, you can replace my use of "fields and subfields" above with "departments and segments of such". That's probably fairer in other ways, too.
Edit: If you like, you can replace my use of "fields and subfields" above with "departments and segments of such". That's probably fairer in other ways, too.
Yeah, OK, maybe I misread your comment.
[deleted]
> and the "grievance scholars" are slowly convincing the rest of academia that real scholarship consists not of neutral methods and honest argument, but agreeing with the grievance scholars in all things, and finding ways to inject it whether it's relevant or not.
What degrees or disciplines would you characterise as "grievance scholars", and what evidence do you have that suggests the influence those disciplines have over scholarship is increasing?
The amount of degrees awarded in women's studies over the last 10 years has significantly dropped.
What degrees or disciplines would you characterise as "grievance scholars", and what evidence do you have that suggests the influence those disciplines have over scholarship is increasing?
The amount of degrees awarded in women's studies over the last 10 years has significantly dropped.
So, first off, "grievance studies" isn't my term. But as for evidence... well, I don't have statistics; it's more of a "look around you" thing. It's not clear to me why I would care about the number of degrees awarded in women's studies; that doesn't at all seem a reliable proxy for what I do care about, i.e., to what extent the academy is becoming less free and more ideological.
So, is it fair for me to say that your criticism of "grievance studies", a criticism of its unscientific and fallible nature, is anecdotal?
...yes? Although, perhaps I'm misunderstanding, but it seems to me that you've entirely mistaken what my complaint is; as best as I can tell you're trying to imply that my complaint is hypocritical by lacking large-scale statistics, when I'm not complaining about a lack of large-scale statistics at all, but rather such things as argument from consequences, lack of freedom of argument, etc.
But if your argument kind of hinges on the idea that these disciplines axiomatically believe in their righteousness due to their lack of scientific data, wouldn't it be quite strange to say you believe they're making scholarship more ideological, without the data to back that axiom up?
> But if your argument kind of hinges on the idea that these disciplines axiomatically believe in their righteousness due to their lack of scientific data
But that's not my argument at all? My argument is that these disciplines have destroyed the preconditions, the negative feedback loop, that keep a group of people truth-tracking, which includes norms of free inquiry and free argument, a basic understanding of what is and is not an argument (e.g. ad hominem and argument from consequences are not arguments), and have instead fallen into the positive feedback loop that results when these are removed, allowing tribal effects to dominate. I haven't said anything about lack of large-scale data. I mean, yes, large-scale data is important when possible, but it's hardly the most important thing. And indeed these fields often do have large-scale data, they just lack coherent arguments that this data supports their claim (data rarely bears directly on the claim being made, so you still need argument!).
But that's not my argument at all? My argument is that these disciplines have destroyed the preconditions, the negative feedback loop, that keep a group of people truth-tracking, which includes norms of free inquiry and free argument, a basic understanding of what is and is not an argument (e.g. ad hominem and argument from consequences are not arguments), and have instead fallen into the positive feedback loop that results when these are removed, allowing tribal effects to dominate. I haven't said anything about lack of large-scale data. I mean, yes, large-scale data is important when possible, but it's hardly the most important thing. And indeed these fields often do have large-scale data, they just lack coherent arguments that this data supports their claim (data rarely bears directly on the claim being made, so you still need argument!).
Which tribal effects do you think dominate? I can make a guess at what thinkers you may be following, given some of the words you've used here. I'm assuming this, but lets just say for a moment you believe that these disciplines uncritically believe that trans women are "real" women.
This is simply, untrue. There are debates about both the semantics and broad claims in that specific case in these fields of study. I often find that criticisms of these fields lack a nuanced understanding of the actual content and behavior of the fields, and instead come from a malformed reading of the field that is readily accepted by the reader. Are there other specific beliefs in these fields that you're more concerned about?
This is simply, untrue. There are debates about both the semantics and broad claims in that specific case in these fields of study. I often find that criticisms of these fields lack a nuanced understanding of the actual content and behavior of the fields, and instead come from a malformed reading of the field that is readily accepted by the reader. Are there other specific beliefs in these fields that you're more concerned about?
> trans women are "real" women.
That's not a false claim but rather a meaningless one; if people are debating that they have other problems. But, I guess, less serious ones; everyone makes meaningless statements occasionally, after all. One should avoid debating meaningless propositions, yes, but that's a mistake you can learn to avoid given good norms of discourse; getting those good norms of discourse is the more fundamental problem. (And of course it's not always obvious in advance what statements are meaningless; that's something that requires discussion to determine.)
Also, please don't put words in my mouth. Which you initially avoided doing, presenting your example purely as an example; but then you attempted to rebut this example as if this were an argument against my claims, which, obviously, makes no sense.
> I can make a guess at what thinkers you may be following
Would you care to make that explicit, since it sounds like you can?
In any case I'd be a little cautious of identifying me too closely with anyone established. If you want to know what I think more generally... well, you can find my writing scattered throughout the comments sections on such places as Slate Star Codex, Shtetl-Optimized, and Thing of Things. :P I've been meaning to write some of this up in a more visible form but, uh, I think I might want to get a new handle first...
> Which tribal effects do you think dominate?
Um, the usual ones? Do I really need to go over how tribalism works? Well, OK. Here are some characteristics, briefly:
Norms of free inquiry weaken and eventually vanish. Arguments are not evaluated by their reasoning, but by a combination of their conclusions (and their conclusions are evaluated by whether they support the tribal creed and the tribe's supremacy over its outgroups) and the social status within the tribe of the person making it. Everything becomes personal; principles are stated but ignored, in that they are applied selectively -- if a principle would be applied in a way that restrains someone in high standing in the group, or help someone in the outgroup, a reason for an exception is found. Agreeing with the tribe becomes considered as essential to personal ethics. Even just asking for clarification about a particular principle marks you as an evil outsider, because if you were a good person you would already understand. The group becomes taken over by those who only know how to politick and ceases to be about anything except its own dominance and the dominance of those who have taken it over. Group beliefs become more and more detached from reality.
I mean, I could go on, but I'm hoping that's sufficient.
This process is, of course, constantly happening everywhere, because it's human nature. But with a good negative-feedback loop, with good defenses against this, you can keep this positive-feedback loop in check, avoid the disastrous later stages, and (over the long term) keep yourself aligned with reality. Cut the negative-feedback loop and disaster is inevitable, even if its particular form is unpredictable.
("Tribalism" may be the wrong word here; I tend to use it as shorthand since these things tend to go together with tribalism proper. But the important thing is the phenomenon being described -- where the nominal purpose of a thing becomes completely corrupted by social considerations and interpersonal politicking -- not what we call it.)
We do have good defenses against this, if perhaps not on the mass scale. But certain people keep trying to toss out those defenses as unnecessary...
> Are there other specific beliefs in these fields that you're more concerned about?
I think this is a bad way of framing things. My concern here is with the meta-level, not the object-level. I'm not concerned not primarily about particular claims I see as wrong -- which, after all, may be correct (determining the truth of many of these claims does require large-scale data, something I'm not in a position to collect or trained in evaluating), but rather the norms of discourse surrounding them. Sure, a few might be correct by coincidence, but being reliably right requires good methods. Happening to get a few things right in your tribal creed is worth no credit.
To the extent I'm concerned with particular object-level claims it's primarily about how they're defended, about the reasoning given for them, not with the claims themselves. (The exception, of course, is those object-level beliefs that most directly justify this illiberalism and warped norms of discourse. I say "most directly" because oftentimes other claims are stated as justifications for this, but aren't actually doing the heavy lifting, or are even correct, and the real key assumption is left unstated.)
Or perhaps I should instead say, when it comes to particular claims, what concerns me is less often the claims that they accept than the claims they don't accept. I say "don't accept" rather than "reject" because they don't seem to explicitly reject these ideas but rather say things that contradict them while basically ignoring them and not engaging with them, despite their fundamental nature. I mean, OK -- these days you are starting to see more explicit rejection of the idea of free speech (for instance), rather than just attempting to ignore it or weasel around it. But there's a lot of just things like, people just ignoring the idea that espistemic rationality is essential to instrumental rationality, you know?
(And oftentimes there's the problem that what they're claiming may actually be a strategic equivocation. You've read "The Vacuity of Postmodernist Methodology", right? I won't bother repeating what Shackel has said better.)
> I often find that criticisms of these fields lack a nuanced understanding of the actual content and behavior of the fields, and instead come from a malformed reading of the field that is readily accepted by the reader.
So, OK. It's entirely possible that I've misidentified the problem, that it has less to do with these academic departments than I'd thought. But that's very different from there being no problem. The problem is very real and very visible. You don't need to go to the departments it's most famously associated with to see it -- you can see at Google; you can see it among Linux kernel developers; you can see it in the math department (my own field). It certainly does seem to me like it's being pushed by these departments and segments thereof it's most associated with, but, hey, I'm an outsider, maybe I'm mistaken there (although the tales I've heard of them are not encouraging). But the problem is apparent even if I am mistaken there and must be addressed all the same. Left unchecked it could seriously damage our ability to find correct answers and build new things (I mean, more than it already has).
That's not a false claim but rather a meaningless one; if people are debating that they have other problems. But, I guess, less serious ones; everyone makes meaningless statements occasionally, after all. One should avoid debating meaningless propositions, yes, but that's a mistake you can learn to avoid given good norms of discourse; getting those good norms of discourse is the more fundamental problem. (And of course it's not always obvious in advance what statements are meaningless; that's something that requires discussion to determine.)
Also, please don't put words in my mouth. Which you initially avoided doing, presenting your example purely as an example; but then you attempted to rebut this example as if this were an argument against my claims, which, obviously, makes no sense.
> I can make a guess at what thinkers you may be following
Would you care to make that explicit, since it sounds like you can?
In any case I'd be a little cautious of identifying me too closely with anyone established. If you want to know what I think more generally... well, you can find my writing scattered throughout the comments sections on such places as Slate Star Codex, Shtetl-Optimized, and Thing of Things. :P I've been meaning to write some of this up in a more visible form but, uh, I think I might want to get a new handle first...
> Which tribal effects do you think dominate?
Um, the usual ones? Do I really need to go over how tribalism works? Well, OK. Here are some characteristics, briefly:
Norms of free inquiry weaken and eventually vanish. Arguments are not evaluated by their reasoning, but by a combination of their conclusions (and their conclusions are evaluated by whether they support the tribal creed and the tribe's supremacy over its outgroups) and the social status within the tribe of the person making it. Everything becomes personal; principles are stated but ignored, in that they are applied selectively -- if a principle would be applied in a way that restrains someone in high standing in the group, or help someone in the outgroup, a reason for an exception is found. Agreeing with the tribe becomes considered as essential to personal ethics. Even just asking for clarification about a particular principle marks you as an evil outsider, because if you were a good person you would already understand. The group becomes taken over by those who only know how to politick and ceases to be about anything except its own dominance and the dominance of those who have taken it over. Group beliefs become more and more detached from reality.
I mean, I could go on, but I'm hoping that's sufficient.
This process is, of course, constantly happening everywhere, because it's human nature. But with a good negative-feedback loop, with good defenses against this, you can keep this positive-feedback loop in check, avoid the disastrous later stages, and (over the long term) keep yourself aligned with reality. Cut the negative-feedback loop and disaster is inevitable, even if its particular form is unpredictable.
("Tribalism" may be the wrong word here; I tend to use it as shorthand since these things tend to go together with tribalism proper. But the important thing is the phenomenon being described -- where the nominal purpose of a thing becomes completely corrupted by social considerations and interpersonal politicking -- not what we call it.)
We do have good defenses against this, if perhaps not on the mass scale. But certain people keep trying to toss out those defenses as unnecessary...
> Are there other specific beliefs in these fields that you're more concerned about?
I think this is a bad way of framing things. My concern here is with the meta-level, not the object-level. I'm not concerned not primarily about particular claims I see as wrong -- which, after all, may be correct (determining the truth of many of these claims does require large-scale data, something I'm not in a position to collect or trained in evaluating), but rather the norms of discourse surrounding them. Sure, a few might be correct by coincidence, but being reliably right requires good methods. Happening to get a few things right in your tribal creed is worth no credit.
To the extent I'm concerned with particular object-level claims it's primarily about how they're defended, about the reasoning given for them, not with the claims themselves. (The exception, of course, is those object-level beliefs that most directly justify this illiberalism and warped norms of discourse. I say "most directly" because oftentimes other claims are stated as justifications for this, but aren't actually doing the heavy lifting, or are even correct, and the real key assumption is left unstated.)
Or perhaps I should instead say, when it comes to particular claims, what concerns me is less often the claims that they accept than the claims they don't accept. I say "don't accept" rather than "reject" because they don't seem to explicitly reject these ideas but rather say things that contradict them while basically ignoring them and not engaging with them, despite their fundamental nature. I mean, OK -- these days you are starting to see more explicit rejection of the idea of free speech (for instance), rather than just attempting to ignore it or weasel around it. But there's a lot of just things like, people just ignoring the idea that espistemic rationality is essential to instrumental rationality, you know?
(And oftentimes there's the problem that what they're claiming may actually be a strategic equivocation. You've read "The Vacuity of Postmodernist Methodology", right? I won't bother repeating what Shackel has said better.)
> I often find that criticisms of these fields lack a nuanced understanding of the actual content and behavior of the fields, and instead come from a malformed reading of the field that is readily accepted by the reader.
So, OK. It's entirely possible that I've misidentified the problem, that it has less to do with these academic departments than I'd thought. But that's very different from there being no problem. The problem is very real and very visible. You don't need to go to the departments it's most famously associated with to see it -- you can see at Google; you can see it among Linux kernel developers; you can see it in the math department (my own field). It certainly does seem to me like it's being pushed by these departments and segments thereof it's most associated with, but, hey, I'm an outsider, maybe I'm mistaken there (although the tales I've heard of them are not encouraging). But the problem is apparent even if I am mistaken there and must be addressed all the same. Left unchecked it could seriously damage our ability to find correct answers and build new things (I mean, more than it already has).
This isn't simply a case of poor quality control. Sure, hoax articles have made it through some (largely pay-to-publish) journals in other fields from time to time. And yes those incidents should inspire reflection and correction, as well as a hit to the reputations of such journals.
But this is much more. This is about a set of ideas that pass for expert scholarship despite having lost any tethering to to the real world. Just look at this reaction to the hoax:
> Some of the pieces were indeed outlandish (and not all were accepted). But it appears that some were simply based in premises (e.g. social constructionism) or political principles (e.g trans equality) that the hoax authors find problematic but we do not.
https://twitter.com/alisonphipps/status/1047421076509261824
Note that word "problematic." I don't want to know whether social constructionism is considered "problematic", I want to know whether it is true.
There is an actual real world out there that scholars are purporting to study. But when science and empiricism are considered oppressive, all we are left with is posturing and grandstanding over ideology.
But this is much more. This is about a set of ideas that pass for expert scholarship despite having lost any tethering to to the real world. Just look at this reaction to the hoax:
> Some of the pieces were indeed outlandish (and not all were accepted). But it appears that some were simply based in premises (e.g. social constructionism) or political principles (e.g trans equality) that the hoax authors find problematic but we do not.
https://twitter.com/alisonphipps/status/1047421076509261824
Note that word "problematic." I don't want to know whether social constructionism is considered "problematic", I want to know whether it is true.
There is an actual real world out there that scholars are purporting to study. But when science and empiricism are considered oppressive, all we are left with is posturing and grandstanding over ideology.
And the ideological errors "leak" into broader culture, they warned in their confession.
"...peer-reviewed journals are the absolute gold standard of knowledge production. And these concepts leak into culture... the problem of corrupt scholarship has already leaked heavily into other fields like education, social work, media, psychology, and sociology, among others—and it openly aims to continue spreading. This makes the problem a grave concern that’s rapidly undermining the legitimacy and reputations of universities, skewing politics, drowning out needed conversations, and pushing the culture war to ever more toxic and existential polarization. Further, it is affecting activism on behalf of women and racial and sexual minorities..."
https://areomagazine.com/2018/10/02/academic-grievance-studi...
"...peer-reviewed journals are the absolute gold standard of knowledge production. And these concepts leak into culture... the problem of corrupt scholarship has already leaked heavily into other fields like education, social work, media, psychology, and sociology, among others—and it openly aims to continue spreading. This makes the problem a grave concern that’s rapidly undermining the legitimacy and reputations of universities, skewing politics, drowning out needed conversations, and pushing the culture war to ever more toxic and existential polarization. Further, it is affecting activism on behalf of women and racial and sexual minorities..."
https://areomagazine.com/2018/10/02/academic-grievance-studi...
If your criteria for publication is what is epistemologically true you are going to find it difficult to publish in any domain except possibly pure mathematics or physics.
Social Constructionism and Blank Slatism are theories that can be tested. There is plenty that can be published on these topics that is based in evidence rather than ideology.
However, if you fake (or, as in this case, completely make up) your evidence, then you'll be able to get false things published - the peer review process has no ability to verify if what you claim is true, it does not and will not re-test the theory, that might be done by other teams after your results are published but not before.
A publication is not required to be true, but it should make some sort of progress towards making a true statement.
When she says that one of the accepted papers was based on a premise of trans equality does anyone know to which paper she is referring to?
That'll be Strategies for Dealing with Cisnormative Discursive Aggression in the Workplace: Disruption, Criticism, Self-Enforcement, and Collusion ("Thesis: That trans people are all oppressed and constrained by cisnormative language in the workplace even if they don’t think they are, that trans activists who are avoided at work are proof of transphobia and trans men who are skeptical of trans activism are afraid of transphobia and/or taking advantage of male privilege"). It was rejected everywhere it was submitted.
Discussed earlier on HN: https://news.ycombinator.com/item?id=18127811
The hoaxers writeup is, I think, pretty misleading. Their work tended to get rejected from high-impact journals. They submitted multiple stories to some journals (notably, to Hypatia, which they brag about getting accepted in) and leave a clear impression that their most ridiculous work was what made it in, rather than a much more banal paper. They quote almost entirely favorable feedback --- including, in their earlier writeup, from papers that were ultimately rejected! They brag about papers that were ultimately rejected. They submit approximately the same paper to multiple journals and are rejected by all but the very most marginal. In at least one case, they cite the same reviewer multiple times, leaving the impression of a consensus.
If you don't think that's misleading, it's worth knowing that Yascha Mounk, cited warmly in this piece, was clearly taken in, dazzled on Twitter by a rejected paper from the hoax.
What have they really uncovered?
1. Peer review isn't replication and never could have been. Peer reviewers get, at best, a couple hours, unpaid; submitting researchers get many months, and are often grant-funded. The hoaxers took ten months to get a few papers accepted in marginal journals. The pop understanding of what peer review is is, simply, broken: peer review depends on good faith, and it was never realistic to believe otherwise.
2. There are lots of crappy journals. Didn't you already know that? The market responds to the demand that professional academics publish, by creating venues that ensure professional academics can reliably publish. That's why we have measures like "impact factors". This is not at all a phenomenon isolated to the social sciences.
If the authors even acknowledged these conclusions, they'd take all the excitement out of their hoax. We'd all nod our heads and say, "of course". Instead, they pretend to have taken down something they've dubbed "grievance studies", while pretending that they couldn't have done the same thing to, say, a marginal economics journal.
The hoaxers writeup is, I think, pretty misleading. Their work tended to get rejected from high-impact journals. They submitted multiple stories to some journals (notably, to Hypatia, which they brag about getting accepted in) and leave a clear impression that their most ridiculous work was what made it in, rather than a much more banal paper. They quote almost entirely favorable feedback --- including, in their earlier writeup, from papers that were ultimately rejected! They brag about papers that were ultimately rejected. They submit approximately the same paper to multiple journals and are rejected by all but the very most marginal. In at least one case, they cite the same reviewer multiple times, leaving the impression of a consensus.
If you don't think that's misleading, it's worth knowing that Yascha Mounk, cited warmly in this piece, was clearly taken in, dazzled on Twitter by a rejected paper from the hoax.
What have they really uncovered?
1. Peer review isn't replication and never could have been. Peer reviewers get, at best, a couple hours, unpaid; submitting researchers get many months, and are often grant-funded. The hoaxers took ten months to get a few papers accepted in marginal journals. The pop understanding of what peer review is is, simply, broken: peer review depends on good faith, and it was never realistic to believe otherwise.
2. There are lots of crappy journals. Didn't you already know that? The market responds to the demand that professional academics publish, by creating venues that ensure professional academics can reliably publish. That's why we have measures like "impact factors". This is not at all a phenomenon isolated to the social sciences.
If the authors even acknowledged these conclusions, they'd take all the excitement out of their hoax. We'd all nod our heads and say, "of course". Instead, they pretend to have taken down something they've dubbed "grievance studies", while pretending that they couldn't have done the same thing to, say, a marginal economics journal.
It's hard to tell specifically how their methodology went, but this bit of the author's write-up jumps out:
> As for our performance, 80% of our papers overall went to full peer review, which keeps with the standard 10-20% of papers that are “desk rejected” without review at major journals across the field. We improved this ratio from 0% at first to 94.4% after a few months of experimenting with much more hoaxish papers.
This makes it sound like the authors wrote a bunch of absurd articles, all of which got desk rejected, and then they iterated until they got papers accepted somewhere. With that methodology, the conclusion ("we got papers accepted somewhere!") doesn't seem very surprising.
Edited to add: if the above is correct and their early submissions were all rejected, one could reasonably say of the results: "Editorial standards at all journals surveyed were more rigorous than the authors initially expected".
> As for our performance, 80% of our papers overall went to full peer review, which keeps with the standard 10-20% of papers that are “desk rejected” without review at major journals across the field. We improved this ratio from 0% at first to 94.4% after a few months of experimenting with much more hoaxish papers.
This makes it sound like the authors wrote a bunch of absurd articles, all of which got desk rejected, and then they iterated until they got papers accepted somewhere. With that methodology, the conclusion ("we got papers accepted somewhere!") doesn't seem very surprising.
Edited to add: if the above is correct and their early submissions were all rejected, one could reasonably say of the results: "Editorial standards at all journals surveyed were more rigorous than the authors initially expected".
The fact that they list 94.4% also gives up a hint. They iterated a lot. To get numbers that precise you need to submit a tremendous number of articles.
Give somebody a year and piles of submissions and of course they will sneak some things through.
Give somebody a year and piles of submissions and of course they will sneak some things through.
> The hoaxers writeup is, I think, pretty misleading. Their work tended to get rejected from high-impact journals.
What's your evidence for that? A number of their papers got accepted into decently high impact journals, including Hypatia.
> They submitted multiple stories to some journals (notably, to Hypatia, which they brag about getting accepted in) and leave a clear impression that their most ridiculous work was what made it in, rather than a much more banal paper
The thesis of the paper that got accepted was that "criticizing feminist theory is unethical". That doesn't seem particularly banal to me.
> They quote almost entirely favorable feedback --- including, in their earlier writeup, from papers that were ultimately rejected!
Who cares if the feedback was on papers that were ultimately rejected? Does that change the fact that they received favorable feedback on an absurd hypothesis?
> They submit approximately the same paper to multiple journals and are rejected by all but the very most marginal.
This is simply not true. They got accepted by high impact journals, including Hypatia. You seem to think that the papers that got accepted were 'banal', but that doesn't change the fact that they were accepted.
> In at least one case, they cite the same reviewer multiple times, leaving the impression of a consensus.
Are you saying that they do not cite a diverse sample of reviewers? Because that is empirically false. What you seem to be doing is latching a single instance where they cited the same reviewer twice, and trying to use that to insinuate that they didn't cite feedback from lots of distinct individuals, which is patently false.
> 1. Peer review isn't replication and never could have been. Peer reviewers get, at best, a couple hours, unpaid; submitting researchers get many months, and are often grant-funded. The hoaxers took ten months to get a few papers accepted in marginal journals. The pop understanding of what peer review is is, simply, broken: peer review depends on good faith, and it was never realistic to believe otherwise.
Is that so? So, you'd have no problem getting a nonsense paper accepted into a math, computer science, physics or even economics journal? I would be extremely surprised if you could pass of work of such low quality in journals of comparable impact factor in any of these disciplines.
What's your evidence for that? A number of their papers got accepted into decently high impact journals, including Hypatia.
> They submitted multiple stories to some journals (notably, to Hypatia, which they brag about getting accepted in) and leave a clear impression that their most ridiculous work was what made it in, rather than a much more banal paper
The thesis of the paper that got accepted was that "criticizing feminist theory is unethical". That doesn't seem particularly banal to me.
> They quote almost entirely favorable feedback --- including, in their earlier writeup, from papers that were ultimately rejected!
Who cares if the feedback was on papers that were ultimately rejected? Does that change the fact that they received favorable feedback on an absurd hypothesis?
> They submit approximately the same paper to multiple journals and are rejected by all but the very most marginal.
This is simply not true. They got accepted by high impact journals, including Hypatia. You seem to think that the papers that got accepted were 'banal', but that doesn't change the fact that they were accepted.
> In at least one case, they cite the same reviewer multiple times, leaving the impression of a consensus.
Are you saying that they do not cite a diverse sample of reviewers? Because that is empirically false. What you seem to be doing is latching a single instance where they cited the same reviewer twice, and trying to use that to insinuate that they didn't cite feedback from lots of distinct individuals, which is patently false.
> 1. Peer review isn't replication and never could have been. Peer reviewers get, at best, a couple hours, unpaid; submitting researchers get many months, and are often grant-funded. The hoaxers took ten months to get a few papers accepted in marginal journals. The pop understanding of what peer review is is, simply, broken: peer review depends on good faith, and it was never realistic to believe otherwise.
Is that so? So, you'd have no problem getting a nonsense paper accepted into a math, computer science, physics or even economics journal? I would be extremely surprised if you could pass of work of such low quality in journals of comparable impact factor in any of these disciplines.
I read over a couple of the papers. The descriptions of them by the authors did not match the content of the papers.
For instance, the one they call "Hooters" is described as "A gender scholar goes to Hooters to try to figure out why it exists." The actual paper is written as a record of transcripts of visits to a restaurant and an extraction of particular conversation themes. At no point did I see the paper questioning why Hooters exists -- the (apparently totally faked) data seems pretty stereotypically motivated, which I think may be their point, but I'm not sure short of a fraud investigation how reviewers are supposed to know that the particular group this author claims to have visited Hooters with didn't say the things supposedly directly recorded. That's a really dramatic claim for a reviewer to make: "This conversation which the author claims is a direct transcription from a recording couldn't possibly have happened because it seems too stereotypical."
Yes, the methodology isn't that great, and the paper was rejected.
More generally, the "study" isn't selecting random journals to see if they could defraud -- it is instead aimed at specific academic targets and a "test to failure" scattershot mechanism is used. Thus we have no idea whether these journals are any more discriminating than, say, PLOS or Nature.
Ultimately, I had some sympathy for the Sokal experiment in that it seemed to say something about the interaction of literary theory and physics. I don't get the same sense that there's much here other than "confirmation bias exists, even among gender studies folks!!!" which seems like it wouldn't take 3 people 10 months to figure out, and could be done in a much more direct way, and honestly isn't that shocking or rattling a conclusion.
For instance, the one they call "Hooters" is described as "A gender scholar goes to Hooters to try to figure out why it exists." The actual paper is written as a record of transcripts of visits to a restaurant and an extraction of particular conversation themes. At no point did I see the paper questioning why Hooters exists -- the (apparently totally faked) data seems pretty stereotypically motivated, which I think may be their point, but I'm not sure short of a fraud investigation how reviewers are supposed to know that the particular group this author claims to have visited Hooters with didn't say the things supposedly directly recorded. That's a really dramatic claim for a reviewer to make: "This conversation which the author claims is a direct transcription from a recording couldn't possibly have happened because it seems too stereotypical."
Yes, the methodology isn't that great, and the paper was rejected.
More generally, the "study" isn't selecting random journals to see if they could defraud -- it is instead aimed at specific academic targets and a "test to failure" scattershot mechanism is used. Thus we have no idea whether these journals are any more discriminating than, say, PLOS or Nature.
Ultimately, I had some sympathy for the Sokal experiment in that it seemed to say something about the interaction of literary theory and physics. I don't get the same sense that there's much here other than "confirmation bias exists, even among gender studies folks!!!" which seems like it wouldn't take 3 people 10 months to figure out, and could be done in a much more direct way, and honestly isn't that shocking or rattling a conclusion.
> I read over a couple of the papers. The descriptions of them by the authors did not match the content of the papers.
So did I. The dog park paper more or less matches its description. The fat bodybuilding paper is exactly as described. The dildos paper is as described. Do you have any descriptions aside from the Hooters one that you take issue with?
> Yes, the methodology isn't that great, and the paper was rejected.
Sure, but lots of them were not. I'm not sure why you would focus on a paper that was rejected when they had 7 that were accepted.
> More generally, the "study" isn't selecting random journals to see if they could defraud -- it is instead aimed at specific academic targets and a "test to failure" scattershot mechanism is used. Thus we have no idea whether these journals are any more discriminating than, say, PLOS or Nature.
True, though I have a hard time believing you could get the same result in any STEM discipline, though i'm certainly open to data to the contrary.
> Ultimately, I had some sympathy for the Sokal experiment in that it seemed to say something about the interaction of literary theory and physics. I don't get the same sense that there's much here other than "confirmation bias exists, even among gender studies folks!!!" which seems like it wouldn't take 3 people 10 months to figure out, and could be done in a much more direct way, and honestly isn't that shocking or rattling a conclusion.
I think the point of all this is that the epistemology of the field is intellectually bankrupt. It's not just confirmation bias. It's that within the epistemology of gender studies, they aren't even wrong.
So did I. The dog park paper more or less matches its description. The fat bodybuilding paper is exactly as described. The dildos paper is as described. Do you have any descriptions aside from the Hooters one that you take issue with?
> Yes, the methodology isn't that great, and the paper was rejected.
Sure, but lots of them were not. I'm not sure why you would focus on a paper that was rejected when they had 7 that were accepted.
> More generally, the "study" isn't selecting random journals to see if they could defraud -- it is instead aimed at specific academic targets and a "test to failure" scattershot mechanism is used. Thus we have no idea whether these journals are any more discriminating than, say, PLOS or Nature.
True, though I have a hard time believing you could get the same result in any STEM discipline, though i'm certainly open to data to the contrary.
> Ultimately, I had some sympathy for the Sokal experiment in that it seemed to say something about the interaction of literary theory and physics. I don't get the same sense that there's much here other than "confirmation bias exists, even among gender studies folks!!!" which seems like it wouldn't take 3 people 10 months to figure out, and could be done in a much more direct way, and honestly isn't that shocking or rattling a conclusion.
I think the point of all this is that the epistemology of the field is intellectually bankrupt. It's not just confirmation bias. It's that within the epistemology of gender studies, they aren't even wrong.
Could you reference some texts from within gender studies that you feel are both representative of the field, and which outline the negative aspects of the epistemology of gender studies that you are perceiving?
> You seem to think that the papers that got accepted were 'banal', but that doesn't change the fact that they were accepted.
The banality of the papers is the central weakness of the whole thing. The whole point of Sokal- or SCIgen-style hoaxes is that the papers were gibberish - so when a journal publishes one, the author can legitimately say "Hey look, this journal published gibberish!".
The authors here can't say anything like that. They wrote real papers, arguing real conclusions they thought the journals might agree with. Then those papers got rejected, so they toned it down and wrote more, and then they got peer-review feedback so they changed things in response, etc. The most one can fairly say about the results is "Hey look, these journals published papers that we think are really bad". Which is fair enough, but surely they could say the same thing about most of what those journals already publish, right?
The liberty of the whole thing is writing it all up as if a Sokal-style "we got them to print gibberish" narrative had occurred, when that seems clearly not to be the case.
The banality of the papers is the central weakness of the whole thing. The whole point of Sokal- or SCIgen-style hoaxes is that the papers were gibberish - so when a journal publishes one, the author can legitimately say "Hey look, this journal published gibberish!".
The authors here can't say anything like that. They wrote real papers, arguing real conclusions they thought the journals might agree with. Then those papers got rejected, so they toned it down and wrote more, and then they got peer-review feedback so they changed things in response, etc. The most one can fairly say about the results is "Hey look, these journals published papers that we think are really bad". Which is fair enough, but surely they could say the same thing about most of what those journals already publish, right?
The liberty of the whole thing is writing it all up as if a Sokal-style "we got them to print gibberish" narrative had occurred, when that seems clearly not to be the case.
> The banality of the papers is the central weakness of the whole thing. The whole point of Sokal- or SCIgen-style hoaxes is that the papers were gibberish - so when a journal publishes one, the author can legitimately say "Hey look, this journal published gibberish!".
My point is that most of them were not banal. In that line, I was responding to an isolated point of tptaceks. If all the papers were banal, i'd agree. But they were not.
My point is that most of them were not banal. In that line, I was responding to an isolated point of tptaceks. If all the papers were banal, i'd agree. But they were not.
Most of the papers were rejected! The ones that were accepted did tend to be banal!
Sure, most got rejected. But 7 got accepted. And while banality is obviously a subjective value judgment, I don't think it applies to all of them. Just to enumerate the hypotheses that were accepted:
> That dog parks are rape-condoning spaces and a place of rampant canine rape culture and systemic oppression against “the oppressed dog” through which human attitudes to both problems can be measured. This provides insight into training men out of the sexual violence and bigotry to which they are prone.
> That it is only oppressive cultural norms which make society regard the building of muscle rather than fat admirable and that bodybuilding and activism on behalf of the fat could be benefited by including fat bodies displayed in non-competitive ways.
> That it is suspicious that men rarely anally self-penetrate using sex toys, and that this is probably due to fear of being thought homosexual (“homohysteria”) and bigotry against trans people (transphobia). (It combines these ideas into a novel concept “transhysteria,” which was suggested by one of the paper’s peer reviewers.) Encouraging them to engage in receptive penetrative anal eroticism will decrease transphobia and increase feminist values.
> That men frequent “breasturants” like Hooters because they are nostalgic for patriarchal dominance and enjoy being able to order attractive women around. The environment that breastaurants provide for facilitating this encourages men to identify sexual objectification and sexual conquest, along with masculine toughness and male dominance, with “authentic masculinity.” The data are clearly nonsense and conclusions drawn from it are unwarranted by it. (NB. One reviewer did raise concerns about the rigor of the data)
> That academic hoaxes or other forms of satirical or ironic critique of social justice scholarship are unethical, characterized by ignorance and rooted in a desire to preserve privilege.
> No clear thesis. A rambling poetic monologue of a bitter, divorced feminist, much of which was produced by a teenage angst poetry generator before being edited into something slightly more “realistic” which is then interspersed with self-indulgent autoethnographical reflections on female sexuality and spirituality written entirely in slightly under six hours.
> That feminism which foregrounds individual choice and responsibility and female agency and strength can be countered by a feminism which unifies in solidarity around the victimhood of the most marginalized women in society.
I will grant you that 'breastaurants' and the Mein Kampf one are fairly banal. The Mein Kampf paper is really only shocking if you happen to know that it is an excerpt from Mein Kampf. But the rest seem pretty non-banal to me, at least with regard to being published in an academic journal.
> That dog parks are rape-condoning spaces and a place of rampant canine rape culture and systemic oppression against “the oppressed dog” through which human attitudes to both problems can be measured. This provides insight into training men out of the sexual violence and bigotry to which they are prone.
> That it is only oppressive cultural norms which make society regard the building of muscle rather than fat admirable and that bodybuilding and activism on behalf of the fat could be benefited by including fat bodies displayed in non-competitive ways.
> That it is suspicious that men rarely anally self-penetrate using sex toys, and that this is probably due to fear of being thought homosexual (“homohysteria”) and bigotry against trans people (transphobia). (It combines these ideas into a novel concept “transhysteria,” which was suggested by one of the paper’s peer reviewers.) Encouraging them to engage in receptive penetrative anal eroticism will decrease transphobia and increase feminist values.
> That men frequent “breasturants” like Hooters because they are nostalgic for patriarchal dominance and enjoy being able to order attractive women around. The environment that breastaurants provide for facilitating this encourages men to identify sexual objectification and sexual conquest, along with masculine toughness and male dominance, with “authentic masculinity.” The data are clearly nonsense and conclusions drawn from it are unwarranted by it. (NB. One reviewer did raise concerns about the rigor of the data)
> That academic hoaxes or other forms of satirical or ironic critique of social justice scholarship are unethical, characterized by ignorance and rooted in a desire to preserve privilege.
> No clear thesis. A rambling poetic monologue of a bitter, divorced feminist, much of which was produced by a teenage angst poetry generator before being edited into something slightly more “realistic” which is then interspersed with self-indulgent autoethnographical reflections on female sexuality and spirituality written entirely in slightly under six hours.
> That feminism which foregrounds individual choice and responsibility and female agency and strength can be countered by a feminism which unifies in solidarity around the victimhood of the most marginalized women in society.
I will grant you that 'breastaurants' and the Mein Kampf one are fairly banal. The Mein Kampf paper is really only shocking if you happen to know that it is an excerpt from Mein Kampf. But the rest seem pretty non-banal to me, at least with regard to being published in an academic journal.
The relevant measure of banality here is in comparison to what the journals normally publish. What makes e.g. Sokal's hoax important is that he exposed a journal for publishing something the editors didn't even understand. Here, the journals weren't duped into anything - they read and understood the papers, and apparently considered them worth publishing. That's what's so banal about the whole thing.
> The relevant measure of banality here is in comparison to what the journals normally publish.
I completely disagree. The point of this story is exactly that this stuff is line with what the journals normally publish. Their entire point is that the stuff they wrote is indistinguishable from these journal's normal fare.
I completely disagree. The point of this story is exactly that this stuff is line with what the journals normally publish. Their entire point is that the stuff they wrote is indistinguishable from these journal's normal fare.
> Their entire point is that the stuff they wrote is indistinguishable from these journal's normal fare.
Journals publishing papers indistinguishable from their normal fare is the null hypothesis! If accomplishing that was the authors' goal they could have taken ten months off, surely ;)
Journals publishing papers indistinguishable from their normal fare is the null hypothesis! If accomplishing that was the authors' goal they could have taken ten months off, surely ;)
Do you not understand what i'm saying? They're criticizing the content of these journals.
In their capacity as authors of a blog post, sure - but in their capacity as conductors of a social science experiment they're also claiming to have verified a hypothesis (that certain journals would publish absurd papers), and that's the claim I think is weak.
Please understand I'm not defending the content of these journals - I'm saying that this ten month hoax doesn't appear to have established anything.
Please understand I'm not defending the content of these journals - I'm saying that this ten month hoax doesn't appear to have established anything.
I think I would state their hypothesis as: that the content of these journals is absurd, and by using slightly more absurd hypotheses than their normal fare, we can clearly demonstrate to other people that their content is absurd when those more obviously ridiculous ideas get accepted.
> he exposed a journal for publishing something the editors didn't even understand
This is your assumption. In fact they understood perfectly what the paper meant and that's why they published it. But because the paper was effectively constructed as something without meaning, Sokal managed to prove that their perception of meaning is a delusion.
Same thing has been done here: the editors of the journals found perfectly reasonable something that was ridiculous and absurd by construction, thus proving that the sense of reasonabless and soundness is lost to them.
This is your assumption. In fact they understood perfectly what the paper meant and that's why they published it. But because the paper was effectively constructed as something without meaning, Sokal managed to prove that their perception of meaning is a delusion.
Same thing has been done here: the editors of the journals found perfectly reasonable something that was ridiculous and absurd by construction, thus proving that the sense of reasonabless and soundness is lost to them.
> editors of the journals found perfectly reasonable something that was ridiculous and absurd by construction
I understand your point, but my argument here is that the hoax papers cannot reasonably be called "absurd by construction". The authors started out with that goal, but their initial papers apparently all got rejected so they iteratively toned down the absurdity until the papers got published. That's effectively "publishable by construction", isn't it?
I understand your point, but my argument here is that the hoax papers cannot reasonably be called "absurd by construction". The authors started out with that goal, but their initial papers apparently all got rejected so they iteratively toned down the absurdity until the papers got published. That's effectively "publishable by construction", isn't it?
They still sound pretty absurd to me.
If the same thing has been done here, what's the point? Sokal already made that point long ago. There is no academic goal in repeating that experiment.
Moreover, repeating it 20-fold? That's not even ethical - either one success proves your point, or were going to need a control group and rigorous methodology.
Since the trio reject the notion of control groups, they should have understood that getting more than one paper accepted was not furthering their experiment, only their ego.
Moreover, repeating it 20-fold? That's not even ethical - either one success proves your point, or were going to need a control group and rigorous methodology.
Since the trio reject the notion of control groups, they should have understood that getting more than one paper accepted was not furthering their experiment, only their ego.
Strange argument. As far as I know, they submitted the papers to different journals. Sokal's hoax targeted again a different journal in a different field twenty years ago. Of course the results are specific to the journal and the discipline- but even in the case of multiple papers accepted by the same journal, it would just reinforce the result.
I think you've misread me.
My own comment leads off by acknowledging they got a paper accepted into Hypatia --- my gripe is that it's not the lurid farce they suggest they got accepted (a paper demanding that white male students sit on the floor in chains in class) but rather a much more boring paper about the legitimacy of hoaxes like theirs.
When I said "They submit approximately the same paper to multiple journals and are rejected by all but the very most marginal", I was referring to a specific paper (in fact, the Mein Kampf paper), not to all the papers. My point was: they tried that paper in several places, and the (academically) reputable venues rejected them.
Yes, I think --- I'm actually pretty confident --- that I could get a false paper accepted in a CS venue. Economists have already remarked on Twitter that you don't have to get "nonsense" papers accepted yourself, because the literature is already full of what turns out to be nonsense (do keep in mind that in large part what these hoaxers submitted was not nonsense, but rather fabricated research including field work).
My own comment leads off by acknowledging they got a paper accepted into Hypatia --- my gripe is that it's not the lurid farce they suggest they got accepted (a paper demanding that white male students sit on the floor in chains in class) but rather a much more boring paper about the legitimacy of hoaxes like theirs.
When I said "They submit approximately the same paper to multiple journals and are rejected by all but the very most marginal", I was referring to a specific paper (in fact, the Mein Kampf paper), not to all the papers. My point was: they tried that paper in several places, and the (academically) reputable venues rejected them.
Yes, I think --- I'm actually pretty confident --- that I could get a false paper accepted in a CS venue. Economists have already remarked on Twitter that you don't have to get "nonsense" papers accepted yourself, because the literature is already full of what turns out to be nonsense (do keep in mind that in large part what these hoaxers submitted was not nonsense, but rather fabricated research including field work).
> My own comment leads off by acknowledging they got a paper accepted into Hypatia --- my gripe is that it's not the lurid farce they suggest they got accepted (a paper demanding that white male students sit on the floor in chains in class) but rather a much more boring paper about the legitimacy of hoaxes like theirs.
IMO the paper they did get accepted is still quite disturbing.
> When I said "They submit approximately the same paper to multiple journals and are rejected by all but the very most marginal", I was referring to a specific paper (in fact, the Mein Kampf paper), not to all the papers. My point was: they tried that paper in several places, and the (academically) reputable venues rejected them.
Ok, but what about the others?
> Yes, I think --- I'm actually pretty confident --- that I could get a false paper accepted in a CS venue. Economists have already remarked on Twitter that you don't have to get "nonsense" papers accepted yourself, because the literature is already full of what turns out to be nonsense (do keep in mind that in large part what these hoaxers submitted was not nonsense, but rather fabricated research including field work).
False, sure. False papers get accepted all the time. That's part of the scientific process. These papers were not merely false, they were absurd. I seriously doubt you could get a truly absurd CS paper published anywhere remotely reputable.
IMO the paper they did get accepted is still quite disturbing.
> When I said "They submit approximately the same paper to multiple journals and are rejected by all but the very most marginal", I was referring to a specific paper (in fact, the Mein Kampf paper), not to all the papers. My point was: they tried that paper in several places, and the (academically) reputable venues rejected them.
Ok, but what about the others?
> Yes, I think --- I'm actually pretty confident --- that I could get a false paper accepted in a CS venue. Economists have already remarked on Twitter that you don't have to get "nonsense" papers accepted yourself, because the literature is already full of what turns out to be nonsense (do keep in mind that in large part what these hoaxers submitted was not nonsense, but rather fabricated research including field work).
False, sure. False papers get accepted all the time. That's part of the scientific process. These papers were not merely false, they were absurd. I seriously doubt you could get a truly absurd CS paper published anywhere remotely reputable.
>> I would be extremely surprised if you could pass of work of such low quality in journals of comparable impact factor in [math, computer science, physics and economics].
What about medicine, genetics and neuroscience? These are the fields investigated by John Ioannidis, whom you've probably heard of [1]. He's the guy that published that paper titled "Why most published research findings are false" [2]. From the abstract of which, I quote:
Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.
Note that that's pretty much the claim of the hoax- that the hoaxed fields are dominated by bias and the majority of their research claims are absurd (so, false, I guess).
If that's true of medicine and genetics, it's true of most of the hard sciences, and no doubt, also CS and economics. So there is no good reason to single out gender studies or feminist philosophy for this kind of fault, or in any case, the hoaxers would have made a much more meaningful contribution if they had hoaxed a maths or physics journal, rather than the ones they did.
Btw- in maths, a recent HN article discussed the Mochizuki proof of the ABC conjecture, that his fellow maths experts consider to be very badly mistaken (if I got that right) [3].
_______________________
[1] https://en.wikipedia.org/wiki/John_Ioannidis#Research_findin...
[2] https://journals.plos.org/plosmedicine/article?id=10.1371/jo...
[3] https://www.quantamagazine.org/titans-of-mathematics-clash-o...
What about medicine, genetics and neuroscience? These are the fields investigated by John Ioannidis, whom you've probably heard of [1]. He's the guy that published that paper titled "Why most published research findings are false" [2]. From the abstract of which, I quote:
Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.
Note that that's pretty much the claim of the hoax- that the hoaxed fields are dominated by bias and the majority of their research claims are absurd (so, false, I guess).
If that's true of medicine and genetics, it's true of most of the hard sciences, and no doubt, also CS and economics. So there is no good reason to single out gender studies or feminist philosophy for this kind of fault, or in any case, the hoaxers would have made a much more meaningful contribution if they had hoaxed a maths or physics journal, rather than the ones they did.
Btw- in maths, a recent HN article discussed the Mochizuki proof of the ABC conjecture, that his fellow maths experts consider to be very badly mistaken (if I got that right) [3].
_______________________
[1] https://en.wikipedia.org/wiki/John_Ioannidis#Research_findin...
[2] https://journals.plos.org/plosmedicine/article?id=10.1371/jo...
[3] https://www.quantamagazine.org/titans-of-mathematics-clash-o...
I am familiar with all of the situations you've cited, and they are nothing like this one. You're really misunderstanding Ionnandis' work if you think it is relevant to this. He's making arguments about bad statistics, which is super important and valuable, but this has almost nothing to do with that.
Again, the Mochizuiki situation is quite different. You're really not understanding these things if you think they're comparable. Mochizuiki's work is legitimate and real and serious. Whether or not it is correct is difficult to ascertain, because of its genuine complexity.
The accusation being made here is that these fields are incapable of separating absolute nonsense from genuine content. Which means that their work is indistinguishable from nonsense and absurdity. The work that Ioannidis did demonstrates that people acting within a good framework are producing work with a higher error rate than you might expect given the claims they make. That work is extremely important. But it doesn't invalidate the field of economics, or medicine. This work attempts to invalidate the entire edifice that these disciplines are built upon. It is extremely different and not comparable at all.
Again, the Mochizuiki situation is quite different. You're really not understanding these things if you think they're comparable. Mochizuiki's work is legitimate and real and serious. Whether or not it is correct is difficult to ascertain, because of its genuine complexity.
The accusation being made here is that these fields are incapable of separating absolute nonsense from genuine content. Which means that their work is indistinguishable from nonsense and absurdity. The work that Ioannidis did demonstrates that people acting within a good framework are producing work with a higher error rate than you might expect given the claims they make. That work is extremely important. But it doesn't invalidate the field of economics, or medicine. This work attempts to invalidate the entire edifice that these disciplines are built upon. It is extremely different and not comparable at all.
Ioannide's work claims that false research claims routinely makes it to
publication, even in high-profile journals. Mochizuki has published a proof
that many in the field believe is nonsense (unintelligible nonsense, at that).
That the shoddy research investigated by Ioannidis is shoddy because it misuses statistics is besides the point. What matters is that the best efforts of peer- review in highly respected venues, in "hard" science fields fails to stop such work from being published.
Which makes the hoaxed fields (gender studies etc) not particularly special in that respect.
>> The accusation being made here is that these fields are incapable of separating absolute nonsense from genuine content.
That's also what Ioannides research concludes, except it does so for medicine, neuroscience, etc.
In Mochizuki's case, his proof was published even though experts in the field could not understand it and were not sure of its validity. It may yet prove to be nonsense, or, indeed- a hoax.
>> You're really misunderstanding Ionnandis' work if you think it is relevant to this.
Obviously, I believe it's you who doesn't understand what you're talking about. But perhaps we'll keep the discourse more civil by avoiding explicitly stated assumptions about each other's cognitive abilities?
That the shoddy research investigated by Ioannidis is shoddy because it misuses statistics is besides the point. What matters is that the best efforts of peer- review in highly respected venues, in "hard" science fields fails to stop such work from being published.
Which makes the hoaxed fields (gender studies etc) not particularly special in that respect.
>> The accusation being made here is that these fields are incapable of separating absolute nonsense from genuine content.
That's also what Ioannides research concludes, except it does so for medicine, neuroscience, etc.
In Mochizuki's case, his proof was published even though experts in the field could not understand it and were not sure of its validity. It may yet prove to be nonsense, or, indeed- a hoax.
>> You're really misunderstanding Ionnandis' work if you think it is relevant to this.
Obviously, I believe it's you who doesn't understand what you're talking about. But perhaps we'll keep the discourse more civil by avoiding explicitly stated assumptions about each other's cognitive abilities?
These disciplines aren't built on the edifice that you imagine they are. What you're doing, in examining the edification of certain disciplines, is quite postmodern. I encourage you to extend that examination to all fields.
Which disciplines aren't built on the edifice I imagine they are? If you're suggesting that math and physics aren't built on what I imagine them to be, then demonstrate your point.
All of them. Every field is fallible, phrenology was once just science. Why would you think something like that couldn't happen again?
You seem to be arguing that all epistemologies are fallible and therefore they are all equivalent. This is a pretty silly line of argument. The fact that a thing is fallible does not mean that it is the equal of things that don't even bother with the truth.
So, you feel like when someone says all fields are fallible and that you should examine their edification critically, that they are simultaneously making the claim that all fields are equivalent?
Do you feel you are fallible in your identification of which fields are concerned with the truth and which ones aren't?
Do you feel you are fallible in your identification of which fields are concerned with the truth and which ones aren't?
I'm saying that you have made no argument whatsoever that math and physics are equivalently fallible to gender studies, you've merely insinuated it. If you've got an argument to make, feel free to make it. But citing the replication crisis does not cut it.
Things are either fallible or they aren't, and since it seems you're willing to accept that all fields are fallible "to some degree" aka are fallible, we shouldn't edify any of them uncritically. Are you saying that you can claim things can be fallible without being equivalent? Because that's what I'm saying, and I'm glad we agree.
The replication crisis is a perfect example of the problem faced by all fields, of papers of questionable veracity being accepted by peer-reviewed journals. Convenient for you to say it is not up for discussion. However, I am casting a critical eye on your seemingly infallible nature, and I choose to consider it relevant.
The replication crisis is a perfect example of the problem faced by all fields, of papers of questionable veracity being accepted by peer-reviewed journals. Convenient for you to say it is not up for discussion. However, I am casting a critical eye on your seemingly infallible nature, and I choose to consider it relevant.
> The replication crisis is a perfect example of the problem faced by all fields, of papers of questionable veracity being accepted by peer-reviewed journals.
You're absolutely right. Now, feel free to explain how that is relevant to THIS issue.
You're absolutely right. Now, feel free to explain how that is relevant to THIS issue.
> Is that so? So, you'd have no problem getting a nonsense paper accepted into a math, computer science, physics or even economics journal? I would be extremely surprised if you could pass of work of such low quality in journals of comparable impact factor in any of these disciplines.
https://en.wikipedia.org/wiki/Replication_crisis
> A 2016 study in the journal Science found that two-thirds of 18 experimental studies from two top-tier economics journals (American Economic Review and the Quarterly Journal of Economics) successfully replicated. A 2017 study in the Economic Journal suggested that "the majority of the average effects in the empirical economics literature are exaggerated by a factor of at least 2 and at least one-third are exaggerated by a factor of 4 or more".
https://en.wikipedia.org/wiki/Replication_crisis
> A 2016 study in the journal Science found that two-thirds of 18 experimental studies from two top-tier economics journals (American Economic Review and the Quarterly Journal of Economics) successfully replicated. A 2017 study in the Economic Journal suggested that "the majority of the average effects in the empirical economics literature are exaggerated by a factor of at least 2 and at least one-third are exaggerated by a factor of 4 or more".
The replication crisis is an entirely different thing. I'm not sure what point you're trying to make.
It's entirely possible to get a nonsense paper into an economics journal, even when not intentionally trying to expose the discipline as "nonsense". I'm not sure what you're not understanding here.
Replication crisis != nonsense papers, i'm not sure what you're not understanding here.
I guess I'm not understanding how you can think that a paper that makes a claim about the real world, but doesn't apply to the real world, could be characterised as anything but nonsensical.
It makes an empirical claim using empirical data. Empirical data is complicated and inherently probabilistic. The fact that empirical studies sometimes conclude things that are wrong does not invalidate the notion of empirical investigation. It is an expected part of the distribution. It is also an issue that sometimes research practitioners do not follow the best practices, and this can lead to an exaggeration of false positive results beyond what you would expect from the underlying statistics. However, this also does not invalidate the idea of empirical investigation. It simply means we need to iterate on our training techniques and peer review.
What this paper asserts is not that gender studies need to iterate on their techniques. It does not assert that the basic methodology of gender studies is good, but it needs some tweaks here and there. It asserts that, at a fundamental level, the concept of the field is flawed. They assert that the entire methodological framework is flawed. No one, not the authors of the replication crisis paper, or Ioannidis, nor any other serious academic asserts this about math, physics, etc...
What this paper asserts is not that gender studies need to iterate on their techniques. It does not assert that the basic methodology of gender studies is good, but it needs some tweaks here and there. It asserts that, at a fundamental level, the concept of the field is flawed. They assert that the entire methodological framework is flawed. No one, not the authors of the replication crisis paper, or Ioannidis, nor any other serious academic asserts this about math, physics, etc...
Empirical data in individual studies are supposed to have statistical controls that mitigate incorrect conclusions, that's why p values are even a thing. To claim that on the one hand individual papers can be wrong, but the additive nature of a field somehow removes the possibility of error in the field, is misguided, to say it gently.
Could it be perhaps that this individual paper, which you claim undermines the "entire methodological framework" of these fields, could be one of those individual studies that comes to incorrect conclusions, not because of any statistical flaw, but because of the inbuilt fallibility and political biases of the authors?
Could it be perhaps that this individual paper, which you claim undermines the "entire methodological framework" of these fields, could be one of those individual studies that comes to incorrect conclusions, not because of any statistical flaw, but because of the inbuilt fallibility and political biases of the authors?
Just pay attention to what i'm saying. I'm saying that the METHODOLOGICAL FRAMEWORK of physics and math is legitimate. The replication crisis et al does NOTHING to contradict that. This paper is ASSERTING that the METHODOLOGICAL FRAMEWORK of gender studies is flawed. You may disagree with that assertion. However, citing the replication crisis, which is a DIFFERENT CATEGORY OF ISSUE, does nothing to make your point.
Pay attention to the article! The authors of these hoax papers don't demonstrate or address a methodological flaw in gender studies. They demonstrate that you can get papers written in bad faith into a peer reviewed journal, which is also trivially true for many disciplines, including economics, which you have previously implied people would have a problem getting a "nonsense" paper into.
An empirical paper with a reasonable hypothesis that happens to be wrong is not a nonsense paper. The papers they got published in these journals were not simply empirically false. They were categorically absurd and in some cases wildly unethical. They weren't simply making empirical claims that were reasonable but happened to be false.
HN limits reply depth, so i'll reply here:
> If you presuppose that the category is absurd, the submission of an absurd paper is a vacuous act that proves nothing, and your point should be able to be easily demonstrated by your knowledge of the field as it stands.
It is my opinion and the thesis of the paper, that the papers that they got accepted to these journals is absurd. I will quote here the theses description of the 7 accepted papers:
Sure, most got rejected. But 7 got accepted. And while banality is obviously a subjective value judgment, I don't think it applies to all of them. Just to enumerate the hypotheses that were accepted:
> That dog parks are rape-condoning spaces and a place of rampant canine rape culture and systemic oppression against “the oppressed dog” through which human attitudes to both problems can be measured. This provides insight into training men out of the sexual violence and bigotry to which they are prone.
> That it is only oppressive cultural norms which make society regard the building of muscle rather than fat admirable and that bodybuilding and activism on behalf of the fat could be benefited by including fat bodies displayed in non-competitive ways.
> That it is suspicious that men rarely anally self-penetrate using sex toys, and that this is probably due to fear of being thought homosexual (“homohysteria”) and bigotry against trans people (transphobia). (It combines these ideas into a novel concept “transhysteria,” which was suggested by one of the paper’s peer reviewers.) Encouraging them to engage in receptive penetrative anal eroticism will decrease transphobia and increase feminist values.
> That men frequent “breasturants” like Hooters because they are nostalgic for patriarchal dominance and enjoy being able to order attractive women around. The environment that breastaurants provide for facilitating this encourages men to identify sexual objectification and sexual conquest, along with masculine toughness and male dominance, with “authentic masculinity.” The data are clearly nonsense and conclusions drawn from it are unwarranted by it. (NB. One reviewer did raise concerns about the rigor of the data)
> That academic hoaxes or other forms of satirical or ironic critique of social justice scholarship are unethical, characterized by ignorance and rooted in a desire to preserve privilege.
> No clear thesis. A rambling poetic monologue of a bitter, divorced feminist, much of which was produced by a teenage angst poetry generator before being edited into something slightly more “realistic” which is then interspersed with self-indulgent autoethnographical reflections on female sexuality and spirituality written entirely in slightly under six hours.
> That feminism which foregrounds individual choice and responsibility and female agency and strength can be countered by a feminism which unifies in solidarity around the victimhood of the most marginalized women in society.
I'll grant you that two of them, the 'breasturants' and Mein Kampf are relatively mild on the absurdity scale. The rest, in my view, are self-evidently ridiculous and/or unethical. If you disagree, feel free to explain why.
> If you presuppose that the category is absurd, the submission of an absurd paper is a vacuous act that proves nothing, and your point should be able to be easily demonstrated by your knowledge of the field as it stands.
It is my opinion and the thesis of the paper, that the papers that they got accepted to these journals is absurd. I will quote here the theses description of the 7 accepted papers:
Sure, most got rejected. But 7 got accepted. And while banality is obviously a subjective value judgment, I don't think it applies to all of them. Just to enumerate the hypotheses that were accepted:
> That dog parks are rape-condoning spaces and a place of rampant canine rape culture and systemic oppression against “the oppressed dog” through which human attitudes to both problems can be measured. This provides insight into training men out of the sexual violence and bigotry to which they are prone.
> That it is only oppressive cultural norms which make society regard the building of muscle rather than fat admirable and that bodybuilding and activism on behalf of the fat could be benefited by including fat bodies displayed in non-competitive ways.
> That it is suspicious that men rarely anally self-penetrate using sex toys, and that this is probably due to fear of being thought homosexual (“homohysteria”) and bigotry against trans people (transphobia). (It combines these ideas into a novel concept “transhysteria,” which was suggested by one of the paper’s peer reviewers.) Encouraging them to engage in receptive penetrative anal eroticism will decrease transphobia and increase feminist values.
> That men frequent “breasturants” like Hooters because they are nostalgic for patriarchal dominance and enjoy being able to order attractive women around. The environment that breastaurants provide for facilitating this encourages men to identify sexual objectification and sexual conquest, along with masculine toughness and male dominance, with “authentic masculinity.” The data are clearly nonsense and conclusions drawn from it are unwarranted by it. (NB. One reviewer did raise concerns about the rigor of the data)
> That academic hoaxes or other forms of satirical or ironic critique of social justice scholarship are unethical, characterized by ignorance and rooted in a desire to preserve privilege.
> No clear thesis. A rambling poetic monologue of a bitter, divorced feminist, much of which was produced by a teenage angst poetry generator before being edited into something slightly more “realistic” which is then interspersed with self-indulgent autoethnographical reflections on female sexuality and spirituality written entirely in slightly under six hours.
> That feminism which foregrounds individual choice and responsibility and female agency and strength can be countered by a feminism which unifies in solidarity around the victimhood of the most marginalized women in society.
I'll grant you that two of them, the 'breasturants' and Mein Kampf are relatively mild on the absurdity scale. The rest, in my view, are self-evidently ridiculous and/or unethical. If you disagree, feel free to explain why.
Moreover, what I'm interested in: what goal did they have that goes beyond Sokal and SciGen-generated papers?
It wasn't clear from the article, and those two categories have already done a number of hoax papers. So we already know that hoax papers get accepted occasionally. A new hoax study would only be interesting if you specifically at our to attack a field of study - at this point, there have been enough hoaxes that show feasibility overall.
It wasn't clear from the article, and those two categories have already done a number of hoax papers. So we already know that hoax papers get accepted occasionally. A new hoax study would only be interesting if you specifically at our to attack a field of study - at this point, there have been enough hoaxes that show feasibility overall.
I've been more disturbed by the response to this than the papers themselves. Saying,well of course we were duped, you really expected the peer review system to not to utterly fail in the presence of a few bad actors? does not inspire my faith in the system.
That's what stuck out to me as well. This quote:
"I am so utterly unimpressed," wrote Jacob T. Levy, a political theorist at McGill University, "by the fact that an enterprise that relies on a widespread presumption of not-fraud can be fooled some of the time by three people with Ph.D.s who spend 10 months deliberately trying to defraud it."
Isn't the ability to separate out the quality work from the garbage the point of the peer review system? If the system was working as intended, the reviewers even with the assumption that they aren't being defrauded should have been able to sort the papers into the garbage pile.
Isn't the ability to separate out the quality work from the garbage the point of the peer review system? If the system was working as intended, the reviewers even with the assumption that they aren't being defrauded should have been able to sort the papers into the garbage pile.
> an enterprise that relies on a widespread presumption of not-fraud
This is section really gets me. How in the world is science supposed to rely on “not-fraud”? Sure if you fake your data you might dupe a diligent reviewer, but this is why results are supposed to be reproduced.
Anything that relies on “not-fraud” seems inherently unscientific and straight up naive.
This is section really gets me. How in the world is science supposed to rely on “not-fraud”? Sure if you fake your data you might dupe a diligent reviewer, but this is why results are supposed to be reproduced.
Anything that relies on “not-fraud” seems inherently unscientific and straight up naive.
Sussing out invalid research is the purpose of replication. A replication is itself a research project. It's not something that peer reviewers do.
Yes. My comment implies that you can get invalid research published. At least it was intended to.
The problem lies in areas where replication and formalization is difficult. There is certainly knowledge to be gained in these areas. Though calling the fields sciences, when the scientific method cannot be effectively applied, seems dubious.
The solution is not obvious, but it is clear that steps should be taken to improve rigor in the less formal (straight forward) bodies of scholarship.
The problem lies in areas where replication and formalization is difficult. There is certainly knowledge to be gained in these areas. Though calling the fields sciences, when the scientific method cannot be effectively applied, seems dubious.
The solution is not obvious, but it is clear that steps should be taken to improve rigor in the less formal (straight forward) bodies of scholarship.
It isn't possible to attempt replication of every scientific paper that's published. A lot of people on this thread seem to have difficulty accepting this fact, but the time and money to do that simply isn't available.
If a large percentage of scientists were to engage in deliberate fraud, there wouldn't be any feasible way to regulate that problem away. All available resources would end up being spent on scientists investigating each other for fraud, and no actual science would get done. Bear in mind that in many cases, establishing that an experiment is not fraudulent could require more time and effort than the original experiment!
As with many other human institutions, we have no choice but to rely on most people being honest most of the time.
If a large percentage of scientists were to engage in deliberate fraud, there wouldn't be any feasible way to regulate that problem away. All available resources would end up being spent on scientists investigating each other for fraud, and no actual science would get done. Bear in mind that in many cases, establishing that an experiment is not fraudulent could require more time and effort than the original experiment!
As with many other human institutions, we have no choice but to rely on most people being honest most of the time.
Social sciences have pretty deep-seated logistical obstacles to reproducibility.
If you want to reproduce an archaeology paper, well, how? You can’t exactly go dig the same site back up.
If you go out as a cultural anthropologist to hang out with some indigenous tribe somewhere, and your experiences aren’t the same as the last guy, does that mean anything? How much is because you’re different people? How much is because the tribe has already gotten used to having a researcher around?
This is part of why shit like the Piltdown Man happens.
If you want to reproduce an archaeology paper, well, how? You can’t exactly go dig the same site back up.
If you go out as a cultural anthropologist to hang out with some indigenous tribe somewhere, and your experiences aren’t the same as the last guy, does that mean anything? How much is because you’re different people? How much is because the tribe has already gotten used to having a researcher around?
This is part of why shit like the Piltdown Man happens.
It's supposed to rely on not-fraud because its incentive structure is not supposed to reward gaming the system (the way that "you get money" incentivizes fraud in a market system). A certain amount of good faith is presumed because the people who are eventually found retracted en mass will have a rough time getting work in the future, and because they won't be any closer to getting work published that will make them well-known scientists in the future.
Exactly this.
If my banking API relies on user's not trying to commit fraud instead of proper validation, then I have a very insecure API. Making a statement that your system works as long as no one tries to break it sounds almost satirical.
If my banking API relies on user's not trying to commit fraud instead of proper validation, then I have a very insecure API. Making a statement that your system works as long as no one tries to break it sounds almost satirical.
The result of successful banking API fraud is "I get a lot of money". The result of successful science publication fraud is, "I get some papers published, and as long as I don't get too famous and people don't look too hard at replicating my results later, maybe I get to include them in a bibliography or resume somewhere".
Reproduction of results happens after publication, though. Simply getting published does not demonstrate that their ideas were accepted as truthful, you should look at things like whether their work was cited by other work.
You better pay reviewers anything at all if you want them to produce few type 2 errors.
The point of the peer review system is to advance careers, not to produce high quality scholarship.
There are two separate issues here. One is that peer review fails in disciplines that are largely bogus and based on mimicry and posturing.
IMO this absolutely applies to a lot of academic social sciences - but it also applies equally, allowing for changes in tone, convention and social register, to their ideological equivalents on the right.
The other is that we really have a crisis of authority and competence in general. Between fake news, fake reviews, more and less professional trolls and manipulators, empty academic tribal signalling, wilfully dishonest fraud in professional science, the entire PR and thinktank industry of spin, manipulation, and lies for influence, and the slightly milder but still toxic ad industry, there's far too little useful signal now among the tsunami of noise that we have to sift through every day.
There are two separate issues here. One is that peer review fails in disciplines that are largely bogus and based on mimicry and posturing.
IMO this absolutely applies to a lot of academic social sciences - but it also applies equally, allowing for changes in tone, convention and social register, to their ideological equivalents on the right.
The other is that we really have a crisis of authority and competence in general. Between fake news, fake reviews, more and less professional trolls and manipulators, empty academic tribal signalling, wilfully dishonest fraud in professional science, the entire PR and thinktank industry of spin, manipulation, and lies for influence, and the slightly milder but still toxic ad industry, there's far too little useful signal now among the tsunami of noise that we have to sift through every day.
Separating out the quality work from the garbage is different than setting out the work done in good faith from the work done in bad faith by people who are otherwise capable of producing good work.
> Isn't the ability to separate out the quality work from the garbage the point of the peer review system
The peer review system is just one step in the scientific process. After passing the review it gets published and more scientists see it. They can then tear it down or build on top of it. If it's actually bad science, it probably won't survive long.
The peer review system is just one step in the scientific process. After passing the review it gets published and more scientists see it. They can then tear it down or build on top of it. If it's actually bad science, it probably won't survive long.
Not at all. Having reviewed papers, my goal is to make the underlying experiment as clear as possible for readers when the paper is eventually published. I care about the methods being clear and the analysis being sound. I am absolutely not looking for "I think the authors are lying" warnings because this way lies madness.
If we expect reviewers to reject papers they find weird or suspicious then a lot of excellent scholarship will be rejected because established researchers are stuck in their belief systems. It also leads to serious problems when an author has obtained some hard-to-get data set and this is now a downside when it comes to review.
If the results are garbage then that will come out later when people work with the ideas presented in the paper.
If we expect reviewers to reject papers they find weird or suspicious then a lot of excellent scholarship will be rejected because established researchers are stuck in their belief systems. It also leads to serious problems when an author has obtained some hard-to-get data set and this is now a downside when it comes to review.
If the results are garbage then that will come out later when people work with the ideas presented in the paper.
That's because reviewers of experimental papers are not expected to reproduce experiments. In theoretical physics or mathematics a reviewer absolutely should follow the proofs and check they are sound.
The humanities (as opposed to the social sciences) are theory fields, they don't have experiment but they advance knowledge by good arguments that require real scholarship to support even if they don't have crispness of proofs. Someone reviewing it should go "Yep, that paper make a real claim and argues for it well enough that convincing counter-argument would need to be a significant work of scholarship in itself".
If a field of study produces a conversation made up of such arguments and counter-arguments, then it stands some chance of being growing repository of truth. Whereas if the field can't distinguish between it's own chatter and a hoax guided by mood affiliation, then the field is likely to be just chatter.
The humanities (as opposed to the social sciences) are theory fields, they don't have experiment but they advance knowledge by good arguments that require real scholarship to support even if they don't have crispness of proofs. Someone reviewing it should go "Yep, that paper make a real claim and argues for it well enough that convincing counter-argument would need to be a significant work of scholarship in itself".
If a field of study produces a conversation made up of such arguments and counter-arguments, then it stands some chance of being growing repository of truth. Whereas if the field can't distinguish between it's own chatter and a hoax guided by mood affiliation, then the field is likely to be just chatter.
If the field doesn't rely on experimentation, why did their papers include fake experimental data (e.g. one of the papers contained a bunch of fake data about their observations of dog genitalia)?
What did you believe peer review was before you learned about this? You couldn't have actually done peer review before, because literally everyone who participates in that system in academia understands that they're at an immense disadvantage against a bad-faith submission. For every accepted paper, there are usually many more rejects, and all of them have to get read, and all in the reviewer's spare time.
Can someone comment on whether these are respectable journals or not, within their fields?
Every few years someone will do a Sokal-hoax-esque thing in scientific fields, but usually they're just getting a paper into very-low-quality predatory journals that have only a nominal peer review process.
I'm wondering if that's what happened here. Do humanities fields even have the same set of incentives leading to a proliferation of those sorts of journals?
Every few years someone will do a Sokal-hoax-esque thing in scientific fields, but usually they're just getting a paper into very-low-quality predatory journals that have only a nominal peer review process.
I'm wondering if that's what happened here. Do humanities fields even have the same set of incentives leading to a proliferation of those sorts of journals?
"We set out with three basic rules: (1) we’ll focus almost exclusively upon ranked peer-reviewed journals in the field, the higher the better and at the top of their subdisciplines whenever possible; (2) we will not pay to publish any paper; and (3) if we are asked at any point by a journal editor or reviewer (but not a journalist!) if any paper we wrote is an attempted hoax, we will admit it. These rules were meant to ensure that any conclusions we derived from the field came from the field itself, not the unrelated but significant problem that also corrupts academic pursuits: the proliferation of predatory and quasi-predatory journals with extremely low standards. With these rules guiding us, we committed to transparently reporting the results, whether we succeeded or failed."
https://areomagazine.com/2018/10/02/academic-grievance-studi...
I'm not qualified to say if these truly were considered respectable journals in their fields but the authors had that as an explicit aim.
https://areomagazine.com/2018/10/02/academic-grievance-studi...
I'm not qualified to say if these truly were considered respectable journals in their fields but the authors had that as an explicit aim.
Well, they claim to have targeted "top journals" in the field, but it's not like "intersectional gender studies" is a particularly big field.
I looked up two journals they had published in and their articles got less than one citation per year typically ([1]&[2]), not exactly a booming field.
[1]: https://www.scimagojr.com/journalsearch.php?q=145138&tip=sid... [2]: https://www.scimagojr.com/journalsearch.php?q=14798&tip=sid&...
I looked up two journals they had published in and their articles got less than one citation per year typically ([1]&[2]), not exactly a booming field.
[1]: https://www.scimagojr.com/journalsearch.php?q=145138&tip=sid... [2]: https://www.scimagojr.com/journalsearch.php?q=14798&tip=sid&...
> it's not like "intersectional gender studies" is a particularly big field
And let's hope it stays that way. This will help.
And let's hope it stays that way. This will help.
[throw away account as I am a recovering academic and wouldn't want to impact colleagues who have not been able to escape]
Everyone is distracted by the politics here when the real story is the effort.
Three people layman to these fields achieved a near tenure track publication list/ratio in a little over a year. I have a STEM PhD and I know that me and three of my friends couldn't do the same if we tried in a closely related STEM discipline much less an entirely different field.
Hypatia is the journal of record for feminist philosophy (and one of the few respected feminist journals in general). It should not be possible to go from 0 to published in a "real" top tier journal. If the paper is "not too crazy" that's even more damning! They produced a "real" paper as a joke? Do you really think that these same authors could get a paper published in The International Journal of Computer Vision in a little less than a year even if they were using fraudulent data? It would take them two years to know what they could and couldn't fake without being caught.
If the liberal arts don't want the sneering dismissal of their fields as low effort to become culturally entrenched they need to dump or reform the "X studies" departments and journals.
Everyone is distracted by the politics here when the real story is the effort.
Three people layman to these fields achieved a near tenure track publication list/ratio in a little over a year. I have a STEM PhD and I know that me and three of my friends couldn't do the same if we tried in a closely related STEM discipline much less an entirely different field.
Hypatia is the journal of record for feminist philosophy (and one of the few respected feminist journals in general). It should not be possible to go from 0 to published in a "real" top tier journal. If the paper is "not too crazy" that's even more damning! They produced a "real" paper as a joke? Do you really think that these same authors could get a paper published in The International Journal of Computer Vision in a little less than a year even if they were using fraudulent data? It would take them two years to know what they could and couldn't fake without being caught.
If the liberal arts don't want the sneering dismissal of their fields as low effort to become culturally entrenched they need to dump or reform the "X studies" departments and journals.
Representative quote:
> "I am so utterly unimpressed," wrote Jacob T. Levy, a political theorist at McGill University, "by the fact that an enterprise that relies on a widespread presumption of not-fraud can be fooled some of the time by three people with Ph.D.s who spend 10 months deliberately trying to defraud it."
> "I am so utterly unimpressed," wrote Jacob T. Levy, a political theorist at McGill University, "by the fact that an enterprise that relies on a widespread presumption of not-fraud can be fooled some of the time by three people with Ph.D.s who spend 10 months deliberately trying to defraud it."
Representative quotes:
"Is there any idea so outlandish that it won’t be published in a Critical/PoMo/Identity/‘Theory’ journal?" tweeted the Harvard psychologist Steven Pinker.
"Three intrepid academics," wrote Yascha Mounk, an author and lecturer on government at Harvard, "just perpetrated a giant version of the Sokal Hoax, placing … fake papers in major academic journals. Call it Sokal Squared. The result is hilarious and delightful. It also showcases a serious problem with big parts of academia."
"Is there any idea so outlandish that it won’t be published in a Critical/PoMo/Identity/‘Theory’ journal?" tweeted the Harvard psychologist Steven Pinker.
"Three intrepid academics," wrote Yascha Mounk, an author and lecturer on government at Harvard, "just perpetrated a giant version of the Sokal Hoax, placing … fake papers in major academic journals. Call it Sokal Squared. The result is hilarious and delightful. It also showcases a serious problem with big parts of academia."
Mounk didn't read the hoaxers writeup carefully enough to be a reliable judge; he gleefully tweeted two papers from the hoax that were rejected everywhere they were submitted, as evidence of the important problem the hoax uncovered --- this despite the fact that the hoaxers did in fact provide an accounting of what got accepted where (even though their writeup at the beginning of the piece misrepresented that accounting).
From what Pinker wrote, who knows if he even read it?
From what Pinker wrote, who knows if he even read it?
> can be fooled some of the time
7⁄20 or 7⁄17, depending on how you want to count is concerning.
7⁄20 or 7⁄17, depending on how you want to count is concerning.
Shouldn't the default presupposition of any research be a lack of novelty?
I think generally people tend to assume that colleagues aren't dishonest, and mostly peer-review is supposed to catch honest mistakes.
You could try working without any faith in your colleagues of course, but you are probably better of switching workplace if that sounds like reasonable behaviour to you.
You could try working without any faith in your colleagues of course, but you are probably better of switching workplace if that sounds like reasonable behaviour to you.
Alternative take: A system designed to detect bad research failed to detect blatant falsehoods, algorithmically generated noise, rabid propaganda and other seemingly easy things to detect. That PhDs worked on this doesn’t mean the system didn’t fail spectacularly.
> doesn’t mean the system didn’t fail spectacularly.
You're mischaracterizing the system; it doesn't end with publication. After publication, the science goes out to a wider audience of scientists who can build on the paper or refute it if they find it's wrong. Sometimes this process can take a long time and many cycles of publicans, but that's just the nature of science.
We have to stop pretending like the peer review process is some kind of magical filter of true science, or that it was ever intended to be as such. That notion leads to the faulty idea that if it's published, it's true. Junk science gets published, even if it's not disingenuous, and that's probably true no matter how great you make the peer review system.
You're mischaracterizing the system; it doesn't end with publication. After publication, the science goes out to a wider audience of scientists who can build on the paper or refute it if they find it's wrong. Sometimes this process can take a long time and many cycles of publicans, but that's just the nature of science.
We have to stop pretending like the peer review process is some kind of magical filter of true science, or that it was ever intended to be as such. That notion leads to the faulty idea that if it's published, it's true. Junk science gets published, even if it's not disingenuous, and that's probably true no matter how great you make the peer review system.
What is the point of journals if quite literally anything can be published in them? Isn't the point that they serve as a basic filter for obvious noise? Shouldn't we be critical of a peer review process that doesn't function at even the most basic level?
A filter that does filter out the vast majority of obvious noise is very useful even if it lets something through. And the current peer-review does that, it throws out almost all of the spam; it pushes useful work of lower quality/impact down to less selective venues (generally with more specific subfields, so that I can read impactful work of domain X and less impactful work of subdomain X.2 but not X.1 and X.3); and even for accepted papers the process often results in significant improvements to them as a result of the reviews.
Explicit fraud isn't that common, since it does have risks to the submitter that far outweigh the (limited) benefits of getting an extra publication on your CV.
Explicit fraud isn't that common, since it does have risks to the submitter that far outweigh the (limited) benefits of getting an extra publication on your CV.
> quite literally anything can be published
I've got a stack of rejected papers that beg to differ.
> Isn't the point that they serve as a basic filter for obvious noise?
Yeah, and obvious noise is usually rejected. But sometimes it is not so obvious what the noise is, and that's why the scientific process is bigger than the peer review system.
> Isn't the point that they serve as a basic filter for obvious noise?
Totally, there is a lot to fix about it. But if you really want to put your effort into changing the system, I don't think you should focus on peer review. Destroying the business model of journals is where I would start.
I've got a stack of rejected papers that beg to differ.
> Isn't the point that they serve as a basic filter for obvious noise?
Yeah, and obvious noise is usually rejected. But sometimes it is not so obvious what the noise is, and that's why the scientific process is bigger than the peer review system.
> Isn't the point that they serve as a basic filter for obvious noise?
Totally, there is a lot to fix about it. But if you really want to put your effort into changing the system, I don't think you should focus on peer review. Destroying the business model of journals is where I would start.
Why is this being downvoted? This is a description of the "invisible boot" as Broad & Wade call it.
I'm not saying that peer review is the end all be all of truth, but it does serve some purpose. And in this case it didn't provide any value at all.
Peer Review is not set up to detect fraudulent research and attempts to do so are fraught with risk, since established researchers often have a hard time believing in results from new paradigms. What the heck is an author supposed to do if a reviewer says "I think these numbers are made up"?
The purpose of peer review is to decide whether a paper is useful to the community, whether the presentation is good enough that the community can precisely understand what happened, and whether it has methodological issues that the authors may have missed. But we always approach reviews with a charitable lens, even when reviewing for extremely low-acceptance outlets.
The process that detects the bad research comes afterwards, as it should.
The purpose of peer review is to decide whether a paper is useful to the community, whether the presentation is good enough that the community can precisely understand what happened, and whether it has methodological issues that the authors may have missed. But we always approach reviews with a charitable lens, even when reviewing for extremely low-acceptance outlets.
The process that detects the bad research comes afterwards, as it should.
> The purpose of peer review is to decide whether a paper is useful to the community, whether the presentation is good enough that the community can precisely understand what happened, and whether it has methodological issues that the authors may have missed.
It clearly failed to do that in this case.
It clearly failed to do that in this case.
Ok? That somebody with a year of free time and a pile of fraudulent attempts can get through sometimes is not exactly a surprise.
A system that prevents what the authors did here would necessarily reject a huge amount of useful research.
A system that prevents what the authors did here would necessarily reject a huge amount of useful research.
The whole point of a peer review system is to detect fraud. If these bullshit papers got published, who knows how many of the other papers are of equal quality.
The point of peer review is to stop bad research, not fake research. The reviewers are supposed to make sure that the methods used to acquire the data are clearly documented and that the conclusions drawn from the data follow from the data that was collected.
They can't verify that the data collected actually was collected without literally watching over the shoulder of the researcher while the researcher was collecting it, which is obviously not feasible. A researcher actively trying to deceive the reviewer with fake data but with a good method for collecting that data and analysis of the data that would be correct if the data were real is not something that should be caught by a reviewer. That should be caught by someone reproducing the research after publication.
They can't verify that the data collected actually was collected without literally watching over the shoulder of the researcher while the researcher was collecting it, which is obviously not feasible. A researcher actively trying to deceive the reviewer with fake data but with a good method for collecting that data and analysis of the data that would be correct if the data were real is not something that should be caught by a reviewer. That should be caught by someone reproducing the research after publication.
>The point of peer review is to stop bad research, not fake research
I would submit that "fake research" is, in fact, bad research.
I would submit that "fake research" is, in fact, bad research.
To a reviewer with three hours to spare, fake research is indistinguishable from good research. Unless you want to give me months to review a paper, I will not be able to detect fraud.
Instead I rely on the fact that the community will attempt to integrate these ideas after publication and arrive at the conclusion that the paper was garbage.
Instead I rely on the fact that the community will attempt to integrate these ideas after publication and arrive at the conclusion that the paper was garbage.
If you only have three hours, can't detect fraud, can't tell if the the paper is even real. Then what's the point of even doing the peer review? Again, peer review offered no value at all in this case.
Peer review can still inspect whether the evidence presented in a paper would—if not fabricated—truly be enough to support the paper's conclusion. Mistaken analysis of honestly collected data, improper experimental methods, flawed proofs, etc. can all show up at review time.
You can certainly come to that conclusion if you'd like. Arxiv exists. But "peer review isn't especially usrful" and "these entire fields are nonsense" are very different conclusions.
Where did you learn that the whole point of peer review is to detect fraud?
The Wikipedia article on peer review says "Peer review...occasionally detects fraud, but is not designed to do so."[1]
The article cites a Nature article that says "Peer review is not currently designed to detect deception, nor does it guarantee the validity of research findings. It should, however, identify flaws in the design, presentation, analysis and interpretation of science and provide prompt, detailed, constructive criticism to improve research."
[1] https://en.wikipedia.org/wiki/Scholarly_peer_review
[2] https://www.nature.com/nature/peerreview/debate/nature05007....
The Wikipedia article on peer review says "Peer review...occasionally detects fraud, but is not designed to do so."[1]
The article cites a Nature article that says "Peer review is not currently designed to detect deception, nor does it guarantee the validity of research findings. It should, however, identify flaws in the design, presentation, analysis and interpretation of science and provide prompt, detailed, constructive criticism to improve research."
[1] https://en.wikipedia.org/wiki/Scholarly_peer_review
[2] https://www.nature.com/nature/peerreview/debate/nature05007....
I see a lot of people complaining that it is done in bad faith, and doesn't prove much.
But is there really much to prove? What's the baseline? I mean, we're talking about a field where this got published - and it was not a hoax:
http://journals.sagepub.com/doi/full/10.1177/030913251562336...
The people behind this hoax had an unenviable task of matching and beating a paper like that (since, apparently, by itself it was not sufficient to prove the point). How would you do so?
But is there really much to prove? What's the baseline? I mean, we're talking about a field where this got published - and it was not a hoax:
http://journals.sagepub.com/doi/full/10.1177/030913251562336...
The people behind this hoax had an unenviable task of matching and beating a paper like that (since, apparently, by itself it was not sufficient to prove the point). How would you do so?
I'd be curious to actually read that Glaciers, gender, and science paper to see if it's as worthless as it sounds.
I think there's some amount of feminist theory that boils down to -- all of the people thinking/doing stuff in A field were men operating under B culture; what if we thought about things from a different perspective & revisited some of our assumptions?
Like most fields, I bet glaciology has some underlying assumptions (is the goal of glaciology to manage glaciers? navigation?) & history (which glaciers have people historically studied? are they representative?)
And I know as an outsider, it's really hard to tell what's valuable about another field. If an outsider was critiquing computer science, I bet they'd easily be able to track down studies about tabs vs. spaces and laugh at the silly stuff that gets published in that field.
I think there's some amount of feminist theory that boils down to -- all of the people thinking/doing stuff in A field were men operating under B culture; what if we thought about things from a different perspective & revisited some of our assumptions?
Like most fields, I bet glaciology has some underlying assumptions (is the goal of glaciology to manage glaciers? navigation?) & history (which glaciers have people historically studied? are they representative?)
And I know as an outsider, it's really hard to tell what's valuable about another field. If an outsider was critiquing computer science, I bet they'd easily be able to track down studies about tabs vs. spaces and laugh at the silly stuff that gets published in that field.
Bad papers get published in all fields. Linking to one bad journal article shows nothing.
Here, for example, is a reference to a bad paper published in a field which is not usually the target of this kind of criticism:
https://academia.stackexchange.com/questions/9602/rediscover...
https://math.berkeley.edu/~ehallman/math1B/TaisMethod.pdf
Here, for example, is a reference to a bad paper published in a field which is not usually the target of this kind of criticism:
https://academia.stackexchange.com/questions/9602/rediscover...
https://math.berkeley.edu/~ehallman/math1B/TaisMethod.pdf
True, but when such papers are discovered, they're usually torn down by people in the field. In this case, it was the other way around - all meaningful criticism came from the outside, and in response, researchers in the field mostly circled the wagons.
Besides, the paper in question wasn't merely published - the author got a prestigious NSF award that lists that paper:
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1253779
This, to me, implies that either such papers are considered broadly acceptable in the field, or they're not actually read. Not sure which is worse.
Besides, the paper in question wasn't merely published - the author got a prestigious NSF award that lists that paper:
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1253779
This, to me, implies that either such papers are considered broadly acceptable in the field, or they're not actually read. Not sure which is worse.
Have you read the paper you linked to, or just the abstract and quotes in the right wing press?
You're confused about how NSF grants work. The publication is listed as an output. The grant was awarded before that paper was published. The listing of the paper on that page implies nothing about its quality.
My example paper was torn down by mathematicians, not diabetes researchers.
You're confused about how NSF grants work. The publication is listed as an output. The grant was awarded before that paper was published. The listing of the paper on that page implies nothing about its quality.
My example paper was torn down by mathematicians, not diabetes researchers.
It's not just the humanities, computer science is also susceptible to hoaxes built from jargon.
SCIgen works like an academic "Mad Libs" of sorts, arbitrarily slotting in computer-science buzzwords like "distributed hash tables" and "Byzantine fault tolerance."
http://news.mit.edu/2015/how-three-mit-students-fooled-scien...
SCIgen works like an academic "Mad Libs" of sorts, arbitrarily slotting in computer-science buzzwords like "distributed hash tables" and "Byzantine fault tolerance."
http://news.mit.edu/2015/how-three-mit-students-fooled-scien...
That paper was accepted as a non-reviewed paper. There was no peer-review process.
Which one? Both that article and this one lists multiple cases where SCIgen-generated papers were waved through:
https://en.wikipedia.org/wiki/SCIgen#Prominent_results
https://en.wikipedia.org/wiki/SCIgen#Prominent_results
From the article:
In April of 2005 the team’s submission, “Rooter: A Methodology for the Typical Unification of Access Points and Redundancy,” was accepted as a non-reviewed paper to the World Multiconference on Systemics, Cybernetics and Informatics (WMSCI), a conference that Krohn says is known for “being spammy and having loose standards.”
In April of 2005 the team’s submission, “Rooter: A Methodology for the Typical Unification of Access Points and Redundancy,” was accepted as a non-reviewed paper to the World Multiconference on Systemics, Cybernetics and Informatics (WMSCI), a conference that Krohn says is known for “being spammy and having loose standards.”
> I don't want to know whether social constructionism is considered "problematic", I want to know whether it is true.
Not all legitimate scholarly work is based on a search for scientific truth. E.g. Interpretations/analysis of Shakespeare's work.
Not all legitimate scholarly work is based on a search for scientific truth. E.g. Interpretations/analysis of Shakespeare's work.
I've noticed a lot of hair-splitting in these comments over whether this was a failure of peer review, whether to call the papers, considered as specimens of scholarship in their appropriate fields, "fraudulent" rather than "bad", or whether the number of hours they actually logged at the dog park is a relevant concern. All of this is quite besides the point. The researchers passed a version of the Turing Test among the reviewers of their papers. All that we need from that point is an acknowledgement of the absurdity and falseness of the papers themselves. This may not be easily forthcoming, though. One of the reviewers of Social Text took the position, in the wake of Sokal's hoax, that his paper constituted good scholarship, despite the stated intentions of its author. Also confer with Poe's Law. However, when we admit this as a possible response, we are in the territory of radical relativism, with no way to adjudicate between the claims of these scholars and their critics, and for that matter, those of religious fanatics, mentally ill delusional people, confidence scammers, or indeed anybody at all. This is a position which seems, at least, to be somewhat at odds with the fundamental scholarly enterprise. At its heart, one has to cross the pons arsinorum of admitting that, yes, the papers are indeed absurd nonsense. At this point, it seems impossible to convince some people to take this step.
If the critics's objections could be distilled into something with a semblance of validity and germaneness, then it would be that the researchers's methods were too unfocused. But had they taken a narrower approach, the response could just have been to rationalize and ignore. Fight fire with fire (being generality, in this case).
If the critics's objections could be distilled into something with a semblance of validity and germaneness, then it would be that the researchers's methods were too unfocused. But had they taken a narrower approach, the response could just have been to rationalize and ignore. Fight fire with fire (being generality, in this case).
The WSJ editorial linked to has more info: http://archive.is/llwSP.
The trio could have reached out to colleagues in physics and other fields, but instead opted for "poor experimental design."
They could, but the Bogdanov brothers did just that years earlier.
They could, but the Bogdanov brothers did just that years earlier.
Journals are for reports from scientists and academics. Would they be more useful if there was some additional review of the authors reputation, and the reliability of the particular report? Upvoting or public signing of the documents, or public signed comments. An expert in a field is more familiar with the reputation of their colleagues than outsiders.
The authors state that this was a long term research project cut short. Which suggests the question:
If you played this game long enough, undetected, could you get real scholars to cite your fake research?
Future work, perhaps.
(I don't have any direct evidence of this, but I strongly suspect this is an hypothesis that the authors would have wanted to test.)
If you played this game long enough, undetected, could you get real scholars to cite your fake research?
Future work, perhaps.
(I don't have any direct evidence of this, but I strongly suspect this is an hypothesis that the authors would have wanted to test.)
I hope there is someone out there right now, working towards a degree in Critical <whatever> Studies for no other reason than to get a PhD publishing nonsense and reveal it all afterwards.
If the author’s intention was to show the dubious quality of research in these fields, wouldn’t a better approach have been to select high impact articles from prestigious journals in the fields which they believed exemplified the problem, and then written rebuttals/critiques arguing why they are flawed?
[deleted]
> To answer these questions, this article engages feminist geography and broader
feminist literature and draws on nearly 1000 h of public observations of dogs and
their human companions conducted at three dog parks in Southeast Portland,
Oregon, beginning on 10 June 2016, and ending on 10 June 2017.
In Sokal's single hoax in 1996 he (and his buddies, I guess) made a claim, followed it by a non-sequitur of true but uninteresting sentences about physics, then repeated the claim. It was a sandwich with no meat in it, and the (small, non-peer-reviewed) post-modern journal ate it up.
Sokal wanted to show that a certain style of post-modern writing substituted scientific jargon for an argument (or at least it used references to esoteric scientific fields as pretentious, uninteresting metaphors). The elegance of parodying this style was that his submission contained no data at all. It wasn't merely that the conclusion of the submission wouldn't be replicated. It was that any conscious human should notice the complete lack of supporting evidence for the claim!
The upshot of the journal accepting the article was that "the emperor had no clothes," so to speak. Sokal went on to write a book showing how several prominent post-modern thinkers used scientific non-sequiturs to make their non-scientific ideas sound more weighty. (Just as he had in his hoax.)
These three authors, however, are going a different route from Sokal-- specifically, one that provides fake data as evidence of a claim. (Or I guess they could have actually sat at dog parks for 1000 hours, but it doesn't matter for the point I'm making.) By providing fake data, this hoax passes an initial filter which the Sokal hoax did not-- namely, the filter that should reject submissions which lack any supporting evidence whatsoever. That means this hoax only reveals a lack of replication and/or verification, whereas the Sokal hoax reveals a lack of both peer review and basic reading comprehension even by a non-physicist. That makes this a weaker class of hoax than the Sokal hoax.
Finally, the authors purport to be going after an entire "ideological slant" in academia. Whatever one thinks of that aim, it is enormous and amorphous when compared to the narrow aim of Sokal keeping scientific jargon from being misused/misplaced.
So to recap:
* 1996: strong hoax successful against a single pomo journal, weak but persuasive claim against coopting scientific jargon for non-scientific aims
* 2018: weak hoax successful against 7 of (total number?) journals, strong and unpersuasive claim against all leftist academic fields
Now I'm curious-- did anyone in the late 90s criticize Sokal's hoax on the grounds that his careful constraints would be ignored by future pranksters?
Edit: clarification
In Sokal's single hoax in 1996 he (and his buddies, I guess) made a claim, followed it by a non-sequitur of true but uninteresting sentences about physics, then repeated the claim. It was a sandwich with no meat in it, and the (small, non-peer-reviewed) post-modern journal ate it up.
Sokal wanted to show that a certain style of post-modern writing substituted scientific jargon for an argument (or at least it used references to esoteric scientific fields as pretentious, uninteresting metaphors). The elegance of parodying this style was that his submission contained no data at all. It wasn't merely that the conclusion of the submission wouldn't be replicated. It was that any conscious human should notice the complete lack of supporting evidence for the claim!
The upshot of the journal accepting the article was that "the emperor had no clothes," so to speak. Sokal went on to write a book showing how several prominent post-modern thinkers used scientific non-sequiturs to make their non-scientific ideas sound more weighty. (Just as he had in his hoax.)
These three authors, however, are going a different route from Sokal-- specifically, one that provides fake data as evidence of a claim. (Or I guess they could have actually sat at dog parks for 1000 hours, but it doesn't matter for the point I'm making.) By providing fake data, this hoax passes an initial filter which the Sokal hoax did not-- namely, the filter that should reject submissions which lack any supporting evidence whatsoever. That means this hoax only reveals a lack of replication and/or verification, whereas the Sokal hoax reveals a lack of both peer review and basic reading comprehension even by a non-physicist. That makes this a weaker class of hoax than the Sokal hoax.
Finally, the authors purport to be going after an entire "ideological slant" in academia. Whatever one thinks of that aim, it is enormous and amorphous when compared to the narrow aim of Sokal keeping scientific jargon from being misused/misplaced.
So to recap:
* 1996: strong hoax successful against a single pomo journal, weak but persuasive claim against coopting scientific jargon for non-scientific aims
* 2018: weak hoax successful against 7 of (total number?) journals, strong and unpersuasive claim against all leftist academic fields
Now I'm curious-- did anyone in the late 90s criticize Sokal's hoax on the grounds that his careful constraints would be ignored by future pranksters?
Edit: clarification
All of the above, because it exposes vulnerability to bad faith.
I cant help but give a Gilfoyle style heh... heh... heh....
What I hope has happened is the 20 revealed papers are not actually all of them and that they do a Wikileaks style drip release, wait on a response, then hit us with more much later.
What I hope has happened is the 20 revealed papers are not actually all of them and that they do a Wikileaks style drip release, wait on a response, then hit us with more much later.
If they had those results, they would have put them front and center. Just like if the TSA had ever caught a terrorist, it would be on the 24-hour networks for days.
This is the best thing I've seen in quite a while.
If the people at the top of your discipline can't tell the difference between quality and inane garbage, what do you really have?
Not a discipline.
[deleted]
This wasn't a hoax it was social commentary and an experiment in prejudice. That nazi language can be published under a progressive guise should be chilling to anyone who is a vehicle for these ideals.
It wasn't a hoax. It's a confounding detail that a meta-commentary article is employing.
Sorry, I can't parse that at all. Could you be a bit clearer on what you're trying to say here?
The article is attempting to explain the events, by characterizing them as a type of deception when it shows the exact opposite (it's not a hoax that papers are accepted on political aim, rather than evidence). It's a small detail, but telling as to the intent and voracity without knowing most the story.
Most published stuff is irrelevant crap that will never be cited or included in the central canon. This is true even in computer science. Also, as we know from computer security, system design becomes massively more difficult if it requires hardening against malicious actors. The presumption of good faith is a valuable component of any community that shouldn't be abused to score stupid points.
For these reasons, Sokal Hoaxes (and their like) are damaging wastes of time and the perpetrators should not be praised.
For these reasons, Sokal Hoaxes (and their like) are damaging wastes of time and the perpetrators should not be praised.
>The presumption of good faith is a valuable component of any community that shouldn't be abused to score stupid points.
Shouldn't we be concerned with the possibility of undetected bad actors who are abusing it for more nefarious reasons?
Shouldn't we be concerned with the possibility of undetected bad actors who are abusing it for more nefarious reasons?
Yeah... that's the point! The whole point here is that you have these "conspicuously ideological" fields or subfields, which are not doing real scholarship, not interested in truth, but which rather are pushing a specific position and attempting to label as evil and shout down anyone who disagrees; and yet they're able to get away with claiming that they're doing real work, that this is what real scholarship is, and it's seriously damaging, especially (as the hoaxers note) to anyone who does want to do real truth-seeking work in these areas.
Sure, to those of us who realize what's going on, more hardly needs to be said. But lots of people don't, and the "grievance scholars" are slowly convincing the rest of academia that real scholarship consists not of neutral methods and honest argument, but agreeing with the grievance scholars in all things, and finding ways to inject it whether it's relevant or not. The point of this hoax is to make obvious what already should be.
To put it another way -- "grievance studies" is, as Dr. Steel says, an easy epistemological target. But it's a hard social target. It's built up a layer of protection around itself consisting not of truth but of social pressure and demands to conform or be ostracized. No claim, true or false, should be so protected, and that shield needs to be shattered. It's only an "easy target" in the imaginary world where people are rational and such transparent bullshit doesn't work. But in reality, it does, which is why there's a need to puncture it.