Spin found in clinical trial abstracts in top psychology and psychiatry journals(sciencebeta.com)
sciencebeta.com
Spin found in clinical trial abstracts in top psychology and psychiatry journals
https://sciencebeta.com/spin-psychiatry-journals/
22 comments
I'm a novice in my IT career. If you're able and willing, would you mind explaining to me how software (spun as it may be) for disaster relief, could be used for data collection outside the scope of the objectives you had in the name of disaster relief? If this question misses the mark you're welcome to ignore it. Thanks!
It's probably image analysis from overflight, if I were to guess, or an ML-augmented database.
This article might be spin itself, given how it captures attention with the accusation but hides what was actually tested with the circumlocation "a previously published definition of spin". For all I know, they just counted every abstract that contained the word "may" and called it spin.
The issue is probably two-pronged:
1. There is pressure from the department (and funders, as the article mentions) to produce publications. The need to publish leads to researchers to spin null results as a way to "cash in" the time and effort put into a research project (I'm assuming good intentions here on the part of the researchers).
2. Articles get published if the authors can argue that their work makes a new and unique contribution to the field. This argument usually rests less on the article's results and more on the discussion of these results. I am not surprised that authors spin the results in their discussion sections.
As a side note, most readers only read the introduction and discussion section of a paper. Those two sections usually have the most spin as they are written to "hook" readers in the paper (and to an extent, to be accessible to a non-technical person sometimes outside of the field). Lit review, methods, and results sections are usually more dry as they report on the technical aspects of a research project.
As a side note, most readers only read the introduction and discussion section of a paper. Those two sections usually have the most spin as they are written to "hook" readers in the paper (and to an extent, to be accessible to a non-technical person sometimes outside of the field). Lit review, methods, and results sections are usually more dry as they report on the technical aspects of a research project.
huh? we read the methods and results first, then the spin (exception, the abstracts)
Several major journals, including nature, put the methods at the end.
so? do you read scientific papers in order? I don't.
[deleted]
Abstracts are typically between 150 and 250 words, yet studies, especially clinical trials, are frequently complex affairs. Fitting nuance into an abstract is just often not possible, and frankly, the clearest most succinct language usually oversells the results.
Ah, but if they had not found spin we would never have heard of their study...
Also: "Spin" found in over 50% of top quantum mechanics journal abstracts.
They're called soft sciences for a reason, they're more about interpretation and opinion than hard irrefutable evidence. So this is not surprising, it's probably quite a bit higher than 50% too, and in all of the soft science fields where agendas or political motives may be present; sociology, psychology, political science, some archaeology, etc
Rubbish.
The papers analyzed here are reporting the outcomes of randomized controlled trials. Political motivations, opinions, and interpretations have nothing to do with any of these papers' results.
A fair number of HN commentators have this bizarre caricature of psychology as someone lying on the couch, chattering about feelings while a tweed-jacketed man scribbles away. However, it hasn't been 1920 for a long time and the field has grown up and moved on. It's an actual science, with experiments and quantitative analysis\* and all that. The problems it tackles are really hard, in some ways evenmoreso than physics: you can generate an endless stream of identical electrons, but every human is different and has their own motivations.
\* Yes, yes, replication crisis, fMRI salmon of doubt. It's serious but psychology should get some credit for a) identifying it in the first place and b) fixing it. These problems certainly aren't confined to psych--it's mostly that psych experiments are easier to replicate.
The papers analyzed here are reporting the outcomes of randomized controlled trials. Political motivations, opinions, and interpretations have nothing to do with any of these papers' results.
A fair number of HN commentators have this bizarre caricature of psychology as someone lying on the couch, chattering about feelings while a tweed-jacketed man scribbles away. However, it hasn't been 1920 for a long time and the field has grown up and moved on. It's an actual science, with experiments and quantitative analysis\* and all that. The problems it tackles are really hard, in some ways evenmoreso than physics: you can generate an endless stream of identical electrons, but every human is different and has their own motivations.
\* Yes, yes, replication crisis, fMRI salmon of doubt. It's serious but psychology should get some credit for a) identifying it in the first place and b) fixing it. These problems certainly aren't confined to psych--it's mostly that psych experiments are easier to replicate.
Have you ever talked to people in the social sciences ? There very much is political motivations, opinions and interpretations. And don't forget careers. Many people have built decade-long careers on being the go-to person (or even department, it can be a dozen careers) for specific treatments. Look up what EMDR is, for a particularly ludicrous example, and read the career of Francine Shapiro. Note that she had decades in which she had a great position to collect support and evidence for her therapy, as a professor, director of a treatment center and practicing clinician ... and somehow she never did (mind if I make the assumption that she did in fact collect plenty of data, it just said what all psychiatric studies say about all treatments)
Psychoanalysis or "someone lying on the couch, chattering about feelings while a tweed-jacketed man scribbles away" was done until deep in the 80s, where it was canceled because of the famous observation that it was exactly as effective as giving patients a free membership card to a sport club (not even checking if they used it at all: so some patients wouldn't have).
But truth is that it was not just done because psychoanalysis wasn't effective. It in fact psychoanalysis is definitely the most effective treatment in psychiatry. It was done so attention from expensive psychiatrists could be removed entirely from patients, as a cost-saving measure. Now all psychiatrists do is draw out treatment plans, to be executed by people with much less training than they have. Now a patient in mental health treatment is lucky to see a psychiatrist once a month.
All these treatments found are known to be far less effective than psychoanalysis. Psychiatry as a whole is simply a fraud. There isn't a single known treatment against depression, other than fixing the problem that causes the depression (which is external to the patient, and never has anything to do with psychiatry)
For the same story with added drama, see https://en.wikipedia.org/wiki/Rosenhan_experiment
Psychoanalysis or "someone lying on the couch, chattering about feelings while a tweed-jacketed man scribbles away" was done until deep in the 80s, where it was canceled because of the famous observation that it was exactly as effective as giving patients a free membership card to a sport club (not even checking if they used it at all: so some patients wouldn't have).
But truth is that it was not just done because psychoanalysis wasn't effective. It in fact psychoanalysis is definitely the most effective treatment in psychiatry. It was done so attention from expensive psychiatrists could be removed entirely from patients, as a cost-saving measure. Now all psychiatrists do is draw out treatment plans, to be executed by people with much less training than they have. Now a patient in mental health treatment is lucky to see a psychiatrist once a month.
All these treatments found are known to be far less effective than psychoanalysis. Psychiatry as a whole is simply a fraud. There isn't a single known treatment against depression, other than fixing the problem that causes the depression (which is external to the patient, and never has anything to do with psychiatry)
For the same story with added drama, see https://en.wikipedia.org/wiki/Rosenhan_experiment
Of course.
I'm not sure that's the sort of politics the original comment had in mind, and I'm certain that these things are not exclusive to the social sciences. I've heard that parts of CERN can be very political, and I've personally seen two adult men nearly come to blows over the synaptic organization of visual cortex. Internal politics, differences of opinion, and interpretation are parts of every human endeavor.
EMDR sounds goofy, but there may actually be something there. When Francine Shapiro died last month, the New York Times ran an article about her and her work, which for which they interviewed Richard McNally, who was one of EMDR's fiercest critics in the 1990s and 2000s; at the time, he called it "merely one of the many therapeutic fuzz-balls that litter the landscape of psychology today." In this article, he says “I’ve changed my mind,” Dr. McNally said in a phone interview. “I’m willing to do that based on new evidence. It looks like there’s something going on there; the representation of the trauma seems to be reconsolidated in a way that doesn’t distress people as much when later recalled.” From here: https://www.nytimes.com/2019/07/11/science/francine-shapiro-... I'm not a clinical psychologist and I don't know a ton about EMDR, but I do know a fair bit about eye movements. As part of my dissertation, I recorded neural activity from monkeys making eye movements. We found changes in the activity of single cells and neural oscillations (LFPs) just before the eye movements started, a finding that has been replicated many times and in many brain areas. It's not immediately obvious to me whether these effects can be harnessed to treat PTSD, but there's at least a shred of a mechanism there.
That brings me to my main point: psychology is a huge field, and contains people studying perception, learning, language acquisition, and more. That work is overwhelmingly experimental and data-driven. Clinical psychology is a small part of it–and it's also probably the hardest part. I'd agree that it and psychiatry haven't had the same success that (say) orthopedics or even oncology have had. Some of the practitioners, especially the ones that get a lot of public attention, are certainly overconfident. That doesn't make the field a fraud though.
I'm not sure that's the sort of politics the original comment had in mind, and I'm certain that these things are not exclusive to the social sciences. I've heard that parts of CERN can be very political, and I've personally seen two adult men nearly come to blows over the synaptic organization of visual cortex. Internal politics, differences of opinion, and interpretation are parts of every human endeavor.
EMDR sounds goofy, but there may actually be something there. When Francine Shapiro died last month, the New York Times ran an article about her and her work, which for which they interviewed Richard McNally, who was one of EMDR's fiercest critics in the 1990s and 2000s; at the time, he called it "merely one of the many therapeutic fuzz-balls that litter the landscape of psychology today." In this article, he says “I’ve changed my mind,” Dr. McNally said in a phone interview. “I’m willing to do that based on new evidence. It looks like there’s something going on there; the representation of the trauma seems to be reconsolidated in a way that doesn’t distress people as much when later recalled.” From here: https://www.nytimes.com/2019/07/11/science/francine-shapiro-... I'm not a clinical psychologist and I don't know a ton about EMDR, but I do know a fair bit about eye movements. As part of my dissertation, I recorded neural activity from monkeys making eye movements. We found changes in the activity of single cells and neural oscillations (LFPs) just before the eye movements started, a finding that has been replicated many times and in many brain areas. It's not immediately obvious to me whether these effects can be harnessed to treat PTSD, but there's at least a shred of a mechanism there.
That brings me to my main point: psychology is a huge field, and contains people studying perception, learning, language acquisition, and more. That work is overwhelmingly experimental and data-driven. Clinical psychology is a small part of it–and it's also probably the hardest part. I'd agree that it and psychiatry haven't had the same success that (say) orthopedics or even oncology have had. Some of the practitioners, especially the ones that get a lot of public attention, are certainly overconfident. That doesn't make the field a fraud though.
> That doesn't make the field a fraud though.
No it doesn't. What does make the field fraud is the Rosenhan experiment, and related experiments, which proved empirically:
1) psychiatrists cannot identify mental illness
2) psychiatrists cannot identify absence of mental illness
And we're not talking 1% false positives or false negatives. We're talking 100% false positives, and ~17% false negatives were observed. That is not reasonable. It's not even that it was up to chance, that they didn't know, as that would obviously result in 50% figures. 100% (and anything over 50%) can only be explained by psychiatrists purposefully, willfully misdiagnosing patients. But it gets worse.
3) ALL psychiatrists tested used drugs on these patients that were misdiagnosed. Every last one.
4) Despite learning the truth, 100% were unwilling to change their opinion and instead used physical violence, and the threat of more physical violence against their patients to extract compliance.
This is not just damning. This is consistent, 100% fraud. Nothing else can explain these figures. Needless to say, the experiment has been repeated, verified through different means, even in popular culture and TV shows. Every time the results were equally damning.
These experiments took place in 1887, 1968, 1973, 1988, and 2008. Similar studies exist for youth protection and mental health diagnosis. There has been no improvement over time.
And many studies claim that diagnoses always follow the space available in mental health programmes, whether inpatient or outpatient, in a way that no other medical discipline has ever seen. Many papers point out that it is supply of care, not demand, that determines the amount of mental disorders in the population.
Furthermore, every real illness has systematically been removed from psychiatry into normal medicine. Not one have they been able to hold on to. Examples: epilepsy, narcolepsy, aneurysms, hysteria, herpes, many forms of intoxication and drug use, various forms of hormone imbalances (and quite a few diseases causing them), TONS of disorders that cause pain, ...
There are still "unsolved" real diseases that are treated in psychiatry, only because there is no treatment for them, well known ones are Parkinson and Dementia, Low IQ, Down syndrome and others. Again there are many types of intoxication that fall under this umbrella, which includes abuse of psychiatric medication itself, sometimes by psychiatrists or other psychiatric personnel.
Other psychiatric illnesses turned out to be racism or social biases of psychiatrists and nothing more, like homosexuality, prostitution (still often done, today), feminism and even then we're ignoring "Aktion T4" and a lot more such efforts.
It is not very well known, but the battle in the US for acceptance of homosexuality and to a lesser extent feminism focused on removal of psychiatry from the lives of people.
There are many reasons the field is fraudulent and people working in it know that it is.
No it doesn't. What does make the field fraud is the Rosenhan experiment, and related experiments, which proved empirically:
1) psychiatrists cannot identify mental illness
2) psychiatrists cannot identify absence of mental illness
And we're not talking 1% false positives or false negatives. We're talking 100% false positives, and ~17% false negatives were observed. That is not reasonable. It's not even that it was up to chance, that they didn't know, as that would obviously result in 50% figures. 100% (and anything over 50%) can only be explained by psychiatrists purposefully, willfully misdiagnosing patients. But it gets worse.
3) ALL psychiatrists tested used drugs on these patients that were misdiagnosed. Every last one.
4) Despite learning the truth, 100% were unwilling to change their opinion and instead used physical violence, and the threat of more physical violence against their patients to extract compliance.
This is not just damning. This is consistent, 100% fraud. Nothing else can explain these figures. Needless to say, the experiment has been repeated, verified through different means, even in popular culture and TV shows. Every time the results were equally damning.
These experiments took place in 1887, 1968, 1973, 1988, and 2008. Similar studies exist for youth protection and mental health diagnosis. There has been no improvement over time.
And many studies claim that diagnoses always follow the space available in mental health programmes, whether inpatient or outpatient, in a way that no other medical discipline has ever seen. Many papers point out that it is supply of care, not demand, that determines the amount of mental disorders in the population.
Furthermore, every real illness has systematically been removed from psychiatry into normal medicine. Not one have they been able to hold on to. Examples: epilepsy, narcolepsy, aneurysms, hysteria, herpes, many forms of intoxication and drug use, various forms of hormone imbalances (and quite a few diseases causing them), TONS of disorders that cause pain, ...
There are still "unsolved" real diseases that are treated in psychiatry, only because there is no treatment for them, well known ones are Parkinson and Dementia, Low IQ, Down syndrome and others. Again there are many types of intoxication that fall under this umbrella, which includes abuse of psychiatric medication itself, sometimes by psychiatrists or other psychiatric personnel.
Other psychiatric illnesses turned out to be racism or social biases of psychiatrists and nothing more, like homosexuality, prostitution (still often done, today), feminism and even then we're ignoring "Aktion T4" and a lot more such efforts.
It is not very well known, but the battle in the US for acceptance of homosexuality and to a lesser extent feminism focused on removal of psychiatry from the lives of people.
There are many reasons the field is fraudulent and people working in it know that it is.
If you can't follow the scientific method most of the time, should their work be considered science at all?
This is my concern. If you can only partially trust the science journals because the science is faulty, what “science” can you trust? ... unless you attempt replicate every piece of research yourself?
Are you actually arguing that randomized controlled trials aren't following the scientific method?
Clinical trials are not true repeatable, scientific experiments. They're really just controlled studies which are the best that can be done in the absence of the possibility of a controlled experiment. In psychology, this however is just the tip of an iceberg of problems with such research being unscientific.
I don't follow.
An experiment needs to be repeatable in principle. You don't actually have to re-run it with the exact same specimens or subjects; they can be sampled from some larger population that you're studying (e.g., patients with depression). Nevertheless, you can often do something like a cross-over design that lets you repeatedly test the same patients in different conditions.
An experiment needs to be repeatable in principle. You don't actually have to re-run it with the exact same specimens or subjects; they can be sampled from some larger population that you're studying (e.g., patients with depression). Nevertheless, you can often do something like a cross-over design that lets you repeatedly test the same patients in different conditions.
I recently reviewed a paper from a research team in China, on a scalable and ubiquitous ad hoc sensing platform. The presented "spin" was that it could potentially be useful for monitoring power distribution networks, thereby reducing blackouts, which was certainly true. However, it could also have been used for an inescapable, totalitarian panopticon (this part was implicit and unstated).
As a reviewer with explicit instructions to evaluate the ethical implications of the work, is it my responsibility (A) to interpret the stated "spin", (B) extract and evaluate strictly the technical contributions of the work, or (C) all of the above, and the implicit "anti-spin" interpretation that was provided courtesy of my Western worldview?
And here's a twist - six months previously, a similar situation occurred with my own work, in which a technology that I envisioned (spun) being used for disaster relief caught the eye of an alphabet agency for low-cost, secure data collection in third world theaters. This was deeply shocking to me at the time, because I hadn't considered this potential use case when I was developing the system. I had spun its impact so effectively in my own mind, I didn't even see the negative alternative.
It's my observation that human brains will inject their own spin if it doesn't already exist, using the social and cultural norms to which that brain is accustomed. I don't hold a researcher's desire to sell their work as effectively as possible against them, especially if the person to whom they're selling most effectively is themselves.