How Well Do We Understand Probabilities in Medicine? (2016)(psychscenehub.com)
psychscenehub.com
How Well Do We Understand Probabilities in Medicine? (2016)
https://psychscenehub.com/psychinsights/well-understand-probabilities-medicine/
22 comments
I've never once in my life been able to give real informed consent to a physician, because I've never had one able to inform me quantitatively of the risks and benefits of accepting or refusing treatment. So like, I assume, everyone else I pretty much just have to take what the doctor says on blind faith. Since the vast majority of physicians are good people who actually want to help their patients, I'm not entirely uncomfortable with this, but it's still troubling given how many people die every year as a result of interventions or complications thereof.
I too wish everything in medicine was cut and dry. But oftentimes precise numbers are unavailable. More qualitative reasoning such as "high risk," "serious morbidity or mortality" is usually given in medical assessments. Plans are developed based off that assessment. Doctors and patients routinely make well-considered decisions and get consent knowing that exact numbers aren't known. On the other hand, many treatments (pharmaceuticals and screening being 2 major ones) are backed by very reliable studies where numbers are known.
Yet the point is that even when the numbers are absolutely clear cut and are more than relevant to medical decision making, doctors' training doesn't help them reason their way out of a wet paper bag. They're hopeless even when they're provided with a problem that has the equivalent of training wheels and handrails.
BSEE and MSEE with 8 years of heavy math and the fact that I still have to read the solution slowly and not really get it scares the bejeezers out of me.
If you have any kind of test that seems relatively accurate (5% failure rate), 1 in every 20 tests will be identified as a positive candidate no matter the actual incidence rate. When applied over a huge population the number of positive candidates is going to also be huge. Since the incidence rate in this scenario is so low (1/1000) the odds of a true positive selected from the positive candidates is also very low.
This is a little surprising to me. I was required to take a probability class for my BS in electrical engineering, and a version of this problem was a homework question.
Funny you should call this out: As an undergrad I got to choose between a prob & statistics path and a complex math / advance calc path. I took the complex math path. However, I was required to take a semester on engineering methods that touched on statistics, but that was part of one semester. As a grad student, I went down the numerical methods path for parallel computing. Not as many classes as undergrad, but still.
College: RPI. Graduation year: '91.
College: RPI. Graduation year: '91.
Bit annoying that this article hat-tips Taleb, when Gerd Gigerenzer has been doing this better and for longer.
This book is a good introduction, includes a lot of anecdote from different health care professionals across a range of different testing (including cancer and HIV), and mentions the research across different health care professionals.
https://www.amazon.com/Reckoning-Risk-Learning-Live-Uncertai...
Here's an extract: https://imgur.com/zO4zkl4
This book is a good introduction, includes a lot of anecdote from different health care professionals across a range of different testing (including cancer and HIV), and mentions the research across different health care professionals.
https://www.amazon.com/Reckoning-Risk-Learning-Live-Uncertai...
Here's an extract: https://imgur.com/zO4zkl4
This topic comes up here periodically, and I've linked to these articles before, but it's probably worth doing so again for those interested. I am a physician who was also puzzled by these matters, and wound up inventing a simple diagram that I use as a mental "widget" to manipulate the ideas of the 2x2 table in my head. The full text articles can be found at: https://www.researchgate.net/profile/Kevin_Johnson40
P.S. I'd love to have someone animate this in Javascript!
P.S. I'd love to have someone animate this in Javascript!
The code for an R version of this is on Github at https://github.com/two-by-two
There is a limited demo there as well.
There is a limited demo there as well.
I always wonder the pros/cons of getting tested for no specific symptom... anxiety.stress.costs of false positives could be enormous.
There's even a book & several articles out there arguing that getting tested for cancer might be a bad idea: https://www.theatlantic.com/health/archive/2015/06/should-I-...
There's even a book & several articles out there arguing that getting tested for cancer might be a bad idea: https://www.theatlantic.com/health/archive/2015/06/should-I-...
Science, Statistics and Lies https://en.wikipedia.org/wiki/Lies,_damned_lies,_and_statist...
Sadly unsurprising. I remember reading a conversation on Twitter where a psychologist wrote openly "I don't use statistics because I have an analytical orientation".
[deleted]
I find it ironic that this article is posted on a website aimed at the mental health industry. The actual challenge faced by mental health practitioners is with the futility of their approach.
I met a young man (early 20's) who'd recently arrived in town for the purposes of "recovery". Having learned something about people's problems with self-medicating I asked, "People usually know when their problems with substances started. When did it start for you?"
He instantly said that when he was in third grade his teacher thought he was disruptive. Parents took him to the doctor, who prescribed medication. When he got to 9th grade he gave his parents an ultimatum: "if you don't take me off these drugs, I'm going to kill myself." His parents promptly took him off the drugs that didn't address the boredom he'd experienced in his suffocating 3rd grade classroom, and that's when he started self-medicating with whatever he could get his hands on.
The DSM is the mental health industry's guide to help practitioners precisely diagnose symptoms. The problem is the industry frequently jumps to treatment without concern as to the cause of the patients' symptoms.
Court-ordered treatment (aka "assisted outpatient treatment") is where the mental health industry decides to force a patient to take the drugs they think they need. I have the affidavits that were filed against my friend. They say, essentially, "Patient expects us to believe that her symptoms are related to consuming 2 bottles of liquor a day. She is clearly in need of court-ordered treatment because she does not believe that she has a chronic condition. We know that she has a chronic condition because she's had two previous orders for treatment. She took herself off our forced services 4 years ago, and here she is again."
Before this went down, I'd decided that my friend's substance abuse problems were related to her having been adopted, exposed to meth amphetamine as a pre-teen, having "lost her future" at 16 years old when she was expelled for self-medicating with what is now approved as a breakthrough therapy for PTSD (MDMA), and having been injected with a prescription endocrine disruptor [0] with the black-box warning of "cortisol deficiency" at 18 or 19 years old. This "birth control" drug is known to make some women suicidally-depressed. Cortisol deficiency is now associated with psychosis. My friend allowed that maybe the injections took her from "drug abuser" to "drug addict".
[0] https://en.wikipedia.org/wiki/Depo-Provera
In the world where mental health professionals address causes rather than symptoms and treat conservatively when the causes of a condition are unknown, those doctors would have recognized my friend's presentation as a form of substance-induced psychosis [1], and provided support to help her sober up.
[1] https://en.wikipedia.org/wiki/Substance-induced_psychosis
Rather than provide rational treatment, they just force her to take "anti-psychotics" in the delusional belief that their drugs (rather than sobriety) are what allows my friend to be functional. At one point they thought she needed two different drugs. My friend explained her presentation as, "I'm sorry, these drugs make us slow".
Every medical specialty has practices that aren't actually justified by the findings of science. I believe Psychiatry is the most important of the medical specialties, but the path "mainstream" practitioners has taken since the 1950's has been a mistake: most psychiatric drugs are just modern FDA-approved "patent medicines" [2] that don't actually address the causes of the patient's symptoms. While some people like their psych drugs, they'd probably do much better with more scientific approach to their complaints.
[2] https://en.wikipedia.org/wiki/Patent_medicine
I met a young man (early 20's) who'd recently arrived in town for the purposes of "recovery". Having learned something about people's problems with self-medicating I asked, "People usually know when their problems with substances started. When did it start for you?"
He instantly said that when he was in third grade his teacher thought he was disruptive. Parents took him to the doctor, who prescribed medication. When he got to 9th grade he gave his parents an ultimatum: "if you don't take me off these drugs, I'm going to kill myself." His parents promptly took him off the drugs that didn't address the boredom he'd experienced in his suffocating 3rd grade classroom, and that's when he started self-medicating with whatever he could get his hands on.
The DSM is the mental health industry's guide to help practitioners precisely diagnose symptoms. The problem is the industry frequently jumps to treatment without concern as to the cause of the patients' symptoms.
Court-ordered treatment (aka "assisted outpatient treatment") is where the mental health industry decides to force a patient to take the drugs they think they need. I have the affidavits that were filed against my friend. They say, essentially, "Patient expects us to believe that her symptoms are related to consuming 2 bottles of liquor a day. She is clearly in need of court-ordered treatment because she does not believe that she has a chronic condition. We know that she has a chronic condition because she's had two previous orders for treatment. She took herself off our forced services 4 years ago, and here she is again."
Before this went down, I'd decided that my friend's substance abuse problems were related to her having been adopted, exposed to meth amphetamine as a pre-teen, having "lost her future" at 16 years old when she was expelled for self-medicating with what is now approved as a breakthrough therapy for PTSD (MDMA), and having been injected with a prescription endocrine disruptor [0] with the black-box warning of "cortisol deficiency" at 18 or 19 years old. This "birth control" drug is known to make some women suicidally-depressed. Cortisol deficiency is now associated with psychosis. My friend allowed that maybe the injections took her from "drug abuser" to "drug addict".
[0] https://en.wikipedia.org/wiki/Depo-Provera
In the world where mental health professionals address causes rather than symptoms and treat conservatively when the causes of a condition are unknown, those doctors would have recognized my friend's presentation as a form of substance-induced psychosis [1], and provided support to help her sober up.
[1] https://en.wikipedia.org/wiki/Substance-induced_psychosis
Rather than provide rational treatment, they just force her to take "anti-psychotics" in the delusional belief that their drugs (rather than sobriety) are what allows my friend to be functional. At one point they thought she needed two different drugs. My friend explained her presentation as, "I'm sorry, these drugs make us slow".
Every medical specialty has practices that aren't actually justified by the findings of science. I believe Psychiatry is the most important of the medical specialties, but the path "mainstream" practitioners has taken since the 1950's has been a mistake: most psychiatric drugs are just modern FDA-approved "patent medicines" [2] that don't actually address the causes of the patient's symptoms. While some people like their psych drugs, they'd probably do much better with more scientific approach to their complaints.
[2] https://en.wikipedia.org/wiki/Patent_medicine
Rather than provide rational treatment, they just force her to take "anti-psychotics" in the delusional belief that their drugs (rather than sobriety) are what allows my friend to be functional.
My best friend's father is a paranoid schizophrenic. Antipsychotic medication allows him to live a relatively normal life.
My best friend's father is a paranoid schizophrenic. Antipsychotic medication allows him to live a relatively normal life.
> My best friend's father is a paranoid schizophrenic. Antipsychotic medication allows him to live a relatively normal life.
How do you know that it's the medication that allows him to "live a relatively normal life", and not the other factors?
If you want to trade anecdotes... My other friend's father fell out of a tree and broke his back when she was a child (iirc). Some time later they decided he was a "schizophrenic". He's been on antipsychotics ever since. His condition has spiraled downward over the subsequent years. Last I heard he was full-on crazy. If the doctors had provided supportive treatment instead of suppressive, his life trajectory would have been totally different.
My aunt's good friend had a "psychotic break" soon after her husband died unexpectedly. The friend was put on "anti-psychotics". She's now dying of liver failure, certainly as a consequence of her long-term drugging.
The ugly truth about so-called "antipsychotic medication" is that the class is palliative rather than curative, and actually causes the deterioration it supposedly treats.
Robert Whitaker makes the case that medications turns people's "episodes" into chronic conditions: https://www.madinamerica.com/2016/07/the-case-against-antips... and https://www.madinamerica.com/2018/03/a-tale-of-two-studies/ and https://www.madinamerica.com/2017/09/thou-shall-not-criticiz... , for example.
How do you know that it's the medication that allows him to "live a relatively normal life", and not the other factors?
If you want to trade anecdotes... My other friend's father fell out of a tree and broke his back when she was a child (iirc). Some time later they decided he was a "schizophrenic". He's been on antipsychotics ever since. His condition has spiraled downward over the subsequent years. Last I heard he was full-on crazy. If the doctors had provided supportive treatment instead of suppressive, his life trajectory would have been totally different.
My aunt's good friend had a "psychotic break" soon after her husband died unexpectedly. The friend was put on "anti-psychotics". She's now dying of liver failure, certainly as a consequence of her long-term drugging.
The ugly truth about so-called "antipsychotic medication" is that the class is palliative rather than curative, and actually causes the deterioration it supposedly treats.
Robert Whitaker makes the case that medications turns people's "episodes" into chronic conditions: https://www.madinamerica.com/2016/07/the-case-against-antips... and https://www.madinamerica.com/2018/03/a-tale-of-two-studies/ and https://www.madinamerica.com/2017/09/thou-shall-not-criticiz... , for example.
I dislike exchanging anecdotes too. If you can point me to a treatment for schizophrenia tested in quality randomized trials with the success rate of antipsychotics (which still have a very poor success rate), please do.
Mad in America makes reference to an approach to psychosis from Finland called "open dialogue". http://psychrights.org/Research/Digest/Effective/OpenDialogu...
It was recently re-established that psychosis is linked to cortisol deficiency: https://psychcentral.com/news/2016/06/04/low-morning-cortiso... - the first such linkage was in the 1950's, but this observation was never put to mainstream use.
Psychosis is fundamentally a condition of metabolic stress. Vitamin B-3 (niacin/niacinamide) was found to help patients recover: https://riordanclinic.org/2014/11/niacin-and-schizophrenia-h... http://orthomolecular.org/resources/omns/v13n23.shtml http://orthomolecular.org/resources/omns/v10n18.shtml
> If you can point me to a treatment for schizophrenia tested in quality randomized trials with the success rate of antipsychotics (which is really miserable), please do.
It will be 15-20 years for a randomized trial initiated today to get such a result. Who would pay for such studies? The businesses who've been making $billions on their FDA-approved drugs for decades? The status quo defends itself, and refuses to admit they made a mistake in forcing palliative treatments on vulnerable patients.
It was recently re-established that psychosis is linked to cortisol deficiency: https://psychcentral.com/news/2016/06/04/low-morning-cortiso... - the first such linkage was in the 1950's, but this observation was never put to mainstream use.
Psychosis is fundamentally a condition of metabolic stress. Vitamin B-3 (niacin/niacinamide) was found to help patients recover: https://riordanclinic.org/2014/11/niacin-and-schizophrenia-h... http://orthomolecular.org/resources/omns/v13n23.shtml http://orthomolecular.org/resources/omns/v10n18.shtml
> If you can point me to a treatment for schizophrenia tested in quality randomized trials with the success rate of antipsychotics (which is really miserable), please do.
It will be 15-20 years for a randomized trial initiated today to get such a result. Who would pay for such studies? The businesses who've been making $billions on their FDA-approved drugs for decades? The status quo defends itself, and refuses to admit they made a mistake in forcing palliative treatments on vulnerable patients.
As I'm sure you're aware, there's a thing called the Dunning-Kruger effect, which predicts that people with less knowledge are more confident in their grasp of the situation than people who really do know what's going on. Reading your comments makes me wonder if perhaps you might want to reflect on whether you've seen 10,000 psychiatric patients on the in-patient wards. If not, I'm guessing you may want to revisit the possibility that the thousands of psychiatrists and nurses on the floors, who have invested their lives in caring for the sick might have a better appreciation for the signal-to-noise ratios that matter in the present topic of discussion.
It seems you consider yourself wise. If that's true, I suspect you will recognize the value of at least projecting humility in the face of the cummulative lifetimes of effort others have invested in a problem.
It seems you consider yourself wise. If that's true, I suspect you will recognize the value of at least projecting humility in the face of the cummulative lifetimes of effort others have invested in a problem.
The opposite side of Dunning-Kruger's coin is that "competent students tended to underestimate their own competence" [0]. I have no idea where I fall on the spectrum, I just try to be observant. I read books, and check references when it seems important. The science library at the nearest State University is quite useful for this purpose. I'm working on a model to help me understand what's going on. There are lots of holes in my model, and I'd certainly take myself straight to the hospital if I sustained a sudden injury (broken bone, etc).
[0] https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect#...
I have videos from 2 weeks before my friend ran out of alcohol that show she was not psychotic when on her maintenance dose of alcohol. She was with me the weekend her mother said she "disappeared" -- I'd encouraged her to call her mother, but she did not do so. She ran out of alcohol when I took her home. She called me the next morning. I knew something was happening, but I didn't know anything about "psychosis" at the time. If I'd known then what I now know about withdrawal psychosis, I'd have called her mother right away...
I have the affidavits from the hospital's application for court-ordered evaluation, and the psychiatrists' affidavits from her first prosecution for having a "persistent or acute disability" (ambiguity in statute), even though the statutes distinguish between "mental disorders" and conditions resulting from "drug abuse, alcoholism or intellectual disability." I witnessed her deterioration while getting the finest of the mental health industry's standard of care. I drove five hours to retrieve my friend the day after she was released from a "stabilization" that left her more delusional at the end of the week than at the beginning.
There is a resistance against conventional psychiatry because of the field's long history of harmful treatment: Lobotomies, water torture, palliative medications, [edit: electrocution], etc. Robert Whitaker examined the predicament of the "mentally ill" in his first book, Mad in America: https://books.google.com/books?isbn=0786723793
The tragedy of the situation is that the Quakers figured out how to compassionately care for people while they had mental episodes (Ref: Whitaker's book). This was without the benefit of the sciences' contributions to humanity's understanding of our condition. Furthermore, 20th Century Science figured out the physiological considerations behind most so-called "mental" conditions. For some reason, psychiatry didn't incorporate these findings into clinical practice. Whitaker makes the case that psychiatry has been compromised by profiteers in the drug industry in his third book, Psychiatry Under the Influence: https://books.google.com/books?isbn=113751602X
People have rights, even people who are declared to have a "mental disorder", but most such patients have trouble asserting their rights on their own. I had a breakthrough last month, and am now confident that my last 3 years of efforts have not been in vain.
Thank you for reminding me of the importance of humility.
[0] https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect#...
I have videos from 2 weeks before my friend ran out of alcohol that show she was not psychotic when on her maintenance dose of alcohol. She was with me the weekend her mother said she "disappeared" -- I'd encouraged her to call her mother, but she did not do so. She ran out of alcohol when I took her home. She called me the next morning. I knew something was happening, but I didn't know anything about "psychosis" at the time. If I'd known then what I now know about withdrawal psychosis, I'd have called her mother right away...
I have the affidavits from the hospital's application for court-ordered evaluation, and the psychiatrists' affidavits from her first prosecution for having a "persistent or acute disability" (ambiguity in statute), even though the statutes distinguish between "mental disorders" and conditions resulting from "drug abuse, alcoholism or intellectual disability." I witnessed her deterioration while getting the finest of the mental health industry's standard of care. I drove five hours to retrieve my friend the day after she was released from a "stabilization" that left her more delusional at the end of the week than at the beginning.
There is a resistance against conventional psychiatry because of the field's long history of harmful treatment: Lobotomies, water torture, palliative medications, [edit: electrocution], etc. Robert Whitaker examined the predicament of the "mentally ill" in his first book, Mad in America: https://books.google.com/books?isbn=0786723793
The tragedy of the situation is that the Quakers figured out how to compassionately care for people while they had mental episodes (Ref: Whitaker's book). This was without the benefit of the sciences' contributions to humanity's understanding of our condition. Furthermore, 20th Century Science figured out the physiological considerations behind most so-called "mental" conditions. For some reason, psychiatry didn't incorporate these findings into clinical practice. Whitaker makes the case that psychiatry has been compromised by profiteers in the drug industry in his third book, Psychiatry Under the Influence: https://books.google.com/books?isbn=113751602X
People have rights, even people who are declared to have a "mental disorder", but most such patients have trouble asserting their rights on their own. I had a breakthrough last month, and am now confident that my last 3 years of efforts have not been in vain.
Thank you for reminding me of the importance of humility.
Clearly, physicians here failed to calculate an exact positive predictive value in the example. The question is does that inability affect their 1) medical care delivered and 2) communication with the patient and the patient's own informed decision-making.
In speaking to 1-- there are many examples to choose from from probably any medical specialty but let's stick to breast cancer screening since that's the example from the article. USPSTF presents their recommendations[1]. I'd encourage anyone with interest to at least skim the rationale presented on the page below those recommendations. They very well consider prevalence as well as efficacy of specific tests given the presence of different risk factors in a patient (age, family history, etc). Importantly, those and many other screening guidelines are applied by primary care physicians who may not otherwise be able to calculate exacting probabilities.
[1] https://www.uspreventiveservicestaskforce.org/Page/Document/...
In speaking to 2-- patient autonomy of course requires an appropriate understanding of the tests they receive, any risk to those tests and the benefits and harms of true positives, false negatives, etc. The associated frequencies, albeit with some degree of imprecision are avaialble by reference and I'd suspect they're memorized by most radiologists reading mammograms and the breast surgeons involved in tested positives-- even if they may not be able to calculate them. If the doctor doesn't have them memorized-- they should be available by referencing the relevant guideline.