Cancer survival rates are misleading(allendowney.com)
allendowney.com
Cancer survival rates are misleading
https://www.allendowney.com/blog/2025/10/27/cancer-survival-rates-are-misleading/
65 comments
> notably randomized controlled trials that compare mortality.
Putting people into a control group so you can observe the effects of not treating them might not make it past the ethics committee.
Putting people into a control group so you can observe the effects of not treating them might not make it past the ethics committee.
You don't not treat the control group. You give the control group the current standard treatment, and you give the experimental group the new trial treatment.
In some cases, the "current standard treatment" is no treatment.
In some studies, although not cancer, I've seen treatment offered to both groups, someone in the control at any time can opt into the same treatment the other group receives if they want to. Some people will make the informed decision to not take the treatment, and they are your control.
No longer randomized though, so it brings in confounding factors. Which one then has to try to control for.
Doesn't really change your point, but I should clarify that what I said was wrong, it was anyone in the study could choose not to take the blinded pill that could be placebo or active, they were removed from the study, and were provided with the active experimental drug + compensation as to not punish them for giving or not giving their consent after being sufficiently informed of the risks of either choice.
Now everyones agree with you and there are no more cases like https://en.wikipedia.org/wiki/Tuskegee_Syphilis_Study (hopefuly).
In the current trials a part of the subjects get the new experimental drug and the control group get the current state of the art treatment.
In the current trials a part of the subjects get the new experimental drug and the control group get the current state of the art treatment.
Yes, but when you compare treatments A vs B for trial 1, and then B vs C for trial 2, and then C, vs D for trail 3, you might not get the same results as comparing A vs D, especially when there may have been other changes in between the three trials (different ages, lifestyles etc).
This is a real problem when the Minister wants to know if it’s worth spending money on treatments, because all you have is a disjointed set of trials, none of which are necessarily representative of the population at large, or the population wide incidence of the disease (assuming there is even data on that (notifiable illnesses are the exception).
This is a real problem when the Minister wants to know if it’s worth spending money on treatments, because all you have is a disjointed set of trials, none of which are necessarily representative of the population at large, or the population wide incidence of the disease (assuming there is even data on that (notifiable illnesses are the exception).
> Yes, but when you compare treatments A vs B for trial 1, and then B vs C for trial 2, and then C, vs D for trail 3, you might not get the same results as comparing A vs D, especially when there may have been other changes in between the three trials (different ages, lifestyles etc).
That's not what happens.
Is this just a hypothetical?
Everything will be compared to one standard of care, or perhaps two which will have been compared to each other. If a new treatment is much better, then it will become standard of care.
Trials cost a lot of money, so they are conducted rationally.
That's not what happens.
Is this just a hypothetical?
Everything will be compared to one standard of care, or perhaps two which will have been compared to each other. If a new treatment is much better, then it will become standard of care.
Trials cost a lot of money, so they are conducted rationally.
That is what happens if C is developed after B becomes the standard treatment, D after C etc.
Suppose D is only slightly less effective than C, but more effective than A, and B, but 100x cheaper, and/or has less bad side effects. If you only compare with C, all you know is it's not as good as C.
Suppose D is only slightly less effective than C, but more effective than A, and B, but 100x cheaper, and/or has less bad side effects. If you only compare with C, all you know is it's not as good as C.
> That is what happens if C is developed after B becomes the standard treatment, D after C etc.
Can you point to particular drugs or are you also making up examples?
I admittedly do not know of every trial that happens everywhere but this is exactly the sort of thing that a layman expects would occur but which does not happen.
Can you point to particular drugs or are you also making up examples?
I admittedly do not know of every trial that happens everywhere but this is exactly the sort of thing that a layman expects would occur but which does not happen.
Stem cell treatments come to mind. Outrageously expensive (or outright unavailable locally), in many areas and for select purposes.
> Stem cell treatments come to mind. Outrageously expensive (or outright unavailable locally), in many areas and for select purposes.
I don't see how stem cells relate to the idea of trials for successive standard-of-care treatments. Can you explain your thinking?
I don't see how stem cells relate to the idea of trials for successive standard-of-care treatments. Can you explain your thinking?
Randomized controlled trials are a bummer of a 'gold standard'. Extremely expensive, extremely slow and in many cases absolutely impossible. I'm not an AI true believer, but I do hope it offers an alternative or at least enables some desperately needed efficiencies.
"Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial"
https://www.bmj.com/content/363/bmj.k5094
But seriously: this is a recognized problem in medicine and there's already a widely used solution. Whenever you're doing trials of an intervention for a condition which already has an accepted treatment, you run a trial to compare your new intervention to that, and see if your test group has better outcomes. After all, the question shouldn't be whether your treatment is effective; it's whether it's better than existing treatments.
Trials against a placebo have a purpose, but they aren't the only way to run a trial.
https://www.bmj.com/content/363/bmj.k5094
But seriously: this is a recognized problem in medicine and there's already a widely used solution. Whenever you're doing trials of an intervention for a condition which already has an accepted treatment, you run a trial to compare your new intervention to that, and see if your test group has better outcomes. After all, the question shouldn't be whether your treatment is effective; it's whether it's better than existing treatments.
Trials against a placebo have a purpose, but they aren't the only way to run a trial.
It also helps to actually go back and look at in terms of "rejecting the null hypothesis".
If you're talking about a treatment for The Common Cold, the null hypothesis is "the subject got better after awhile because people get better after awhile", and you can't disprove that's what's happening without a very rigorous study with a well designed control.
If you're talking about "here's some robot eyes that cure blindness", you don't really need a control group to test if it works. The null hypothesis is they're blind; you just need to demonstrate they can see to disprove the null hypothesis and prove efficacy.
If you're talking about a treatment for The Common Cold, the null hypothesis is "the subject got better after awhile because people get better after awhile", and you can't disprove that's what's happening without a very rigorous study with a well designed control.
If you're talking about "here's some robot eyes that cure blindness", you don't really need a control group to test if it works. The null hypothesis is they're blind; you just need to demonstrate they can see to disprove the null hypothesis and prove efficacy.
Ok, suppose the current standard treatment is A. You have a new treatment B that is 100x cheaper than B, and doesn't trigger allergic reactions that some people have to A. You test against A, and the B group has slightly worse outcomes overall than A. Does that mean that B is useless? What if it is almost as effective, but it's lower cost means more people would be able to use it?
"better" is not a total order, one treatment may be better in some ways and worse in other ways. Especially if you include things like cost and availability.
"better" is not a total order, one treatment may be better in some ways and worse in other ways. Especially if you include things like cost and availability.
Even if that's the case, the metric you're interested in isn't "is Treatment B better than doing nothing"; it's "how effective is Treatment B relative to Treatment A". And the most effective way to determine that will be to trial them against each other, not against a placebo.
Ideally, for getting good information, you compare to both. Because you also want to know if your treatment is more effective than a placebo.
You just discovered the entire scientific field of cost-effectiveness.
Wouldn’t those factors be detected and reported on in the trial?
And years later we may have some useful data, if a study can be conducted ethically in the first place. Meanwhile, the environment around us continues to change at a pace the likes of which humans have never experienced. Whatever this era of LLMs does (or does not) do to improve the situation, I am firmly confident that years-to-decades-long human testing is not the endgame of medical science, but rather a long and inconvenient pitstop. There's a lot of those in the history of medicine.
RCTs aren't always necessary to change the medical standard of care. Sometimes lower quality studies plus a theoretical understanding of the physiologic is good enough.
There has never been an RCT to show that smoking causes lung cancer but doctors now all recommend that their patients not smoke.
There has never been an RCT to show that smoking causes lung cancer but doctors now all recommend that their patients not smoke.
I'm surprised this doesn't really talk about the thing that was most obvious to me: assuming the 5 year survival rate is five years from diagnosis, that means that if a tumor is diagnosed earlier, even if the cancer kills you, your death is more likely to be outside the five year window.
So for example, if you have (hypothetically) an untreatable cancer that would take six years to kill you, if it is diagnosed right away, you would be counted as a survivor, but if you are diagnosed at year five, you'll only survive a year.
So for example, if you have (hypothetically) an untreatable cancer that would take six years to kill you, if it is diagnosed right away, you would be counted as a survivor, but if you are diagnosed at year five, you'll only survive a year.
Tumour evolution and progression is complex, being diagnosed early does not guarantee a linear growth. Even when it's biopsied at a timed interval you can't get a full picture of the cancer (invasive pattern etc) evolution trajectory. In some cases low grade tumours will be put on surveilance without radical treatment.
Diagnosis is complex too, you don't want the test to have low specificity. False positive is sometimes tolerated.
Diagnosis is complex too, you don't want the test to have low specificity. False positive is sometimes tolerated.
Colon cancer is an interesting one, Hank Green [1] recently covered a new paper [2] that showed a massive reduction in colon cancer risk for folks that engaged in moderate, regular, exercise. The authors speculated that mechanical stress leading to increased shedding might play an important role.
Weirdly enough that's the same mechanism hypothesized to play a partial role in why breast feeding is also associated with a reduced cancer risk.
Fascinating, weird, stuff.
1. https://www.youtube.com/watch?v=4RXSX93mvg8
2. https://www.nejm.org/doi/pdf/10.1056/NEJMoa2502760
Weirdly enough that's the same mechanism hypothesized to play a partial role in why breast feeding is also associated with a reduced cancer risk.
Fascinating, weird, stuff.
1. https://www.youtube.com/watch?v=4RXSX93mvg8
2. https://www.nejm.org/doi/pdf/10.1056/NEJMoa2502760
imho if you don't exercise regularly and don't eat clean you're asking for troubles and simply can't complain about your health. The vast majority of people seem not to care until they get a serious diagnostic, by that time you can barely mitigate the issue. It certainly isn't a silver bullet, but at least it stacks the odds in your favor.
But that's haaard. Can't I just do the easy solution instead? Don't you have a pill I can take or something?
Aldous Huxley was correct, we truely are amusing ourselves to death. The new meta glasses are really scaring me.
Sorry for the doomerism. There's lots of other stuff to be optimistic about. Maybe this is just an evolutionary filter. Those that fit into these new circumstances will survive.
Aldous Huxley was correct, we truely are amusing ourselves to death. The new meta glasses are really scaring me.
Sorry for the doomerism. There's lots of other stuff to be optimistic about. Maybe this is just an evolutionary filter. Those that fit into these new circumstances will survive.
We managed to convince my almost 70 years old father in law, he dropped 30+ kilos in two years. He could barely walk without getting out of breath, now he walks 20-30k steps a day, every day. We got rid of his cardiologist who kept telling him to take pills because his symptoms were just due to old age... Now he needs half the pills, and the ones he still has to take are much lower doses.
No idea if it'll have an impact on his lifespan but it definitely bought him years of health and changed his day to day life
No idea if it'll have an impact on his lifespan but it definitely bought him years of health and changed his day to day life
Executive dysfunction is one son of a b**
I mean as long as you don't put systematic barriers in the way of doing the right thing, eg. missing sidewalks and food deserts.
A food desert is defined as food still probably closer to you than a lot of people had in Huxley's day.
The rate of poverty and starvation globally was massively higher in Huxley's day.
Globally yes, but not US-style food deserts.
well india has such low cancers and people there dont exercise much.
There are a lot of weird health statistic anomalies around the world. For example, the 11% of US adults are smokers compared to 17% in Japan and 25% in Spain. But both Japan and Spain have a lower lung-cancer incidence rate than Americans (Spain is much lower than both!)
Is the rate of cancer actually low or are a lot of cases never formally diagnosed? People in India might not "exercise" much in terms of going to the gym or running but they might still be physically active in other ways.
word of mouth spreads really fast in india. if people are dying horrible deaths of a disease, and cancer sufferers seem to die horribly, it would be known. things like tuberculosis and leprosy became well known. AIDS too. but cancer seems like a disease of the first world.
>The purpose of the model is to show that we can reproduce the survival rates we see in reality, even if there are no effective treatments.
That's a great argument in the abstract, but it ignores the fact that there are effective treatments for colon cancer. The fact that we can reproduce real survival rates in a counterfactual world where there are no effective treatments for colon cancer does not actually give us a model of the real world because the counterfactual explicitly contradicts known scientific facts.
What you have to do in order to make this argument is to show that there are Markov models where early detection does not work despite the fact that some cancers will cause death if untreated and not if treated. You cannot simply rely on models that have clearly impossible transition probabilities. You need possible models. Or you have to show that the absolutely massive amount of scientific literature and clinical experience about how to treat colon cancer is somehow flawed.
Some people are defending this because the blog post is attacking a specific argument, but I don't see how that can work. I am pretty sure that Nassim Taleb and most other people who are capable of putting together a coherent statistical argument (even a flawed one) understand that colon cancer can be treated sometimes.
That's a great argument in the abstract, but it ignores the fact that there are effective treatments for colon cancer. The fact that we can reproduce real survival rates in a counterfactual world where there are no effective treatments for colon cancer does not actually give us a model of the real world because the counterfactual explicitly contradicts known scientific facts.
What you have to do in order to make this argument is to show that there are Markov models where early detection does not work despite the fact that some cancers will cause death if untreated and not if treated. You cannot simply rely on models that have clearly impossible transition probabilities. You need possible models. Or you have to show that the absolutely massive amount of scientific literature and clinical experience about how to treat colon cancer is somehow flawed.
Some people are defending this because the blog post is attacking a specific argument, but I don't see how that can work. I am pretty sure that Nassim Taleb and most other people who are capable of putting together a coherent statistical argument (even a flawed one) understand that colon cancer can be treated sometimes.
I think this is a technical article about a narrow aspect of public health policy, not advice to individual patients.
One point in the article is that early detection would give you more years to live even if there were no treatment. Because "early" means "more years". This wasn't obvious to me right away.
But he is not saying don't get screened! He is not saying there are no cancer treatments! He's saying that the 5-year survival rate, considered alone, is a tricky measure that can fool our intuition. In my case he's right.
---------------------
Details.
Dumb toy model. Let Tumor X kill you exactly 8 years after it becomes detectable in screening. Assume screening is 100% accurate with no false positives. Assume X cancer kills you exactly 2 years after it causes symptoms. Imagine that there is no treatment for X cancer.
In this dumb model, everybody dies at exactly the same time after the tumor became detectable. The people who caught it in screening had more warning, but otherwise they didn't get a better outcome. Even though screening boosts the 5-year survival rate from 0% to 100%.
Never mind his like 7-state Markov model. OMG. Why.
One point in the article is that early detection would give you more years to live even if there were no treatment. Because "early" means "more years". This wasn't obvious to me right away.
But he is not saying don't get screened! He is not saying there are no cancer treatments! He's saying that the 5-year survival rate, considered alone, is a tricky measure that can fool our intuition. In my case he's right.
---------------------
Details.
Dumb toy model. Let Tumor X kill you exactly 8 years after it becomes detectable in screening. Assume screening is 100% accurate with no false positives. Assume X cancer kills you exactly 2 years after it causes symptoms. Imagine that there is no treatment for X cancer.
In this dumb model, everybody dies at exactly the same time after the tumor became detectable. The people who caught it in screening had more warning, but otherwise they didn't get a better outcome. Even though screening boosts the 5-year survival rate from 0% to 100%.
Never mind his like 7-state Markov model. OMG. Why.
Because for some types of cancer, the stage of cancer is important. A cancer should reach a late stage to kill the patient.
Of course they're misleading.
What did the doctor tell us when my mom was diagnosed? Don't do research, do not trust Dr. Google, Dr. Google lies. At best, Dr. Google is behind the times.
The specifics of your case will strongly affect what happens to you. And even for cancers that are a guaranteed death sentence, survival has increased significantly in recent years.
The specifics of your case will strongly affect what happens to you. And even for cancers that are a guaranteed death sentence, survival has increased significantly in recent years.
Only life itself carries a "guaranteed death sentence".
You tell that to everybody with a glioblastoma.
Go ahead, tell them the aggressive mass in their brain, the thing that shortened whatever potential life expectancy they had to, at best, single digit years, isn’t a guaranteed death sentence.
They’ll be so comforted by the idea that maybe their already shortened life expectancy will be further reduced by a car accident or some idiot with a knife.
Go ahead, tell them the aggressive mass in their brain, the thing that shortened whatever potential life expectancy they had to, at best, single digit years, isn’t a guaranteed death sentence.
They’ll be so comforted by the idea that maybe their already shortened life expectancy will be further reduced by a car accident or some idiot with a knife.
Perfect example. I myself survived a grade 4 GBMO. "5-year survival rate: 10%, median life expectancy: 18 months." That was 13 years ago next month.
Your idea that hope is going to lead to a car crash? or murder by knife? is one of the most bizarre and ridiculous things I've ever read.
Your idea that hope is going to lead to a car crash? or murder by knife? is one of the most bizarre and ridiculous things I've ever read.
This article is a criticism of reasoning, not health advice or suggestions for cancer screening. Maybe he should put a big warning at the top, rather than explain it throughout. A lot of people seem to be missing it.
We're so used to argument that criticizing logic is taken as criticizing the conclusion.
We're so used to argument that criticizing logic is taken as criticizing the conclusion.
The only thing the author seems to be directly arguing against is speculative full-body MRI scanning, which is already mainstream medical advice, for many of the reasons he offers.
> We're so used to argument that criticizing logic is taken as criticizing the conclusion.
This may be so, but his examples are so poor that one is distracted from any type of subtle claim he would make. They are bad in obvious ways (every cancer patient is staged, but we pretend in the article like staging is ancillary to researching survival rates).
This may be so, but his examples are so poor that one is distracted from any type of subtle claim he would make. They are bad in obvious ways (every cancer patient is staged, but we pretend in the article like staging is ancillary to researching survival rates).
This article, in the world as it exists right now, is wrong about colon cancer. Anyone reading this of a certain age: get that colonoscopy, and those polyps removed. Snip it in the bud. That's the great thing about a colonoscopies - all-in-one screening + treatment.
Evidence: https://www.nejm.org/doi/full/10.1056/NEJMoa1301969
Large prospective cohorts (Nurses’ Health Study + Health Professionals Follow-Up Study) with long follow-up - screening colonoscopy was associated with a 68% lower risk of death from colorectal cancer overall (multivariable HR ≈ 0.32, 95% CI 0.24–0.45) and showed significant reduction for proximal colon mortality as well (HR ≈ 0.47, 95% CI 0.29–0.76).
Evidence: https://www.nejm.org/doi/full/10.1056/NEJMoa1301969
Large prospective cohorts (Nurses’ Health Study + Health Professionals Follow-Up Study) with long follow-up - screening colonoscopy was associated with a 68% lower risk of death from colorectal cancer overall (multivariable HR ≈ 0.32, 95% CI 0.24–0.45) and showed significant reduction for proximal colon mortality as well (HR ≈ 0.47, 95% CI 0.29–0.76).
> For colon cancer, the rates from the SEER data are are 91%, 74%, and 16%.
This is the only claim the article makes directly about colon cancer. Otherwise, it's saying that early detection being beneficial isn't supported by survival rates alone.
This is the only claim the article makes directly about colon cancer. Otherwise, it's saying that early detection being beneficial isn't supported by survival rates alone.
"Otherwise, it's saying that early detection being beneficial isn't supported by survival rates alone."
That claim may be obvious to everybody except me. Anyway it turns out to be true.
That claim may be obvious to everybody except me. Anyway it turns out to be true.
Weirder still that Taleb misses the base rate flaw in the logic of full-body MRI screening and cancer screening, an observation that is pretty up his alley and is kind of a well-known thing in this domain.
> Because it is based on past cases, it doesn’t apply to present cases if (1) the effectiveness of treatment has changed or – often more importantly – (2) diagnostic practices have changed.
This was my key takeaway. In a society organized around statistics, we're struggling through an era where those statistics expire faster everyday, and faster than new data can be generated. I can almost relate to the mindset that devalues "facts" because they're increasingly complicated, rapidly changing and come with too many caveats.
This was my key takeaway. In a society organized around statistics, we're struggling through an era where those statistics expire faster everyday, and faster than new data can be generated. I can almost relate to the mindset that devalues "facts" because they're increasingly complicated, rapidly changing and come with too many caveats.
Near as I can tell, the only valid point the author makes is that since mortality rates increase as cancer progresses stages and since progression through stages takes time, a 5 year mortality rate is not a great metric and it would be better to also have 10 and 15 year mortality rates to determine the degree to which early detection + treatment actually increases life expectancy.
So their entire argument is statistics does not tell the entire story. Didn't we all learn this truth when we learned statistics?
I don't think it's nearly as obvious as that; the same misunderstanding of 5-year survival statistics also biases international health system comparisons --- countries can look like they're really good at treating XYZ cancer, but in fact only be better at detecting it at an earlier stage.
The site, in very elaborate ways, is saying that Stage IV cancer (in the author's words 'tumor has spread to distant organs or lymph nodes') is worse than Stage I (localized) cancer.
I don't think there is any person who is aware of the idea of cancer mortality who would equate 'Stage IV' to lead to 'average' survival.
So maybe the article's only point (which is very obvious, and does not require Markov modeling) is that if you increase the number of people who live a long time in a sample, then the average of that sample will go up.
This feels like someone saw a fact on the internet and didn't try to read about it before writing a blog post.
I don't think there is any person who is aware of the idea of cancer mortality who would equate 'Stage IV' to lead to 'average' survival.
So maybe the article's only point (which is very obvious, and does not require Markov modeling) is that if you increase the number of people who live a long time in a sample, then the average of that sample will go up.
This feels like someone saw a fact on the internet and didn't try to read about it before writing a blog post.
this article is trivial nonsense. of course he's technically correct, but the article contains no useful information and boils down to just saying that people (including doctors) aren't looking at literally only 5 year survival rate charts.
like the colon cancer thing. he talks about how it would only be more effective to catch colon cancer early if you assume we have treatments for it that would work early. but we don't need to just assume blindly. we already know we do have those treatments!
like the colon cancer thing. he talks about how it would only be more effective to catch colon cancer early if you assume we have treatments for it that would work early. but we don't need to just assume blindly. we already know we do have those treatments!
Yeah, not sure what the point here is. Could see someone who reads this and comes away thinking that a cancer screening isn't worth it.
Agreed. I think his blog title, "Probably Overthinking It", is appropriate named.
Essentially every assertion in the article is either an oversimification, cherry picking a random niche situation to highlight, or just flat out factually inaccurate.
Let's take this paragraph for example:
"Catching cancer early is beneficial only if (1) the cancers we catch would otherwise cause disease and death, and (2) we have treatments that prevent those outcomes, and (3) these benefits outweigh the costs of additional screening. This table does not show that any of those things is true."
To address these one by one:
1. Obviously cancer causes disease and death. The same graphic he references makes that abundantly clear. Sure, there might be some rare exceptions (elderly patients with slow growing colon cancer for example), but we're talking about the general population.
2. All cancers have treatment options available in some form (could be chemo, radiation, surgical resection, etc), so this assumption doesn't even make sense to include. Let's assume for a second though that treatments might not be available. Even if that were true, there ARE treatments that can help treat cancer symptoms, and but may not affect the tumor directly. Often these are specific to the specific type of cancer.
3. This assertion is dumb - is the author really trying to argue that providing symptomatic or other relief to a cancer patient isn't a sufficient benefit to warrant additional screening?
I could go on, but you get the point. Some people just like arguing for the sake of arguing I guess.
Essentially every assertion in the article is either an oversimification, cherry picking a random niche situation to highlight, or just flat out factually inaccurate.
Let's take this paragraph for example:
"Catching cancer early is beneficial only if (1) the cancers we catch would otherwise cause disease and death, and (2) we have treatments that prevent those outcomes, and (3) these benefits outweigh the costs of additional screening. This table does not show that any of those things is true."
To address these one by one:
1. Obviously cancer causes disease and death. The same graphic he references makes that abundantly clear. Sure, there might be some rare exceptions (elderly patients with slow growing colon cancer for example), but we're talking about the general population.
2. All cancers have treatment options available in some form (could be chemo, radiation, surgical resection, etc), so this assumption doesn't even make sense to include. Let's assume for a second though that treatments might not be available. Even if that were true, there ARE treatments that can help treat cancer symptoms, and but may not affect the tumor directly. Often these are specific to the specific type of cancer.
3. This assertion is dumb - is the author really trying to argue that providing symptomatic or other relief to a cancer patient isn't a sufficient benefit to warrant additional screening?
I could go on, but you get the point. Some people just like arguing for the sake of arguing I guess.
the economist put out a piece a few months ago providing just that. Specifically it compares overall cancer mortality rates (and more interestingly, mortality rates adjusted for age) and shows that cancer deaths have been dropping.
https://www.economist.com/briefing/2025/07/17/the-world-is-m...
https://archive.is/TNjoi