Kheiron raises $22M for AI that helps radiologists detect cancer earlier(venturebeat.com)
venturebeat.com
Kheiron raises $22M for AI that helps radiologists detect cancer earlier
https://venturebeat.com/2019/09/23/kheiron-raises-22-million-for-machine-learning-that-helps-radiologists-detect-cancer-earlier/
7 comments
Problem with AI in medicine, and in radiology in particular, is an inability to discover things it's not trained on.
The issue with AI systems that I've seen and evaluated, (and I've seen just about all of the best ones), is that they are good at finding what a radiologist would find, with the added benefit of finding a few that a radiologist would miss.
Here's the problem, AI finds a few that "A" radiologist would miss, but multiple radiologists would not miss. What I mean by that is that the nature of the misses are somewhere between 'facepalm' and 'other radiologists give rad that missed pathology a slap upside head'. Essentially, the pathology that the AI found but the rad missed is obvious, and the rad says, "Oh yeah! I should have caught that." The sort of misses that might be caused by fatigue, or flipping through too many studies too quickly.
What you really want is AI that can synthesize a set of studies and catch things that 99.999% of radiologists on the planet would never catch. Essentially, things not reflected in the datasets, and things that have not been discovered. You need that because most delivery facilities already outsource image reading to a centralized radiology reading facility that may be miles away. So, financially, they are already saving the money of not paying radiologists. So any centralized radiology reading facility powered by AI, would need to be better than any centralized radiology reading facility powered by humans.
Despite claims made everywhere from papers at RSNA to marketing materials from Silicon Valley, every AI I've seen has failed this test. In both accuracy, as well as the more esoteric metric of finding things that it's not trained on.
[Just as a matter of full disclosure, I have a research background in medical imaging, and have actually had a startup exit in the field.]
The issue with AI systems that I've seen and evaluated, (and I've seen just about all of the best ones), is that they are good at finding what a radiologist would find, with the added benefit of finding a few that a radiologist would miss.
Here's the problem, AI finds a few that "A" radiologist would miss, but multiple radiologists would not miss. What I mean by that is that the nature of the misses are somewhere between 'facepalm' and 'other radiologists give rad that missed pathology a slap upside head'. Essentially, the pathology that the AI found but the rad missed is obvious, and the rad says, "Oh yeah! I should have caught that." The sort of misses that might be caused by fatigue, or flipping through too many studies too quickly.
What you really want is AI that can synthesize a set of studies and catch things that 99.999% of radiologists on the planet would never catch. Essentially, things not reflected in the datasets, and things that have not been discovered. You need that because most delivery facilities already outsource image reading to a centralized radiology reading facility that may be miles away. So, financially, they are already saving the money of not paying radiologists. So any centralized radiology reading facility powered by AI, would need to be better than any centralized radiology reading facility powered by humans.
Despite claims made everywhere from papers at RSNA to marketing materials from Silicon Valley, every AI I've seen has failed this test. In both accuracy, as well as the more esoteric metric of finding things that it's not trained on.
[Just as a matter of full disclosure, I have a research background in medical imaging, and have actually had a startup exit in the field.]
An AI that reliably catches stuff that humans sometimes miss is still an advancement in the state of the art.
I have a friend who was living with Multiple Sclerosis for two decades until they moved to a new neighbourhood and started looking for a new specialist. The new MS specialist looked at her history for thirty seconds and said, “you do not have Multiple Sclerosis.”
It turns out that about two decades ago she got married to a chilli-lover, and it was the newly introduced chillies in her diet that were causing inflammation of basically everything inside her body. About a year after the last consumption of chillies she’s basically normal. As much as you can be after two decades of restricted mobility caused by inflamed tissues.
AI only has to be reliably better than the worst humans can be to be incredibly useful.
“Here’s this morning’s batch of radiology that has been pre-vetted by the computer. Some interesting results on case 34, and the computer needs advice on 56 and 80.”
There’s also the issue that you point out that we are training AI to find specific deviations from nominal, rather than teaching the AI to detect anything that deviates sufficiently from nominal to be of medical concern.
I have a friend who was living with Multiple Sclerosis for two decades until they moved to a new neighbourhood and started looking for a new specialist. The new MS specialist looked at her history for thirty seconds and said, “you do not have Multiple Sclerosis.”
It turns out that about two decades ago she got married to a chilli-lover, and it was the newly introduced chillies in her diet that were causing inflammation of basically everything inside her body. About a year after the last consumption of chillies she’s basically normal. As much as you can be after two decades of restricted mobility caused by inflamed tissues.
AI only has to be reliably better than the worst humans can be to be incredibly useful.
“Here’s this morning’s batch of radiology that has been pre-vetted by the computer. Some interesting results on case 34, and the computer needs advice on 56 and 80.”
There’s also the issue that you point out that we are training AI to find specific deviations from nominal, rather than teaching the AI to detect anything that deviates sufficiently from nominal to be of medical concern.
If you don't mind me asking, what did your startup to achieve an exit? Curious, since you're arguing that AI outdoing one radiologist but not an ensemble of radiologists does not create sufficient business value.
You just need a robot that's 80% as good as a radiologist. That means you don't have to pay the bottom 80% of radiologists anymore.
I don't think it's about costs here. No hospital in their right mind would trust an AI completely to not have to pay for a radiologist. It's more about augmenting their work.
// problem.. is an inability to discover things it's not trained on.
What happens if we take all the hard, weird cases, from across the country, diagnosed by the best doctors , and train a system on that ?
Would that system will be better than the average set of 2 radiologists who are what an average patient get ?
What happens if we take all the hard, weird cases, from across the country, diagnosed by the best doctors , and train a system on that ?
Would that system will be better than the average set of 2 radiologists who are what an average patient get ?
I'm clearly contrasting these news with all the other stuff I hear about personal assistants and social media.