Human urine sensing and deep learning-assisted cancer screening(sciencedirect.com)
sciencedirect.com
Human urine sensing and deep learning-assisted cancer screening
https://www.sciencedirect.com/science/article/abs/pii/S0956566323000180
57 comments
Can't access the full text, but I'm curious to see the experimental design, statistical analysis, and what "near-perfect" means. Too many breathless deep-learning-in-diagnosis headlines turn out to be great big false positive machines or random result generators [0]
[0] https://www.statnews.com/2019/10/23/advancing-ai-health-care...
[0] https://www.statnews.com/2019/10/23/advancing-ai-health-care...
In a parallel universe, we would have leveraged dogs sense of smell and developed a better way to communicate w animals. Then everyone would be able to have a cancer detector in their home without enriching big Pharma
Worked in a lab that did R&Din traceability of nefarious substances for a US three-letter agency (I couldn't touch those projects since I wasn't a citizen). And they made it very clear at the time that the best detector in airports for explosives and drugs were, by far, the dogs. No tech that was even in the pipeline at the time (~2010) could come close and glancing at detection limits of current scanners they're not there yet.
So yes I would love to see clinical trials of disease-sniffing canines
So yes I would love to see clinical trials of disease-sniffing canines
> The dataset was randomly divided into 70% training subset (1420 for prostate and 692 for pancreatic), 15% validation subset (304 and 148), and 15% test subset (304 and 148). The batch training task was performed based on conjugate gradient model for 100 epochs with a learning rate of 1e−5. The RNN deep learning model trained for discrimination of normal and prostate cancer urines showed the best validation performance at 93 epochs (Figs. S12a–c). As shown in Fig. S12b, using the total of 304 human urine test datasets, our RNN binary classification model achieved a 0.9997 AUC from the receiver operating characteristic (ROC) curve and a sensing performance with a sensitivity of 99.4%, a specificity of 100%, and an accuracy of 99.7% (confusion matrix). In the same manner, the trained deep learning for pancreatic cancer urine dataset (Figs. S13a–c) clearly showed superior classification performance with a sensitivity of 98.6% and a specificity of 100% (99.3% accuracy, 0.9892 AUC, 59 epochs). The false discovery rate (FDR) of RNN-assisted binary classification was 0%, 0.7%, and 1.3% for normal, prostate cancer, and pancreatic cancer, respectively. In CNN-assisted three-class classification, the dataset was randomly divided into 70% training subset (1,557) and 30% test subset (666). The batch training task was performed based on conjugate gradient model for 100 epochs with single conventional layer. To find more general diagnostic model for unknown urine sample analysis, multi-class classification method was applied on normal, prostate and pancreatic cancer urine based on CNN deep leaning model. The CNN deep learning model trained for normal and cancer type classification showed excellent performance with <1% loss and 100% accuracy (Fig. S14) after 33 epochs. Our CNN multi-class classification model achieved a sensing performance with a sensitivity of 96.8%, a specificity of 99.6% (prostate) and 97.7% (pancreatic), and an accuracy of 96.8% (confusion matrix) from test dataset. The FDR of CNN-assisted multi-class classification was 4.4%, 1.8%, and 3.6% for normal, prostate cancer, and pancreatic cancer, respectively. Therefore, our 3D-PCNUS platform supported by a fast deep learning bioclassification method successfully distinguishes cancer urine from normal urine with high clinical sensitivity and specificity, suggesting the use of the developed system for on-site cancer diagnostic technology. The developed system is suitable for patient screening with higher pathogenic probability.
Interesting. Withings announced a reusable urine analyzer for your toilet bowl at CES, might be cool to see that upgraded with these tests.
Neat: https://www.youtube.com/watch?v=uyo4hyM3SXo
Hopefully something like this comes to the US at some point.
Subscribed to their newsletter thing here:
https://www.withings.com/us/en/u-scan
> You're in! Thanks for signing up.
The number of pee jokes I've experienced in the past 20 minutes of looking into this thing have been delightful :-D
Hopefully something like this comes to the US at some point.
Subscribed to their newsletter thing here:
https://www.withings.com/us/en/u-scan
> You're in! Thanks for signing up.
The number of pee jokes I've experienced in the past 20 minutes of looking into this thing have been delightful :-D
> Withings announced a reusable urine analyzer for your toilet bowl at CES
Doing several tests a day on someone could create a lot of false positives!
Perhaps if the sensor detected possible symptoms as well: urinating more frequently in the night, difficulty in starting, weak or interrupted flow, blood in the urine etc.
Doing several tests a day on someone could create a lot of false positives!
Perhaps if the sensor detected possible symptoms as well: urinating more frequently in the night, difficulty in starting, weak or interrupted flow, blood in the urine etc.
> Doing several tests a day on someone could create a lot of false positives!
Only if they don't adjust the threshold/filter, properly calibrated you'd expect false positives to be reduced.
Only if they don't adjust the threshold/filter, properly calibrated you'd expect false positives to be reduced.
The submission title has taken the liberty of heavily editorializing.
This is a submission to a minor journal, of a complicated material, attempting to do something that is clinically very difficult.
I'm not trying to dismiss this research, but please don't expect this to ever be clinically available.
This is a submission to a minor journal, of a complicated material, attempting to do something that is clinically very difficult.
I'm not trying to dismiss this research, but please don't expect this to ever be clinically available.
We've changed the title. Submitted title was "Urine Test Detects Prostate and Pancreatic Cancers with Near-Perfect Accuracy".
Submitters: "Please use the original title, unless it is misleading or linkbait; don't editorialize." https://news.ycombinator.com/newsguidelines.html
If the original title won't fit HN's 80 char limit, it can usually be shortened, as we did here.
Submitters: "Please use the original title, unless it is misleading or linkbait; don't editorialize." https://news.ycombinator.com/newsguidelines.html
If the original title won't fit HN's 80 char limit, it can usually be shortened, as we did here.
From the paper and for contextualisation:
> In the same manner, the trained deep learning for pancreatic cancer urine dataset (Figs. S13a–c) clearly showed superior classification performance with a sensitivity of 98.6% and a specificity of 100% (99.3% accuracy, 0.9892 AUC, 59 epochs).
> In the same manner, the trained deep learning for pancreatic cancer urine dataset (Figs. S13a–c) clearly showed superior classification performance with a sensitivity of 98.6% and a specificity of 100% (99.3% accuracy, 0.9892 AUC, 59 epochs).
Seems strange to me that they put the epoch number there. How many people did they test?
My suspicion in reading the abstract and intro is that this is a translation issue. Inference informed by:
1) All the authors have South Korean institutional affiliations
2) "Detection of human biofluids such as blood, tears, saliva, sweat, and urine is important for clinical analysis of various physiological patterns" (First sentence, seems to be missing a word)
3) "differentiate patients from the normal group with high sensitivity and specificity" (abstract, patients and normal group is an odd way of phrasing this)
That being said...it had never occurred to me that nueral networks might be a useful way of interpreting spectroscopy data, that is a really cool insight
1) All the authors have South Korean institutional affiliations
2) "Detection of human biofluids such as blood, tears, saliva, sweat, and urine is important for clinical analysis of various physiological patterns" (First sentence, seems to be missing a word)
3) "differentiate patients from the normal group with high sensitivity and specificity" (abstract, patients and normal group is an odd way of phrasing this)
That being said...it had never occurred to me that nueral networks might be a useful way of interpreting spectroscopy data, that is a really cool insight
Neural nets operating on mel sprectograms (spectrograms shifted to bias frequencies that are of interest to humans) have remarkable ability for audio classification and synthesis. It stands to reason it would not be difficult to adapt methods towards nearly anything which can be mapped to an image in a similar way. I don’t think its particularly novel in concept and kind of surprised it isn’t significantly further along.
I think we are at the technology level to really make medicine significantly more affordable and available, at the cost of many high paying doctor’s jobs. Same can be said for lawyers. These are powerful groups that will not take being automated lightly.
I think we are at the technology level to really make medicine significantly more affordable and available, at the cost of many high paying doctor’s jobs. Same can be said for lawyers. These are powerful groups that will not take being automated lightly.
Also from the paper for more context:
> The dataset was randomly divided into 70% training subset (1420 for prostate and 692 for pancreatic), 15% validation subset (304 and 148), and 15% test subset (304 and 148).
> The dataset was randomly divided into 70% training subset (1420 for prostate and 692 for pancreatic), 15% validation subset (304 and 148), and 15% test subset (304 and 148).
I think maybe the most interesting thing about it from my perspective is that deep learning is being integrated into this kind of technology. For all of the corporate immorality of Google and Facebook, the value to society of Tensorflow and Torch being free software cannot be overstated.
Is it a minor journal? They claim impact factor over 12. This is, of course, field related, but it's not nothing.
If I understand well the WP article about 'impact factor', it tells that in 2016 Nature was graded at 38 and Plos One at 3, so 'impact factor' might not mean too much.
And the criticism section of the WP article is quite long:
https://en.wikipedia.org/wiki/Impact_factor
And the criticism section of the WP article is quite long:
https://en.wikipedia.org/wiki/Impact_factor
The devil is in the details. Near perfect should be weighted against prevalence….
I agree that the HN title is editorialised, but can we talk about the actual paper? The paper’s title and abstract aren’t unreasonable, why is it that it will never be clinically viable?
Please consider that the researchers reported a 99% detection rate of those particular cancers in normal people, using a urine test. One would be hard-pressed to come up with a less editorialized title, given HN's constraints.
I didn't read the paper, but in case you did: is it "near perfect accuracy", or is it 99%? The base rate of pancreatic cancer is very low, so a test with 99% specificity is going to overwhelmingly generate false positives.
Since most people find probability pretty unintuitive, I figured I’d recommend an article that covers the math involved:
https://www.untrammeledmind.com/2018/01/cancer-screening-an-...
https://www.untrammeledmind.com/2018/01/cancer-screening-an-...
Indeed, Reverend Bayes’ dark side rears its ugly head right at the point that people claim that two nines of accuracy is good in the context of detecting a rare thing in a large sample set, without mentioning precision, recall, AUC, or other more useful metrics.
The abstract claims that both sensitivity and specificity are high:
> The developed platform successfully classified the human prostate and pancreatic cancer urines in a label-free method supported by two types of deep learning networks, with high clinical sensitivity and specificity.
I don't have access to the full article so I can't see the numbers. It's not on the hub of science yet either.
> The developed platform successfully classified the human prostate and pancreatic cancer urines in a label-free method supported by two types of deep learning networks, with high clinical sensitivity and specificity.
I don't have access to the full article so I can't see the numbers. It's not on the hub of science yet either.
It's not a sensitivity / specificity thing; it's a base rate thing [1].
Suppose that the percentage of people with prostate cancer at any one time is 1 in 1000 (i.e., then "base rate" is 1 in 1000). (Turns out this is actually on the same order of magnitude as real number [2].). And suppose this test has 99% sensitivity and 99% specificity.
And suppose you test 1,000,000 people.
Of those 1,000,000 people, 1,000 will actually have prostate cancer, and 999,000 will not.
Of those 1000 that actually have it, 990 will have a positive test (true positive), and 10 will have a negative test (false negative).
Of the 999,000 people who don't actually have it, 989,010 will have a negative test (true negative), and 9,990 will have a positive test (false positive).
So even with a test of 99% accuracy, if you get a positive result, your chances of actually having prostate cancer are still only 990 / 10980, or about 9%; 91% of the positives will be false positives.
And of course, the more rare the cancer, the worse it gets.
EDIT2: So, to follow on with GGP's point: "Near perfect accuracy" isn't very specific, but colloquially would imply that if you have a positive test, you have a high chance of actually having cancer. To get that number to 95% you'd need to have only 52 false positives, would require a specificity closer to 99.995%.
EDIT: Fixed some math
[1] https://en.wikipedia.org/wiki/Base_rate_fallacy [2] https://www.cancer.org/cancer/prostate-cancer/about/key-stat...
Suppose that the percentage of people with prostate cancer at any one time is 1 in 1000 (i.e., then "base rate" is 1 in 1000). (Turns out this is actually on the same order of magnitude as real number [2].). And suppose this test has 99% sensitivity and 99% specificity.
And suppose you test 1,000,000 people.
Of those 1,000,000 people, 1,000 will actually have prostate cancer, and 999,000 will not.
Of those 1000 that actually have it, 990 will have a positive test (true positive), and 10 will have a negative test (false negative).
Of the 999,000 people who don't actually have it, 989,010 will have a negative test (true negative), and 9,990 will have a positive test (false positive).
So even with a test of 99% accuracy, if you get a positive result, your chances of actually having prostate cancer are still only 990 / 10980, or about 9%; 91% of the positives will be false positives.
And of course, the more rare the cancer, the worse it gets.
EDIT2: So, to follow on with GGP's point: "Near perfect accuracy" isn't very specific, but colloquially would imply that if you have a positive test, you have a high chance of actually having cancer. To get that number to 95% you'd need to have only 52 false positives, would require a specificity closer to 99.995%.
EDIT: Fixed some math
[1] https://en.wikipedia.org/wiki/Base_rate_fallacy [2] https://www.cancer.org/cancer/prostate-cancer/about/key-stat...
Right, the "lots of false positives" conclusion is still the correct analysis.
I just wanted to point out that both "P(Detect | Positive)" and "P(Not Detect | Negative)" are both high, since GP only mentioned one and not the other.
I just wanted to point out that both "P(Detect | Positive)" and "P(Not Detect | Negative)" are both high, since GP only mentioned one and not the other.
[deleted]
Show HN: Time for me to reveal the details of my smart toilet with full-spectrum disease detection. It detects diabetes, pre-diabetes, cancer, covid/flu/cold, smallpox, hypertension, pregnancy, pre-infertility, and heroin addiction.
IANAL, but feel free to use this now open-sourced idea for your competitive product.
IANAL, but feel free to use this now open-sourced idea for your competitive product.
Of course this happens when I finally have come to appreciate the finger prostate check.
That's actually a colon cancer check - the prostate is checked along with it, but you could well do that via an abdominal USG.
Finger check will usually only detect a huge prostate or rectal cancer(not other portions of the colon), or a rectal cancer just next to the anal canal.
Colonoscopy(can detect polyps BEFORE they become cancer, but more costly and hard for patient) is the gold standard for detecting colon cancer, along with fecal occult blood which is cheap and easier to do but will detect only more advanced cancers. For prostate, the early detection and screening is usually using done using a PSA blood test.
Colonoscopy(can detect polyps BEFORE they become cancer, but more costly and hard for patient) is the gold standard for detecting colon cancer, along with fecal occult blood which is cheap and easier to do but will detect only more advanced cancers. For prostate, the early detection and screening is usually using done using a PSA blood test.
You can actually detect relatively small prostate cancers, even more so for superficial ones. PSA is not raised in all prostate cancers and cannot be used with prostate examination to rule out prostate cancer.
Should read: Cannot be used ^without prostate examination
Not a doctor, but my SO is one. AFAIK the finger is used to check for prostate inflammation/abnormalities. Colon cancer checks are done either via analysis of hidden blood in feces or colonoscopy.
At best, I can imagine this being a rectum cancer check, not a colon cancer check - the colon is around 1.6-1.7m long. Even the rectum is longer than a typical finger (typically ~12cm long according to Wikipedia).
You cannot reliably assess for prostate cancer using an ultrasound. PSA blood test in concert with rectal examination of prostate, with MRI of prostate of abnormality detected as the next step.
Here, "label free" seems to mean that the detection is done more or less directly, instead of relying on a consumable extra material (the label) that binds to the thing being searched for, thus marking it. At least that's my understanding after trying to goog my way out of that confusion for a few seconds.
See [1] for instance, or [2] but that seems to be focusing on maas spectroscopy.
[1]: https://www.sciencedirect.com/topics/biochemistry-genetics-a...
[2]: https://en.wikipedia.org/wiki/Label-free_quantification
See [1] for instance, or [2] but that seems to be focusing on maas spectroscopy.
[1]: https://www.sciencedirect.com/topics/biochemistry-genetics-a...
[2]: https://en.wikipedia.org/wiki/Label-free_quantification
How long will it take for it to be commercially available?
Considering that the test doesn't require violation of the patient's body, I would not expect it to take extremely long.
Ehh .. this is a very preliminary paper in a small journal and medical devices have to undergo rigorous clinical testing before being used in clinical settings. If this will ever be clinically available, it will probably take at least 7 - 10 years.
If it's cost effective probably <5 years, somewhere it'll see screening usage.
Wide spread rollout who knows.
But FebriDX was relatively new when it was rolled out in parts of the UK. And with the "COVID backlog" if this is cost effective (which includes if it even works), it'll see usage somewhere like substance misuse clinics.
Wide spread rollout who knows.
But FebriDX was relatively new when it was rolled out in parts of the UK. And with the "COVID backlog" if this is cost effective (which includes if it even works), it'll see usage somewhere like substance misuse clinics.
Ehh... I wouldn't say "rigorous clinic testing". Many devices use/abuse the 510k process.
As far as I understand, pancreatic cancer has such poor survival rates because it typically doesn't present symptoms until it is too late to treat effectively. Is this test able to detect it early enough for the sufferer to gain more benefit from treatment?
For whatever reason, I can't access the pdf file - but with pancreatic cancer being as latent and asymptomatic as it is, does this study mention how early it can be detected? It would be absolutely fantastic news if pancreatic cancer can be detected early on - as once the symptoms set in, the prognosis tends to be very poor. It's one of the few diseases I've seen where people have gone from feeling uneasy, to being dead in just a couple of weeks.
twelve40(2)
Kinda hard to say anything here with only the abstract. The interesting part for me isn't so much the test method itself, but whether the biomarkers used here are actually good for anything.
And as already mentioned in another comment, you need a very high specificity for this kind of screening test to be useful, because "no cancer" is much, much more likely than "cancer" you get a lot of false positives even with relatively high specificity.
And as already mentioned in another comment, you need a very high specificity for this kind of screening test to be useful, because "no cancer" is much, much more likely than "cancer" you get a lot of false positives even with relatively high specificity.
Articles paywalled but 100% this will be commercialised if it works at all!
Did they say near-perfect accuracy? It's very odd phrase to hear from an academic paper and no mention of sens/spec in abstract.
Did they say near-perfect accuracy? It's very odd phrase to hear from an academic paper and no mention of sens/spec in abstract.
The abstract mentions both, says that they are both high, but does not provide numbers.
Haha, hit my brains content filter.
That's a shame (no figures).
That's a shame (no figures).
Sounds great maybe too great this isn’t like another single drop of blood style test that is magic and results in billions in fraud?
This is a highly technical article describing the results of current research in a scientific peer-reviewed academic journal. The particular urine tests being used are, as far as I can tell from the abstract, intended as demonstration or proof of concept, showing that the technology actually works, rather than the final product of the research. It would be very surprising if this were some kind of fraud, and it's also probable that a lot of work still needs to be done in order to make this a useful tool in regular doctor offices.
*Urinos*
A device that you pee into every once in a while, and it updates your profile giving you health information based on the tests it ran. It would have plugable modules for each test you want to run, maybe 10 or so slots, and you would pay for subscriptions where a new module would come in the mail every time it needs to be swapped out.