Addressing false-positive findings with artificial intelligence for polyp detection

Mori, Yuichi; Bretthauer, Michael

doi:10.1055/a-1381-7849

Cited by 4 publications

(4 citation statements)

References 7 publications

(8 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inevitably, a low FP alert results in lower sensitivity. 19,23,24 Our findings highlight the effects of different FP thresholds on the number of FPs reported. Currently, there is no consensus on the optimal FP threshold for AI systems, and various definitions of FP threshold have been used in different CADe studies, ranging from >0.5 to >2 seconds, while some studies have not specified the definition of FP threshold at all.…”

Section: Discussionmentioning

confidence: 54%

“…High false-positive alarm rates can cause stress, visual disturbances, unnecessary checking of non-pathological areas, and prolonged procedure times. 19 , 20 However, lowering the false-positive threshold may also decrease detection sensitivity. 21 Thus, we developed “Deep-GI,” an AI model for colonic polyp detection that aimed for a lower FP alarm rate with comparable polyp detection sensitivity (PDS).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Performance comparison between two computer-aided detection colonoscopy models by trainees using different false positive thresholds: a cross-sectional study in Thailand

Tiankanon,

Karuehardsuwan,

Aniwan

et al. 2024

Clin Endosc

View full text Add to dashboard Cite

Background/Aims: This study aims to compare polyp detection performance of “Deep-GI,” a newly developed artificial intelligence (AI) model, to a previously validated AI model computer-aided polyp detection (CADe) using various false positive (FP) thresholds and determining the best threshold for each model.Methods: Colonoscopy videos were collected prospectively and reviewed by three expert endoscopists (gold standard), trainees, CADe (CAD EYE; Fujifilm Corp.), and Deep-GI. Polyp detection sensitivity (PDS), polyp miss rates (PMR), and false-positive alarm rates (FPR) were compared among the three groups using different FP thresholds for the duration of bounding boxes appearing on the screen.Results: In total, 170 colonoscopy videos were used in this study. Deep-GI showed the highest PDS (99.4% vs. 85.4% vs. 66.7%, p<0.01) and the lowest PMR (0.6% vs. 14.6% vs. 33.3%, p<0.01) when compared to CADe and trainees, respectively. Compared to CADe, Deep-GI demonstrated lower FPR at FP thresholds of ≥0.5 (12.1 vs. 22.4) and ≥1 second (4.4 vs. 6.8) (both p<0.05). However, when the threshold was raised to ≥1.5 seconds, the FPR became comparable (2 vs. 2.4, p=0.3), while the PMR increased from 2% to 10%.Conclusions: Compared to CADe, Deep-GI demonstrated a higher PDS with significantly lower FPR at ≥0.5- and ≥1-second thresholds. At the ≥1.5-second threshold, both systems showed comparable FPR with increased PMR.

show abstract

Section: Discussionmentioning

confidence: 54%

Section: Introductionmentioning

confidence: 99%

Performance comparison between two computer-aided detection colonoscopy models by trainees using different false positive thresholds: a cross-sectional study in Thailand

Tiankanon,

Karuehardsuwan,

Aniwan

et al. 2024

Clin Endosc

View full text Add to dashboard Cite

show abstract

“…However, as far as we know, no studies have explored the impact of models with different levels of FPs on endoscopists. 28 Our study suggests a safety threshold and a novel method to assess FPs for CADe, which may provide a new direction for future design of CADe to avoid the potential harm caused by FPs.…”

Section: Discussionmentioning

confidence: 92%

Assessment of the role of false‐positive alerts in computer‐aided polyp detection for assistance capabilities

Zhang,

Yao,

Jiang

et al. 2024

J of Gastro and Hepatol

View full text Add to dashboard Cite

Background and AimFalse positives (FPs) pose a significant challenge in the application of artificial intelligence (AI) for polyp detection during colonoscopy. The study aimed to quantitatively evaluate the impact of computer‐aided polyp detection (CADe) systems' FPs on endoscopists.MethodsThe model's FPs were categorized into four gradients: 0–5, 5–10, 10–15, and 15–20 FPs per minute (FPPM). Fifty‐six colonoscopy videos were collected for a crossover study involving 10 endoscopists. Polyp missed rate (PMR) was set as primary outcome. Subsequently, to further verify the impact of FPPM on the assistance capability of AI in clinical environments, a secondary analysis was conducted on a prospective randomized controlled trial (RCT) from Renmin Hospital of Wuhan University in China from July 1 to October 15, 2020, with the adenoma detection rate (ADR) as primary outcome.ResultsCompared with routine group, CADe reduced PMR when FPPM was less than 5. However, with the continuous increase of FPPM, the beneficial effect of CADe gradually weakens. For secondary analysis of RCT, a total of 956 patients were enrolled. In AI‐assisted group, ADR is higher when FPPM ≤ 5 compared with FPPM > 5 (CADe group: 27.78% vs 11.90%; P = 0.014; odds ratio [OR], 0.351; 95% confidence interval [CI], 0.152–0.812; COMBO group: 38.40% vs 23.46%, P = 0.029; OR, 0.427; 95% CI, 0.199–0.916). After AI intervention, ADR increased when FPPM ≤ 5 (27.78% vs 14.76%; P = 0.001; OR, 0.399; 95% CI, 0.231–0.690), but no statistically significant difference was found when FPPM > 5 (11.90% vs 14.76%, P = 0.788; OR, 1.111; 95% CI, 0.514–2.403).ConclusionThe level of FPs of CADe does affect its effectiveness as an aid to endoscopists, with its best effect when FPPM is less than 5.

show abstract

“…This additional time is mainly attributed to the time spent on false-positive detection which is a significant disadvantage of every AI-assisted colonoscopy [11,31,32]. The CADe system with higher polyp detection sensitivity usually accompanies a lower false-positive alarm threshold, which can cause stress, visual disturbance, unnecessary checking on non-pathological areas, and a prolonged procedure time [33,34]. In a study using CADe "GI-GENIUS" (Medtronic, Minneapolis, Minn, USA), Hassan et al reported the mean number of false positives was 27 alarms per colonoscopy, and the overall time spent on each false positive was variable with the average ranging from 1 to 6.9 seconds [31].…”

Section: Adenoma Per Colonoscopymentioning

confidence: 99%

Improvement of adenoma detection rate by two computer-aided colonic polyp detection systems in high adenoma detectors: a randomized multicenter trial

Tiankanon,

Aniwan,

Kerr

et al. 2023

Endoscopy

View full text Add to dashboard Cite

Objective: This study aimed to evaluate the benefits of a self-developed computer-aided polyp detection system (SD-CADe) and a commercial CADe (CM-CADe) in high adenoma detectors compared to white-light colonoscopy (WLE) as a control. Methods: Average-risk 50–75-year-old individuals who underwent screening colonoscopy at five referral centers were randomized to SD-CADe, CM-CADe, or WLE groups (1:1:1 ratio). Trainees and staff with recorded ADR>35% were recruited. The primary outcome was the adenoma detection rate (ADR). Secondary outcomes were the proximal adenoma detection rate (pADR), advanced adenoma detection rate (AADR), and the number of adenomas, proximal and advanced adenomas per colonoscopy (APC, pAPC, and AAPC, respectively). Results: The study enrolled 1,200 participants. The ADR in the control, CM-CADe, and SD-CADe groups was 38.3%, 50%, and 54.8%, respectively. pADR was 23%, 32.3%, and 38.8%, respectively. AADR was 6%, 10.3%, and 9.5%, respectively. After imbalance factor adjustments, ADR and pADR in both intervention groups were significantly higher than controls (p<0.001 all). The APC found in the control, CM-CADe, and SD-CADe groups was 1.05, 1.63, and 1.51, respectively. The pAPC was 0.69, 1.03, 1.06, respectively, and the AAPC was 0.07, 0.12, and 0.10, respectively. Both CADe systems showed significantly higher APC and pAPC. AADR and AAPC were improved in both CADe groups versus control, although not statistically significant. Conclusion: Even in high adenoma detectors, CADes significantly improved ADR and APC. The AADR tended to be improved by both systems, and this may enhance colorectal cancer prevention.

show abstract

Addressing false-positive findings with artificial intelligence for polyp detection

Cited by 4 publications

References 7 publications

Performance comparison between two computer-aided detection colonoscopy models by trainees using different false positive thresholds: a cross-sectional study in Thailand

Performance comparison between two computer-aided detection colonoscopy models by trainees using different false positive thresholds: a cross-sectional study in Thailand

Assessment of the role of false‐positive alerts in computer‐aided polyp detection for assistance capabilities

Improvement of adenoma detection rate by two computer-aided colonic polyp detection systems in high adenoma detectors: a randomized multicenter trial

Contact Info

Product

Resources

About