Frame-by-Frame Analysis of a Commercially Available Artificial Intelligence Polyp Detection System in Full-Length Colonoscopies

Abstract

<b><i>Introduction:</i></b> Computer-aided detection (CADe) helps increase colonoscopic polyp detection. However, little is known about other performance metrics like the number and duration of false-positive (FP) activations or how stable the detection of a polyp is. <b><i>Methods:</i></b> 111 colonoscopy videos with total 1,793,371 frames were analyzed on a frame-by-frame basis using a commercially available CADe system (GI-Genius, Medtronic Inc.). Primary endpoint was the number and duration of FP activations per colonoscopy. Additionally, we analyzed other CADe performance parameters, including per-polyp sensitivity, per-frame sensitivity, and first detection time of a polyp. We additionally investigated whether a threshold for withholding CADe activations can be set to suppress short FP activations and how this threshold alters the CADe performance parameters. <b><i>Results:</i></b> A mean of 101 ± 88 FPs per colonoscopy were found. Most of the FPs consisted of less than three frames with a maximal 66-ms duration. The CADe system detected all 118 polyps and achieved a mean per-frame sensitivity of 46.6 ± 26.6%, with the lowest value for flat polyps (37.6 ± 24.8%). Withholding CADe detections up to 6 frames length would reduce the number of FPs by 87.97% (<i>p</i> < 0.001) without a significant impact on CADe performance metrics. <b><i>Conclusions:</i></b> The CADe system works reliable but generates many FPs as a side effect. Since most FPs are very short, withholding short-term CADe activations could substantially reduce the number of FPs without impact on other performance metrics. Clinical practice would benefit from the implementation of customizable CADe thresholds.

Real-time colorectal polyp detection using a novel computer-aided detection system (CADe): a feasibility study

“…In contrast, in a frame-by-frame analysis by Brand et al, false positives were defined as any frame that included a "detected area that was not in contact with a polyp." This more strict definition resulted in a rather high number of false positives per colonoscopy (mean number per of false positives colonoscopy: 101) [34]. [35].…”

Direct comparison of multiple computer-aided polyp detection systems

“…A board-certified gastroenterologist and experienced endoscopist, with over 4000 colonoscopies performed, screened all the videos as described previously 14 . Using a custom-made annotation tool, the colonoscopies were analyzed in a deep frame-by-frame process and in each frame that contained a polyp, a bounding box was drawn around the lesion 15 .…”

“…A board-certified gastroenterologist and experienced endoscopist with over 4,000 performed colonoscopies screened all the videos as described before [14]. The inclusion criteria were examinations performed for screening purposes or post polypectomy surveillance.…”

Utilizing adaptive deformable convolution and position embedding for colon polyp segmentation with a visual transformer

“…The introduction of a semi-automated video annotation tool by Krenzer et al 43 , aimed at streamlining the machine learning annotation process for medical professionals, was foundational for the training of accurate AI systems. This was complemented by the development of a benchmark dataset, ENDOTEST, by Fitting et al 44 , designed to rigorously evaluate computer-aided polyp detection systems, and the efforts of Brand et al 45,46 to develop, evaluate, and analyze the effectiveness of deep learning models and commercially available AI systems in real-world clinical settings were highlighted, showcasing the practical applications and challenges of integrating AI into current medical practices.…”