Establishing key research questions for the implementation of artificial intelligence in colonoscopy: a modified Delphi method

Abstract

Background and Aims Artificial intelligence (AI) research in colonoscopy is progressing rapidly but widespread clinical implementation is not yet a reality. We aimed to identify the top implementation research priorities. Methods An established modified Delphi approach for research priority setting was used. Fifteen international experts, including endoscopists and translational computer scientists/engineers from 9 countries participated in an online survey over 9 months. Questions related to AI implementation in colonoscopy were generated as a long-list in the first round, and then scored in two subsequent rounds to identify the top 10 research questions. Results The top 10 ranked questions were categorised into 5 themes. Theme 1: Clinical trial design/end points (4 questions), related to optimum trial designs for polyp detection and characterisation, determining the optimal end-points for evaluation of AI and demonstrating impact on interval cancer rates. Theme 2: Technological Developments (3 questions), including improving detection of more challenging and advanced lesions, reduction of false positive rates and minimising latency. Theme 3: Clinical adoption/Integration (1 question) concerning effective combination of detection and characterisation into one workflow. Theme 4: Data access/annotation (1 question) concerning more efficient or automated data annotation methods to reduce the burden on human experts. Theme 5: Regulatory Approval (1 question) related to making regulatory approval processes more efficient. Conclusions This is the first reported international research priority setting exercise for AI in colonoscopy. The study findings should be used as a framework to guide future research with key stakeholders to accelerate the clinical implementation of AI in endoscopy.

On evaluation metrics for medical applications of artificial intelligence

“…These potential advances are mainly expected from artificial neural networks, specifically deep learning-based methods 4 . Safe and efficient adoption of ML tools in clinical gastroenterology requires a thorough understanding of the performance metrics of the resulting models and confirmation of their clinical utility 5 .…”

Computer-Aided Detection False Positives in Colonoscopy

“…An example of a simple method that could be used to reduce FPs is re-training the CADe algorithms with scenarios that currently lead to FPs. Another approach could be the adoption of recurrent neural networks, which have memory and can process temporal sequences of frames in a way that is similar to the learning process of human brains [ 10 ]. Misawa et al reported that when they changed their old algorithm [ 17 ] to YoloV3 (You Only Look Once, Version 3), a state-of-the-art, real-time object detection algorithm, better specificity was achieved (increasing from 90.9% to 93.7%) [ 19 ].…”

“…An accompanying limitation of the CADe is false positives (FPs), which occur when the algorithm identifies a “polyp” that the endoscopist would disagree with. FPs were ranked 3rd in importance among 59 future research questions related to CADe [ 10 ]. Therefore, we conducted this systemic review on the definitions, causes, and adverse effects of the CADe FPs.…”

Addressing false-positive findings with artificial intelligence for polyp detection

“…However, it is difficult to compare different AI algorithms/settings/products due to the lack of established definition and measurement criteria for false-positive alarms in AI-assisted colonoscopy. This problem was also emphasized as one of the most crucial issues at a recent international expert meeting [4].…”