Artificial intelligence in GI endoscopy: stumbling blocks, gold standards and the role of endoscopy societies

Schmitz, Rüdiger; Werner, René; Repici, Alessandro; Bisschops, Raf; Meining, Alexander; Zornow, Michael; Messmann, Helmut; Hassan, Cesare; Rösch, Thomas

doi:10.1136/gutjnl-2020-323115

Cited by 11 publications

(14 citation statements)

References 31 publications

(28 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to these two examples, we observe a recent surge of studies in the field of gastroenterology 2 . The use of ML in gastroenterology is expected to significantly improve detection and characterization of colon polyps and other precancerous lesions of the Gastrointestinal (GI) tract 3 . These potential advances are mainly expected from artificial neural networks, specifically deep learning-based methods 4 .…”

Section: Introductionmentioning

confidence: 99%

On evaluation metrics for medical applications of artificial intelligence

Hicks

Strümke

Thambawita

et al. 2022

Sci Rep

261

109

View full text Add to dashboard Cite

Clinicians and software developers need to understand how proposed machine learning (ML) models could improve patient care. No single metric captures all the desirable properties of a model, which is why several metrics are typically reported to summarize a model’s performance. Unfortunately, these measures are not easily understandable by many clinicians. Moreover, comparison of models across studies in an objective manner is challenging, and no tool exists to compare models using the same performance metrics. This paper looks at previous ML studies done in gastroenterology, provides an explanation of what different metrics mean in the context of binary classification in the presented studies, and gives a thorough explanation of how different metrics should be interpreted. We also release an open source web-based tool that may be used to aid in calculating the most relevant metrics presented in this paper so that other researchers and clinicians may easily incorporate them into their research.

show abstract

Section: Introductionmentioning

confidence: 99%

On evaluation metrics for medical applications of artificial intelligence

Hicks

Strümke

Thambawita

et al. 2022

Sci Rep

261

109

View full text Add to dashboard Cite

show abstract

“…The use of ML in gastroenterology is expected to significantly improve detection and characterization of colon polyps and other precancerous lesions of the Gastrointestinal (GI) tract 3 . These potential advances are mainly expected from artificial neural networks, specifically deep learning-based methods 4 .…”

mentioning

confidence: 99%

On evaluation metrics for medical applications of artificial intelligence

Hicks

Strümke

Thambawita

et al. 2021

Preprint

View full text Add to dashboard Cite

Clinicians and model developers need to understand how proposed machine learning (ML) models could improve patient care. In fact, no single metric captures all the desirable properties of a model and several metrics are typically reported to summarize a model's performance. Unfortunately, these measures are not easily understandable by many clinicians. Moreover, comparison of models across studies in an objective manner is challenging, and no tool exists to compare models using the same performance metrics. This paper looks at previous ML studies done in gastroenterology, provides an explanation of what different metrics mean in the context of the presented studies, and gives a thorough explanation of how different metrics should be interpreted. We also release an open source web-based tool that may be used to aid in calculating the most relevant metrics presented in this paper so that other researchers and clinicians may easily incorporate them into their research.

show abstract

“…AI systems in general, and weakly supervised systems in particular, are potentially susceptible to biases in the distribution of ground truth labels 29 . One of the most important prerequisites for medical AI is that it provides explainability to gain trust among users and allow identification of biases 30 . While our weakly supervised AI system achieved high diagnostic performance and enables explainability, it is not immune to biases in the image dataset.…”

Section: Discussionmentioning

confidence: 99%

Weakly supervised end-to-end artificial intelligence in gastrointestinal endoscopy

Buendgens

Cifci

Laleh

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Artificial intelligence (AI) is widely used to analyze gastrointestinal (GI) endoscopy image data. AI has led to several clinically approved algorithms for polyp detection, but application of AI beyond this specific task is limited by the high cost of manual annotations. Here, we show that a weakly supervised AI can be trained on data from a clinical routine database to learn visual patterns of GI diseases without any manual labeling or annotation. We trained a deep neural network on a dataset of N = 29,506 gastroscopy and N = 18,942 colonoscopy examinations from a large endoscopy unit serving patients in Germany, the Netherlands and Belgium, using only routine diagnosis data for the 42 most common diseases. Despite a high data heterogeneity, the AI system reached a high performance for diagnosis of multiple diseases, including inflammatory, degenerative, infectious and neoplastic diseases. Specifically, a cross-validated area under the receiver operating curve (AUROC) of above 0.70 was reached for 13 diseases, and an AUROC of above 0.80 was reached for two diseases in the primary data set. In an external validation set including six disease categories, the AI system was able to significantly predict the presence of diverticulosis, candidiasis, colon and rectal cancer with AUROCs above 0.76. Reverse engineering the predictions demonstrated that plausible patterns were learned on the level of images and within images and potential confounders were identified. In summary, our study demonstrates the potential of weakly supervised AI to generate high-performing classifiers and identify clinically relevant visual patterns based on non-annotated routine image data in GI endoscopy and potentially other clinical imaging modalities.

show abstract

Artificial intelligence in GI endoscopy: stumbling blocks, gold standards and the role of endoscopy societies

Cited by 11 publications

References 31 publications

On evaluation metrics for medical applications of artificial intelligence

On evaluation metrics for medical applications of artificial intelligence

On evaluation metrics for medical applications of artificial intelligence

Weakly supervised end-to-end artificial intelligence in gastrointestinal endoscopy

Contact Info

Product

Resources

About