Weakly supervised end-to-end artificial intelligence in gastrointestinal endoscopy

Abstract

Artificial intelligence (AI) is widely used to analyze gastrointestinal (GI) endoscopy image data. AI has led to several clinically approved algorithms for polyp detection, but application of AI beyond this specific task is limited by the high cost of manual annotations. Here, we show that a weakly supervised AI can be trained on data from a clinical routine database to learn visual patterns of GI diseases without any manual labeling or annotation. We trained a deep neural network on a dataset of N = 29,506 gastroscopy and N = 18,942 colonoscopy examinations from a large endoscopy unit serving patients in Germany, the Netherlands and Belgium, using only routine diagnosis data for the 42 most common diseases. Despite a high data heterogeneity, the AI system reached a high performance for diagnosis of multiple diseases, including inflammatory, degenerative, infectious and neoplastic diseases. Specifically, a cross-validated area under the receiver operating curve (AUROC) of above 0.70 was reached for 13 diseases, and an AUROC of above 0.80 was reached for two diseases in the primary data set. In an external validation set including six disease categories, the AI system was able to significantly predict the presence of diverticulosis, candidiasis, colon and rectal cancer with AUROCs above 0.76. Reverse engineering the predictions demonstrated that plausible patterns were learned on the level of images and within images and potential confounders were identified. In summary, our study demonstrates the potential of weakly supervised AI to generate high-performing classifiers and identify clinically relevant visual patterns based on non-annotated routine image data in GI endoscopy and potentially other clinical imaging modalities.

Deep Neural Network Models for Colon Cancer Screening

“…A study by Iizuka et al demonstrated high AUC values (0.96–0.99) for adenomas in the gastric and colonic epithelium by applying the same techniques, with few limitations [ 68 ]. A study on weakly supervised e2e AI in gastrointestinal endoscopy found that the AUC for the diagnoses of 13 diseases had a value of 0.7–0.8 and was able to predict the presence of colorectal cancer with an AUC > 0.76 [ 49 ]. The accuracy of a few CNN models in recognizing features in colorectal histopathological WSIs was compared.…”

“…It accomplished good predictive performance in the identification of several diagnoses from gastrointestinal endoscopy images. This displays the potential of weakly supervised AI in clinical imaging modalities, in contrast to claims that manual annotations are a bottleneck for the future clinical application of AI [ 49 ]. The image dataset was preprocessed in MATLAB R2021a, and the ResNet-18 model was trained with the datasets.…”

“…Another study used a larger image size (768 × 768 pixels) to preserve tissue architecture information and reduce computational cost, as opposed to a smaller patch size (384 × 384 pixels), which produced the same result but had a higher computational cost [ 48 ]. In the study using WSIs of cytokeratin immunohistochemistry obtained using a digital slide scanner, images were standardized to set 1 μm = 1 pixel and were saved as non-layered joint photographic experts group (JPEG) images, which were then converted into binary images after deletion of non-cancerous areas [ 49 ]. Another study utilized an automatic cropping approach, which removed black margins and resulted in a square image with a 1:1 ratio [ 50 ].…”

Seed storage allergens tackled via next-generation research assistant

“…Hence, in this global vision, PGE became an instrument, in which synthesis of DNA/RNA fragments, vector assembly, and choice of the transformation method became directly available for the researcher, depending on the final research goal. So far, ML has already proven to be a revolutionary tool in supporting clinical studies, medical and precise diagnosis (Mansour et al, 2021;Fang et al, 2022) with application in medical diagnostics and for detecting, forecasting, and predicting of immunity responses (Nawaz et al, 2021;Buendgens et al, 2022). In this context, the utilization of an NGA for the comprehensive analysis of allergens could be completely revolutionary in human health.…”

Artificial intelligence in Barrett's oesophagus and the need for shared and combined data

“…Second, in cases in which clinical data cannot be shared, federated learning, swarm learning and other techniques may help leverage the combined information from different datasets without actually merging them 15 . Third, the creation of public, large‐scale, not necessarily densely labelled, 16 datasets (‘GI ImageNet’ 17 ) would be a considerable step forward. These efforts, particularly the latter, may be cross funded by offering certification processes for commercial AI products against benchmarking datasets 13 .…”