Screening of normal endoscopic large bowel biopsies with artificial intelligence: a retrospective study

Abstract

Objectives: Develop an interpretable AI algorithm to rule out normal large bowel endoscopic biopsies saving pathologist resources. Design: Retrospective study. Setting: One UK NHS site was used for model training and internal validation. External validation conducted on data from two other NHS sites and one site in Portugal. Participants: 6,591 whole-slides images of endoscopic large bowel biopsies from 3,291 patients (54% Female, 46% Male). Main outcome measures: Area under the receiver operating characteristic and precision recall curves (AUC-ROC and AUC-PR), measuring agreement between consensus pathologist diagnosis and AI generated classification of normal versus abnormal biopsies. Results: A graph neural network was developed incorporating pathologist domain knowledge to classify the biopsies as normal or abnormal using clinically driven interpretable features. Model training and internal validation were performed on 5,054 whole slide images of 2,080 patients from a single NHS site resulting in an AUC-ROC of 0.98 (SD=0.004) and AUC-PR of 0.98 (SD=0.003). The predictive performance of the model was consistent in testing over 1,537 whole slide images of 1,211 patients from three independent external datasets with mean AUC-ROC = 0.97 (SD=0.007) and AUC-PR = 0.97 (SD=0.005). Our analysis shows that at a high sensitivity threshold of 99%, the proposed model can, on average, reduce the number of normal slides to be reviewed by a pathologist by 55%. A key advantage of IGUANA is its ability to provide an explainable output highlighting potential abnormalities in a whole slide image as a heatmap overlay in addition to numerical values associating model prediction with various histological features. Example results with interpretable features can be viewed online at https://iguana.dcs.warwick.ac.uk/. Conclusions: An interpretable AI model was developed to screen abnormal cases for review by pathologists. The model achieved consistently high predictive accuracy on independent cohorts showing its potential in optimising increasingly scarce pathologist resources and for achieving faster time to diagnosis. Explainable predictions of IGUANA can guide pathologists in their diagnostic decision making and help boost their confidence in the algorithm, paving the way for future clinical adoption.

Holding AI to Account: Challenges for the Delivery of Trustworthy AI in Healthcare

“…Providing local xAI is generally easier than global xAI and various local xAI techniques have been developed for AI systems in healthcare (Poceviciute et al 2020, Zhang et al 2022. Saliency maps, a visual display highlighting features that contribute to a prediction (e.g., heat maps), are a natural fit with AI systems that make predictions from image data and are also intuitive (Guidotti et al 2018, Machlev et al 2022, Graham et al 2022. Couture et al (2018) tested a saliency map technique in an AI system for detecting breast cancer tissue.…”

“…This could also be a solution to the problem of providing progressively richer and recipient designed explanations of behaviour (Button & Dourish 1996) and go some way to providing a degree of natural accountability in AI systems. For AI systems designed to assist in the diagnosis of medical images, a more general recommendation would be that local account design should be informed by the practices that healthcare professionals use to interpret images, i.e., clinically meaningful features (Graham et al 2022, Nix et al 2022) and how underlying physical structures manifest themselves. In mammography, this would suggest that local accounts also be sensitive to the 'geographies of suspicion', but also how radiologists' make use of their understanding of the physics of mammograms and knowledge of breast architecture.…”

Why do errors arise in artificial intelligence diagnostic tools in histopathology and how can we minimize them?

“…34 Another colon biopsy screening tool, aimed at separating colonic biopsies into normal and abnormal cases, did not detect several entities that human pathologists would expect to, such as signet ring cells, giant cells, mitotic figures, and spirochaetosis. 35 This means that although such algorithm results are promising, to be used safely pathologists need to understand their limitations and delineate further work required to improve performance.…”

Bi-level Graph Learning Unveils Prognosis-Relevant Tumor Microenvironment Patterns from Breast Multiplexed Digital Pathology

“…On the other hand, there is a growing interest in harnessing graph-based learning techniques to analyze the association between the TME and disease without the need for explicit hypotheses in a data-driven manner [21,22]. Numerous recent studies have reported promising results by leveraging graph neural networks (GNNs) to model the TME and predict the presence [23,24], grade [25,26], stage [27], subtype [28,29], and prognosis [30][31][32][33][34] of diverse types of cancers. Yet, the limited size, biological heterogeneity, and differences in staining and imaging protocols of clinical datasets constrain the quality of modern machine learning models by impeding their generalization capacity, particularly in crossstudy scenarios that are often overlooked [35,36].…”

“…Additionally, the interpretability of deep learning models, in general, remains a challenge [37, 38]. Many of these studies do not discuss explanation in their works [25, 26, 28, 29, 32, 33], some employ gradient-based [23, 27, 34] and permutation-based [24] methods to generate an importance heatmap as the models’ explanation, and others conduct post-hoc analyses on learned graph features to obtain some clinical implications from their models [30, 31]. These explanations, although providing some insights into learned information, might not be clear, intuitive, or meaningful, especially for clinical experts and cancer researchers.…”