Uncertainty-informed deep learning models enable high-confidence predictions for digital histopathology

Abstract

A model’s ability to express its own predictive uncertainty is an essential attribute for maintaining clinical user confidence as computational biomarkers are deployed into real-world medical settings. In the domain of cancer digital histopathology, we describe a clinically-oriented approach to uncertainty quantification for whole-slide images, estimating uncertainty using dropout and calculating thresholds on training data to establish cutoffs for low- and high-confidence predictions. We train models to identify lung adenocarcinoma vs. squamous cell carcinoma and show that high-confidence predictions outperform predictions without uncertainty, in both cross-validation and testing on two large external datasets spanning multiple institutions. Our testing strategy closely approximates real-world application, with predictions generated on unsupervised, unannotated slides using predetermined thresholds. Furthermore, we show that uncertainty thresholding remains reliable in the setting of domain shift, with accurate high-confidence predictions of adenocarcinoma vs. squamous cell carcinoma for out-of-distribution, non-lung cancer cohorts.

Application of simultaneous uncertainty quantification for image segmentation with probabilistic deep learning: Performance benchmarking of oropharyngeal cancer target delineation as a use-case

“…Previous work in DL uncertainty estimation has been extensively investigated in segmenting lung-related [19][20][21] and brain-related [22][23][24] structures. While DL uncertainty estimation has been applied to a broad range of HNSCC-related classification tasks [25][26][27][28] and dose prediction 29 , only a limited number of studies have investigated uncertainty estimation for 3-dimensional HNSCC medical image segmentation, predominantly for nasopharyngeal cancer 30 or organs at risk 31,32 ; to our knowledge only one study has attempted to investigate segmentation uncertainty estimation in OPC 33 . Therefore, there exists a significant gap in knowledge about how to construct DL auto-segmentation models that lend themselves to uncertainty estimation and subsequently how to quantify the model uncertainty at individual patient and voxel-wise levels for OPC GTVp segmentation.…”

Spatial uncertainty aggregation for false negatives detection in breast cancer metastases segmentation

“…A common approach has been to mark the most uncertain predictions for a manual review by a physician. 17,18 This strategy aims to reduce the workload of the medical doctors by enabling them to spend less time on the reliable predictions and focus on the complicated or unreliably-predicted cases. Alternatively, other researchers have argued for a manual visual review of generated uncertainty heatmaps that are overlayed on the original input image.…”

Diagnosis with confidence: deep learning for reliable classification of laryngeal dysplasia

“…For effective integration into pathologists' workflows, AI models should provide a confidence estimate for each prediction. While the concept of AI model uncertainty has been studied extensively in the past years, 27‐29 its application to computational pathology has been sparse 30,31 . Tempering the AI model's predictions with a measure of its confidence could help pathologists to better blend them into their routine grading, thereby fostering model acceptance.…”