Uncertainty-Aware Deep Learning Methods for Robust Diabetic Retinopathy Classification

Abstract

Automatic classification of diabetic retinopathy from retinal images has been increasingly studied using deep neural networks with impressive results. However, there is clinical need for estimating uncertainty in the classifications, a shortcoming of modern neural networks. Recently, approximate Bayesian neural networks (BNNs) have been proposed for this task, but previous studies have only considered the binary referable/non-referable diabetic retinopathy classification applied to benchmark datasets. We present novel results for 9 BNNs by systematically investigating a clinical dataset and 5-class classification scheme, together with benchmark datasets and binary classification scheme. Moreover, we derive a connection between entropy-based uncertainty measure and classifier risk, from which we develop a novel uncertainty measure. We observe that the previously proposed entropy-based uncertainty measure improves performance on the clinical dataset for the binary classification scheme, but not to such an extent as on the benchmark datasets. It improves performance in the clinical 5-class classification scheme for the benchmark datasets, but not for the clinical dataset. Our novel uncertainty measure generalizes to the clinical dataset and to one benchmark dataset. Our findings suggest that BNNs can be utilized for uncertainty estimation in classifying diabetic retinopathy on clinical data, though proper uncertainty measures are needed to optimize the desired performance measure. In addition, methods developed for benchmark datasets might not generalize to clinical datasets.

Application of simultaneous uncertainty quantification for image segmentation with probabilistic deep learning: Performance benchmarking of oropharyngeal cancer target delineation as a use-case

“…In order to compute the measure, we need M ( M −1)/2 comparisons, which is 44850 comparisons for the MC Dropout Ensemble; we deemed the measure prohibitive to compute in practice. Inspired by the fusion of the Dice coefficient and the uncertainty estimation, we developed a novel uncertainty measure that we call Dice-risk , which is based on the expected conditional risk introduced in a recent work 47 . In this work, the authors noted that the entropy-based uncertainty measures can be interpreted as computing the average negative log-likelihood provided that the target is distributed as the network predictive distribution describes.…”

“…We also examined uncertainty-based referral simulation that is common in uncertainty-aware classification tasks 39,46,47 . In the batch referral process, each patient is assigned an uncertainty score using one of the uncertainty measures and the patients are sorted based on the score.…”

DR-GPT: a large language model for medical report analysis of diabetic retinopathy patients

“…The class distributions of ETDRS-T in patient, report, and eye -level are presented in Table 1. In addition, we have also evaluated the DR-GPT on a binary DR classification system (RDR) used in previous studies [5, 6, 7]. The RDR system is defined as moderate or worse diabetic retinopathy on the proposed international diabetic retinopathy classification (ICDR) system [8], with ICDR classes lower than moderate DR assigned the label 0 and moderate or worse the label 1.…”

“…We then resized each image to a standard resolution of 512 × 512. During the training of the convolutional neural networks, we utilized training augmentations based on recent literature [5, 7, 9], i.e., random spatial flips both vertically and horizontally (p=0.5), random rotations uniformly within the range of [ − 180 ° , 180 ° ], random translations within the range of [-25,25] pixels in both spatial axes, and random zooms within range [90%,110%]. Finally, the image pixel values were mapped to the range [-1,1], during both the training and inference.…”

“…To evaluate the classification performance of the five class ETDRS-T quantitatively, we use the quadratic weighted Cohen’s kappa (QWK) [17], accuracy, and balanced accuracy measures. The QWK measure has previously been used in the evaluation of the deep learning algorithms for diabetic retinopathy classification with the five-class ICDR system [7, 18, 19], whereas the accuracy and balanced accuracy are common classification evaluation measures. For the binary classification tasks of RDR and gradability classification, we use the area under the receiver operating characteristic curve (AUROC), accuracy, and balanced accuracy.…”