Expert-validated estimation of diagnostic uncertainty for deep neural networks in diabetic retinopathy detection

Ayhan, Murat Seçkin; Kühlewein, Laura; Aliyeva, Gulnar; Inhoffen, Werner; Ziemssen, Focke; Berens, Philipp

doi:10.1101/19002154

Cited by 43 publications

(64 citation statements)

References 43 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For comparative studies of existing uncertainty estimation methods, we choose VGGNet with 16 layers as our baseline DNN. We implemented SelectiveNet (Geifman and El-Yaniv 2019), TTAUG (Ayhan et al 2020) and DbFF (Sarker et al 2020) on the same baseline DNN to ensure fair comparison. Among these three methods, SelectiveNet requires most modification.…”

Section: Methodsmentioning

confidence: 99%

“…First, we present the experimental studies on the effectiveness of existing methods on the COVIDx dataset (Wang and Wong 2020). We compare state-of-the-art methods, Selec-tiveNet (Geifman and El-Yaniv 2019), TTAUG (Ayhan et al 2020) and DbFF (Sarker et al 2020), and report the results in Table 1. Please note, for fair comparison, we report Se-lectiveNet results when trained their model with 100% coverage and then calibrated to the desired abstention rate.…”

Section: Comparative Study Of Existing Methodsmentioning

confidence: 99%

“…However, most of these works only experimented with benchmark datasets ignoring the stochasticity associated with real-world scenarios. Though greatly understudied, some research works on uncertainty estimation, are curated for real-world healthcare datasets (Leibig et al 2017;Ayhan et al 2020). Despite that, no work till now addresses the issue of uncertainty on COVID detection.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Reducing Risk and Uncertainty of Deep Neural Networks on Diagnosing COVID-19 Infection

Sarker¹,

Pandit²,

Sarker³

et al. 2021

Preprint

View full text Add to dashboard Cite

Effective and reliable screening of patients via Computer-Aided Diagnosis can play a crucial part in the battle against COVID-19. Most of the existing works focus on developing sophisticated methods yielding high detection performance, yet not addressing the issue of predictive uncertainty. In this work, we introduce uncertainty estimation to detect confusing cases for expert referral to address the unreliability of stateof-the-art (SOTA) DNNs on COVID-19 detection. To the best of our knowledge, we are the first to address this issue on the COVID-19 detection problem. In this work, we investigate a number of SOTA uncertainty estimation methods on publicly available COVID dataset and present our experimental findings. In collaboration with medical professionals, we further validate the results to ensure the viability of the best performing method in clinical practice.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Comparative Study Of Existing Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Reducing Risk and Uncertainty of Deep Neural Networks on Diagnosing COVID-19 Infection

Sarker¹,

Pandit²,

Sarker³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…networks that are inher-ently well calibrated, for the task of diabetic retinopathy classification on both RDR and PIRC classifications. For this reason we leave out the analysis of post-hoc methods, such as the test-time augmentation introduced in Ayhan and Berens [11] and Ayhan et al [19], neural network softmax temperature scaling, and probability binning strategies [4]. Many measures of uncertainty have been used in the previous works.…”

Section: Mainmentioning

confidence: 99%

“…In Leibig et al [9], the standard deviation of the output of the BNN and the entropy of the posterior predictive distribution were considered as measures of uncertainty and found to perform similarly. In Filos et al [12], Ayhan et al [19], and Band et al [18] the entropy was also selected as the measure of uncertainty. On the other hand, the mutual information between the parameters of the model and the output was considered as the measure of uncertainty in Farquhar et al [10].…”

Section: Mainmentioning

confidence: 99%

Uncertainty-aware deep learning methods for robust diabetic retinopathy classification

Jaskari¹,

Sahlsten²,

Damoulas³

et al. 2022

Preprint

View full text Add to dashboard Cite

Automatic classification of diabetic retinopathy from retinal images has been widely studied using deep neural networks with impressive results. However, there is a clinical need for estimation of the uncertainty in the classifications, a shortcoming of modern neural networks. Recently, approximate Bayesian deep learning methods have been proposed for the task but the studies have only considered the binary referable/non-referable diabetic retinopathy classification applied to benchmark datasets. We present novel results by systematically investigating a clinical dataset and a clinically relevant 5-class classification scheme, in addition to benchmark datasets and the binary classification scheme. Moreover, we derive a connection between uncertainty measures and classifier risk, from which we develop a new uncertainty measure. We observe that the previously proposed entropy-based uncertainty measure generalizes to the clinical dataset on the binary classification scheme but not on the 5-class scheme, whereas our new uncertainty measure generalizes to the latter case.

show abstract

ESLO: Enhanced sea lion optimization based bi‐directional CNN‐RNN for accurate detection of diabetic retinopathy

Vinayaki

Kalaiselvi²

2022

Concurrency and Computation

View full text Add to dashboard Cite

Summary Diabetic retinopathy is considered to be one of the major causes of eye disorder which commonly occurs in working age group of people worldwide. Deficiency of proper screening and early diagnosing leads the diabetic patients completely affected by vision loss. Therefore, an effective detection of diabetic retinopathy before visual symptoms reduces the impact of visual impairment. In recent times, numerous research methods are adopted by researchers in this domain to automatically detect the existence of eye diseases; however, they provide only limited performances like less accuracy with high training error. Therefore, in this article, an attention based bi‐directional convolutional neural network‐recurrent neural network model along with enhanced sea lion optimization (ABCDM‐ESLO) technique is proposed for accurate detection and classification of healthy and unhealthy diabetic retinopathy cases based on the severity levels. In this, seven different datasets are used, analyzed, and classified into five categories (i.e., normal, mild, moderate, severe, and proliferative). Moreover, the performance metrics such as accuracy, sensitivity, specificity, and f‐score are evaluated with respect to severity levels thereby determining the efficiency of the proposed method. The performance of the proposed technique is examined by comparing it with other existing methods. The proposed method achieves 99.2% of detection accuracy, 94% of sensitivity, 96% of specificity, and 92.75% of f‐score. This reveals that the proposed ABCDM‐ESLO technique performs well and classifies than other existing methods in terms of different performance metrics.

show abstract

Expert-validated estimation of diagnostic uncertainty for deep neural networks in diabetic retinopathy detection

Cited by 43 publications

References 43 publications

Reducing Risk and Uncertainty of Deep Neural Networks on Diagnosing COVID-19 Infection

Reducing Risk and Uncertainty of Deep Neural Networks on Diagnosing COVID-19 Infection

Uncertainty-aware deep learning methods for robust diabetic retinopathy classification

ESLO: Enhanced sea lion optimization based bi‐directional CNN‐RNN for accurate detection of diabetic retinopathy

Contact Info

Product

Resources

About