Performance of deep neural network-based artificial intelligence method in diabetic retinopathy screening: a systematic review and meta-analysis of diagnostic test accuracy

Abstract

Objective Automatic diabetic retinopathy screening system based on neural networks has been used to detect diabetic retinopathy (DR). However, there is no quantitative synthesis of performance of these methods. We aimed to estimate the sensitivity and specificity of neural networks in DR grading. Methods Medline, Embase, IEEE Xplore, and Cochrane Library were searched up to 23 July 2019. Studies that evaluated performance of neural networks in detection of moderate or worse DR or diabetic macular edema using retinal fundus images with ophthalmologists’ judgment as reference standard were included. Two reviewers extracted data independently. Risk of bias of eligible studies was assessed using QUDAS-2 tool. Results Twenty-four studies involving 235 235 subjects were included. Quantitative random-effects meta-analysis using the Rutter and Gatsonis hierarchical summary receiver operating characteristics (HSROC) model revealed a pooled sensitivity of 91.9% (95% CI: 89.6% to 94.3%) and specificity of 91.3% (95% CI: 89.0% to 93.5%). Subgroup analyses and meta-regression did not provide any statistically significant findings for the heterogeneous diagnostic accuracy in studies with different image resolutions, sample sizes of training sets, architecture of convolutional neural networks, or diagnostic criteria. Conclusions State-of-the-art neural networks could effectively detect clinical significant DR. To further improve diagnostic accuracy of neural networks, researchers might need to develop new algorithms rather than simply enlarge sample sizes of training sets or optimize image quality.

Facial Recognition Intensity in Disease Diagnosis Using Automatic Facial Recognition

“…Representative and high-quality studies focused on different diseases using various known AI methods and were conducted in different countries. Our study summarized and quantitatively analyzed heterogeneous studies on the automatic diagnosis of different diseases based on facial features, showing a pooled sensitivity of 89% (95% CI 82% to 93%) and a specificity of 92% (95% CI 87% to 95%), similar to the results of previous meta-analyses on automatic image recognition for diabetic retinopathy screening [8,33,34], colorectal neoplasia, and breast cancer [35][36][37][38], indicating a promising diagnostic performance of facial recognition based on AI for heterogeneous diseases. A sensitivity analysis was conducted to evaluate the robustness of the results.…”

“…Previous studies have demonstrated that the image characteristics of diseases play an important role in the performance of image recognition by AI methods [43], including the automatic screening of pulmonary nodules [7,44,45], referable glaucomatous optic neuropathy (GON) [46], colorectal adenoma and polyps [47,48], which also indicates that IRI describes image characteristics of diseases and is critical for AI performance in automatic image recognition. As has been shown before for diabetic retinopathy screening, no statistically significant contribution to heterogeneous diagnostic accuracy has been demonstrated for sample size of the training sets and architecture of convolutional neural networks [34]. Therefore, the complexity theory explains the relationship between complexity of a disease and AI performance and should be extended to other AI applications.…”

Diagnostic accuracy of teleretinal screening for detection of diabetic retinopathy and age-related macular degeneration: a systematic review and meta-analysis

“…Our study results are only applicable to teleretinal programmes using human graders. Recent diagnostic test accuracy meta-analyses have provided very promising accuracy estimates for machine-learning-based teleretinal screening programmes for DR. 38 39 Future studies should assess the diagnostic accuracy of automated systems using artificial intelligence and deep-learning algorithms in teleophthalmology screening programmes for ocular diseases. 40 Lastly, the focus of this review was on teleretinal screening for the most common retinal pathologies.…”

Are visual disturbances (excluding diabetic retinopathy) more common in geriatric DM patients? Are they risks factor for the progression of disability?

“…In the short-to medium-period, the clinical management of geriatric DM can be improved by initiating multidisciplinary, multi-specialist care networks, in which ophthalmologists and orthoptist/ophthalmic assistant are involved to develop: i) tele-medicine procedures for the screening of diabetic retinopathy and other diabetes-related eye disorders using either conventional or innovative AI-based approaches [37][38][39] ; ii) algorithms for personalized treatments of diabetic macular edema as well as other eye disorders frequently present in geriatric DM patients, analyzing their outcomes also using AI-equipped systems [40][41][42] ; iii) multidisciplinary rehabilitation programs that can meet the needs of the visually impaired DM persons among geriatric population [43][44][45][46] .…”