Screening Referable Diabetic Retinopathy Using a Semi-automated Deep Learning Algorithm Assisted Approach

Abstract

Purpose: To assess the accuracy and efficacy of a semi-automated deep learning algorithm (DLA) assisted approach to detect vision-threatening diabetic retinopathy (DR).Methods: We developed a two-step semi-automated DLA-assisted approach to grade fundus photographs for vision-threatening referable DR. Study images were obtained from the Lingtou Cohort Study, and captured at participant enrollment in 2009–2010 (“baseline images”) and annual follow-up between 2011 and 2017. To begin, a validated DLA automatically graded baseline images for referable DR and classified them as positive, negative, or ungradable. Following, each positive image, all other available images from patients who had a positive image, and a 5% random sample of all negative images were selected and regraded by trained human graders. A reference standard diagnosis was assigned once all graders achieved consistent grading outcomes or with a senior ophthalmologist's final diagnosis. The semi-automated DLA assisted approach combined initial DLA screening and subsequent human grading for images identified as high-risk. This approach was further validated within the follow-up image datasets and its time and economic costs evaluated against fully human grading.Results: For evaluation of baseline images, a total of 33,115 images were included and automatically graded by the DLA. 2,604 images (480 positive results, 624 available other images from participants with a positive result, and 1500 random negative samples) were selected and regraded by graders. The DLA achieved an area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and accuracy of 0.953, 0.970, 0.879, and 88.6%, respectively. In further validation within the follow-up image datasets, a total of 88,363 images were graded using this semi-automated approach and human grading was performed on 8975 selected images. The DLA achieved an AUC, sensitivity, and specificity of 0.914, 0.852, 0.853, respectively. Compared against fully human grading, the semi-automated DLA-assisted approach achieved an estimated 75.6% time and 90.1% economic cost saving.Conclusions: The DLA described in this study was able to achieve high accuracy, sensitivity, and specificity in grading fundus images for referable DR. Validated against long-term follow-up datasets, a semi-automated DLA-assisted approach was able to accurately identify suspect cases, and minimize misdiagnosis whilst balancing safety, time, and economic cost.

Health Economic Implications of Artificial Intelligence Implementation for Ophthalmology in Australia: A Systematic Review

“…From a cost perspective, the total cost per retinal image assessed by AI was $0.017 versus $0.032 per image graded by humans. This result represents a 90.1% cost saving 24. A large sample size of 88,363 images was assessed, further validating the results.…”

“…This result represents a 90.1% cost saving. 24 A large sample size of 88,363 images was assessed, further validating the results. The minimal available CBA evidence on the topic indicates the use of AI in ophthalmology is an economically viable practice with efficacy comparable to human grading.…”

Emerging Trends and Research Foci in Artificial Intelligence for Retinal Diseases: Bibliometric and Visualization Study

“…In addition, AI technology has been applied to the screening, referral, diagnosis, health care, and follow-up visits of patients with a variety retinal illness. Wang et al [ 33 ] developed a two-step semiautomatic deep learning algorithm–assisted technique to identify fundus pictures and aid in the detection of DR with vision-threatening complications. Optical coherence tomographic (OCT) angiography is a noninvasive imaging technology that may generate angiograms at precise depths within the retina as well as visualize the microvasculature in real time [ 34 ].…”

Artificial Intelligence for Predicting and Diagnosing Complications of Diabetes

“…52,53 Recent studies, however, have shown poorer performance of these algorithms in real-world settings. 54 Researchers have also expressed reservations because of a lack of (1) detail in grading, 55 (2) grading of nondiabetic retinal conditions and glaucoma, and (3) an explanation of how the algorithm determines the retinal grade, which often incorporates multiple features from fundus imaging. This technique produces a two-dimensional image that represents the three-dimensional structure of the retina.…”