Accuracy of Artificial Intelligence in Estimating Best-Corrected Visual Acuity From Fundus Photographs in Eyes With Diabetic Macular Edema

Abstract

ImportanceBest-corrected visual acuity (BCVA) is a measure used to manage diabetic macular edema (DME), sometimes suggesting development of DME or consideration of initiating, repeating, withholding, or resuming treatment with anti–vascular endothelial growth factor. Using artificial intelligence (AI) to estimate BCVA from fundus images could help clinicians manage DME by reducing the personnel needed for refraction, the time presently required for assessing BCVA, or even the number of office visits if imaged remotely.ObjectiveTo evaluate the potential application of AI techniques for estimating BCVA from fundus photographs with and without ancillary information.Design, Setting, and ParticipantsDeidentified color fundus images taken after dilation were used post hoc to train AI systems to perform regression from image to BCVA and to evaluate resultant estimation errors. Participants were patients enrolled in the VISTA randomized clinical trial through 148 weeks wherein the study eye was treated with aflibercept or laser. The data from study participants included macular images, clinical information, and BCVA scores by trained examiners following protocol refraction and VA measurement on Early Treatment Diabetic Retinopathy Study (ETDRS) charts.Main OutcomesPrimary outcome was regression evaluated by mean absolute error (MAE); the secondary outcome included percentage of predictions within 10 letters, computed over the entire cohort as well as over subsets categorized by baseline BCVA, determined from baseline through the 148-week visit.ResultsAnalysis included 7185 macular color fundus images of the study and fellow eyes from 459 participants. Overall, the mean (SD) age was 62.2 (9.8) years, and 250 (54.5%) were male. The baseline BCVA score for the study eyes ranged from 73 to 24 letters (approximate Snellen equivalent 20/40 to 20/320). Using ResNet50 architecture, the MAE for the testing set (n = 641 images) was 9.66 (95% CI, 9.05-10.28); 33% of the values (95% CI, 30%-37%) were within 0 to 5 letters and 28% (95% CI, 25%-32%) within 6 to 10 letters. For BCVA of 100 letters or less but more than 80 letters (20/10 to 20/25, n = 161) and 80 letters or less but more than 55 letters (20/32 to 20/80, n = 309), the MAE was 8.84 letters (95% CI, 7.88-9.81) and 7.91 letters (95% CI, 7.28-8.53), respectively.Conclusions and RelevanceThis investigation suggests AI can estimate BCVA directly from fundus photographs in patients with DME, without refraction or subjective visual acuity measurements, often within 1 to 2 lines on an ETDRS chart, supporting this AI concept if additional improvements in estimates can be achieved.

Study on the efficacy of different injection regimens of aflibercept in the treatment of diabetic macular edema

“…Therefore, the most important goal of DME treatment is to maximize the maintenance and protection of the patient's vision. Previous studies have shown that diabetes can lead to retinal ischemia and hypoxia in patients, promote the growth of vascular endothelial growth factor (VEGF) and then increase the permeability of the vascular wall, as well as induce neovascularization, thereby leading to the damage of the inner and outer retinal barrier, and the occurrence of DME [12].…”

Artificial Intelligence for Visual Acuity—Gaps From Algorithm to Actualization

“…In parallel with growing awareness of these technologies is increasingly nuanced knowledge regarding potential limitations and considerations for clinical implementation. In this issue of JAMA Ophthalmology , Paul et al describe the development of an AI algorithm for estimating best-corrected visual acuity (BCVA) from color fundus photographs among eyes with diabetic macular edema (DME) using data from the VISTA clinical trial that compared efficacy and safety of intravitreal aflibercept injection vs macular laser photocoagulation for DME over 3 years. The rationale is to potentially decrease the need for manual refraction and visual acuity measurement that can often be time-consuming and effort intensive.…”

“…The AI algorithms described in this study tended to perform worse among patients with poorer visual acuities. The authors hypothesized that this may have been due to a relatively small number of eyes with poorer visual acuities represented in the training data. AI algorithms, especially deep-learning algorithms like the ones used in this particular study, are data hungry and often require large amounts of training data to achieve high performance.…”

“…Another illustrative point is the concern for generalizability. In this study, there is a lack of racial and ethnic diversity among the data used to train the algorithm, given that the study cohort was over 80% White. This is not unique to this study and represents a more pervasive issue among clinical trials in general.…”

“…Adoption of telemedicine increased substantially during the COVID-19 pandemic, but adoption rates long-term in the postpandemic era remain an open question, particularly in a landscape of evolving regulatory and reimbursement frameworks. The promise of this type of algorithm would be to offer potential gains in efficiency in both clinic and home settings and the authors should be commended for pushing the boundaries in applying AI in ophthalmology. However, before this potential can be realized, ongoing work will be needed in this and other AI studies for application in clinical care to ensure that the algorithm has the ability to recognize when predictions may be inaccurate, can perform well in a wide range of disease severities, and produces consistent results among diverse populations.…”