Evaluation of a deep learning image assessment system for detecting severe retinopathy of prematurity

Abstract

BackgroundPrior work has demonstrated the near-perfect accuracy of a deep learning retinal image analysis system for diagnosing plus disease in retinopathy of prematurity (ROP). Here we assess the screening potential of this scoring system by determining its ability to detect all components of ROP diagnosis.MethodsClinical examination and fundus photography were performed at seven participating centres. A deep learning system was trained to detect plus disease, generating a quantitative assessment of retinal vascular abnormality (the i-ROP plus score) on a 1–9 scale. Overall ROP disease category was established using a consensus reference standard diagnosis combining clinical and image-based diagnosis. Experts then ranked ordered a second data set of 100 posterior images according to overall ROP severity.Results4861 examinations from 870 infants were analysed. 155 examinations (3%) had a reference standard diagnosis of type 1 ROP. The i-ROP deep learning (DL) vascular severity score had an area under the receiver operating curve of 0.960 for detecting type 1 ROP. Establishing a threshold i-ROP DL score of 3 conferred 94% sensitivity, 79% specificity, 13% positive predictive value and 99.7% negative predictive value for type 1 ROP. There was strong correlation between expert rank ordering of overall ROP severity and the i-ROP DL vascular severity score (Spearman correlation coefficient=0.93; p<0.0001).ConclusionThe i-ROP DL system accurately identifies diagnostic categories and overall disease severity in an automated fashion, after being trained only on posterior pole vascular morphology. These data provide proof of concept that a deep learning screening platform could improve objectivity of ROP diagnosis and accessibility of screening.

Automated diagnosis and quantitative analysis of plus disease in retinopathy of prematurity based on deep convolutional neural networks

“…Redd et al. () applied the same network used by Brown et al. to study ROP (instead of Plus) and developed an index to evaluate the severeness of the ROP.…”

Artificial Intelligence and Ophthalmology

“…In an application developed by Redd et al 51 based on the same deep learning technology, the software was found to have 0.96 and 0.91 area under the curve values, respectively, in the identification of type 1 ROP and clinically significant ROP.…”

Telemedicine for Retinopathy of Prematurity

“…More recently, deep learning, where CBIA systems have been trained to automatically recognize and evaluate images, has been used for ROP screening. 59,60 Deep learning allows the system to continually learn and re-evaluate its process autonomously and consists of multiple layers of algorithms that data flow through to form a neural networks. 60 Convolutional neural networks have to be trained through exposure to a large number and variety of pathological and normal images to then apply a series of filters to produce the desired output, which in this case would be diagnosis or classification of ROP.…”

“…After analysis of 4,861 images, they found that the system could accurately detect clinically significant ROP with 94% sensitivity and a 99.7% negative predictive value based on posterior pole fundus photographs alone. 59 Wang et al also developed two deep neural networks, Id-Net and Gr-Net, which were, respectively, designed for the identification and grading of ROP. 62 Id-Net achieved a sensitivity of 96.62% and specificity of 99.32% for identification of any ROP and Gr-Net achieved 88.46% sensitivity and 92.31% specificity for grading of ROP severity, which was comparable with three expert graders.…”