Development and validation of a new clinical decision support tool to optimize screening for retinopathy of prematurity

Abstract

Background/AimsPrematurely born infants undergo costly, stressful eye examinations to uncover the small fraction with retinopathy of prematurity (ROP) that needs treatment to prevent blindness. The aim was to develop a prediction tool (DIGIROP-Screen) with 100% sensitivity and high specificity to safely reduce screening of those infants not needing treatment. DIGIROP-Screen was compared with four other ROP models based on longitudinal weights.MethodsData, including infants born at 24–30 weeks of gestational age (GA), for DIGIROP-Screen development (DevGroup, N=6991) originate from the Swedish National Registry for ROP. Three international cohorts comprised the external validation groups (ValGroups, N=1241). Multivariable logistic regressions, over postnatal ages (PNAs) 6–14 weeks, were validated. Predictors were birth characteristics, status and age at first diagnosed ROP and essential interactions.ResultsROP treatment was required in 287 (4.1%)/6991 infants in DevGroup and 49 (3.9%)/1241 in ValGroups. To allow 100% sensitivity in DevGroup, specificity at birth was 53.1% and cumulatively 60.5% at PNA 8 weeks. Applying the same cut-offs in ValGroups, specificities were similar (46.3% and 53.5%). One infant with severe malformations in ValGroups was incorrectly classified as not needing screening. For all other infants, at PNA 6–14 weeks, sensitivity was 100%. In other published models, sensitivity ranged from 88.5% to 100% and specificity ranged from 9.6% to 45.2%.ConclusionsDIGIROP-Screen, a clinical decision support tool using readily available birth and ROP screening data for infants born GA 24–30 weeks, in the European and North American populations tested can safely identify infants not needing ROP screening. DIGIROP-Screen had equal or higher sensitivity and specificity compared with other models. DIGIROP-Screen should be tested in any new cohort for validation and if not validated it can be modified using the same statistical approaches applied to a specific clinical setting.

Early prediction of severe retinopathy of prematurity requiring laser treatment using physiological data

“…Recently, the DIGIROP-Birth and DIGIROP-Screen have been developed to predict ROP requiring treatment using birth demographics and ROP progression data. 17,28 The DIGIROP-Birth probability in our data was significantly higher in the laser group compared to the non-laser group. In initial model development we aimed to achieve high accuracy; however, clinically it is essential that no infants requiring treatment are missed.…”

“…Varying the probability threshold of our model enabled us to achieve 100% sensitivity, with a specificity of 39%, similar to the results of DIGIROP-Screen at birth, and similar or higher compared to former developed models including the WINROP, G-ROP criteria, and a model based on Swiss data. [28][29][30][31] As physiological data from only the first 30 days after birth were included in our model, the number of infants who are unnecessarily screened could already be reduced by 40% prior to the first screening. We speculate that combining the physiological data with the ROP progression data could further improve prediction, especially when using photographic documentation and telemedicine.…”

Validation of DIGIROP models and decision support tool for prediction of treatment for retinopathy of prematurity on a contemporary Swedish cohort

“…GA-specific cut-offs, constructed based on the model development cohort to achieve 100% sensitivity, were used for validation of the clinical decision support tool. 9 The model's generalisability/transportability on this Swedish contemporary validation cohort was evaluated by sensitivity, specificity, cumulative specificity (along the screening for DIGIROP-Screen), positive predictive value (PPV), negative predictive value (NPV), model accuracy and area under the receiver operating characteristic (ROC) curve (AUC) with 95% CI. Cumulative specificity is the proportion of infants suggested by the tool to not require screening (at the current time point or any time before) who did not actually require treatment.…”

“…The specificity was 50% at birth and increased up to 79% cumulatively during the screening (37%-73% among infants born at GA <30 weeks), compared with 53%-81% obtained for the 6991 infants included in the development cohort, and 46%-75% in the external validation performed on a Swedish (n=314), German (n=322) and US (n=605) cohorts in the original publication. 9 Specificity in this study stands for the percentage of low-risk infants never requiring ROP treatment correctly identified not needing ROP screening. Achieving specificity of 50% using only birth characteristics (GA, BW and sex) is noteworthy, considering that about 30% of all screened Swedish prematurely born infants develop any ROP.…”

“…6 7 Others are Children's Hospital of Philadelphia ROP, the Colorado ROP model, Omaha ROP and the Postnatal Growth and ROP study (G-ROP). [8][9][10][11] Weight at specific postnatal days, however, is not always available to the screening ophthalmologists for all infants but necessary for the models noted above. Therefore, we sought to construct a ROP prediction tool using easily obtainable variables.…”