External Validation of a Retinopathy of Prematurity Screening Model Using Artificial Intelligence in 3 Low- and Middle-Income Populations

Coyner, Aaron S.; Shah, Parag K; Singh, Praveer; Ostmo, Susan; Valikodath, Nita; Cole, Emily; Al-Khaled, Tala; Bajimaya, Sanyam; Sagun, K C; Chuluunbat, Tsengelmaa; Munkhuu, Bayalag; Subramanian, Prema; Narendran, Venkatapathy; Jonas, Karyn; Hallak, Joelle; Chan, Robison Vernon Paul; Chiang, Michael F.; Kalpathy–Cramer, Jayashree; Campbell, J. Peter

doi:10.1001/jamaophthalmol.2022.2135

Cited by 27 publications

(35 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pilot projects in Kathmandu and Nepal in ROP screening were conducted, followed by expansion for its used at Aravind Eye Hospital (Coimbatore, India) for both telemedicine and telescreening of ROP, and has promising utility in the adoption of artificial intelligence-assisted screening programs. [19][20][21][22] The current screening guidelines utilized in Mongolia are gestational age <34 weeks and birth weight <2000 g, which are evidence-based guidelines developed from this screening program. In the Mongolian cohort, 18 infants (9.3%), including 8 with type 1, were outside of US screening guidelines, demonstrating that guidelines must be specific to the region in which the screening takes place.…”

Section: Case Study: Development Of a Rop Program In Mongoliamentioning

confidence: 99%

Research Partnerships and Guidance in Academic Global Ophthalmology

Ehrlich¹,

Shantha²,

Mathenge³

et al. 2022

International Ophthalmology Clinics

Self Cite

View full text Add to dashboard Cite

AAO) Task Force on Academic Global Ophthalmology was convened to catalyze development of AAO guidance and perspectives related to clinical service, education, and research initiatives within the field of Global Ophthalmology. The term "Global Ophthalmology" represents an expansive, cross-cutting, yet growing field with initiatives that may range from teaching to clinical service, to research facilitated through in-country partnerships. This article highlights the why, what, and how related to research and investigation in international settings. While much of this article focuses

show abstract

Section: Case Study: Development Of a Rop Program In Mongoliamentioning

confidence: 99%

Research Partnerships and Guidance in Academic Global Ophthalmology

Ehrlich¹,

Shantha²,

Mathenge³

et al. 2022

International Ophthalmology Clinics

Self Cite

View full text Add to dashboard Cite

show abstract

“…The prediction model included 2 parameters: gestational age and an AI-derived vascular severity score, based on fundus images collected from each infant during the first examination after 30 weeks postmenstrual age. The model demonstrated a sensitivity of 100% (no missed cases) in all 3 validation cohorts with a corresponding 38.4% to 51.3% projected reduction in the required screening examinations of low-risk infants …”

mentioning

confidence: 95%

“…Screening for retinopathy of prematurity (ROP) is a promising application of AI-based prediction models that will require thoughtful strategies for implementation. In this issue of JAMA Ophthalmology , Coyner et al describe the validation of a risk prediction model incorporating an AI-based assessment of vascular severity to improve screening efforts for treatment-requiring ROP. Building on prior work in US-based cohorts, the authors demonstrate generalizability of their modeling approach in 3 Asian low- and middle-income countries (LMICs) …”

mentioning

confidence: 99%

“…ROP screening in LMICs is complicated not only by the large numbers of infants requiring screening, but also by physician and resource shortages. To address differences in disease presentation patterns between the US and LMICs, the prediction model was retrained on a cohort of preterm infants from India (n = 2351) and externally validated in 3 cohorts from India (n = 762), Mongolia (n = 317), and Nepal (n = 330) …”

mentioning

confidence: 99%

“…Of machine learning articles published between 2015 and 2019, patient cohorts used to train algorithms for image-based diagnostic tasks were disproportionately from 3 states (California, Massachusetts, and New York) with minimal representation of the remaining parts of the country . Although the prediction model in the Coyner study was validated in LMICs, the underlying algorithm to calculate the vascular severity score was trained on a large multi-institution image bank, which may not be representative of the vascular patterns of the newborn LMIC population being screened. Though likely of minimal clinical importance, this example illustrates how unintended bias could potentially enter into an algorithm and supports increasing transparency of prediction models in health care .…”

mentioning

confidence: 99%

See 2 more Smart Citations

The Pursuit of Generalizability and Equity Through Artificial Intelligence-Based Risk Prediction Models

Oke

2022

JAMA Ophthalmol

View full text Add to dashboard Cite

show abstract

Efficacy of Smartphone-Based Telescreening for Retinopathy of Prematurity With and Without Artificial Intelligence in India

et al. 2023

Self Cite

View full text Add to dashboard Cite

ImportanceRetinopathy of prematurity (ROP) telemedicine screening programs have been found to be effective, but they rely on widefield digital fundus imaging (WDFI) cameras, which are expensive, making them less accessible in low- to middle-income countries. Cheaper, smartphone-based fundus imaging (SBFI) systems have been described, but these have a narrower field of view (FOV) and have not been tested in a real-world, operational telemedicine setting.ObjectiveTo assess the efficacy of SBFI systems compared with WDFI when used by technicians for ROP screening with both artificial intelligence (AI) and human graders.Design, Setting, and ParticipantsThis prospective cross-sectional comparison study took place as a single-center ROP teleophthalmology program in India from January 2021 to April 2022. Premature infants who met normal ROP screening criteria and enrolled in the teleophthalmology screening program were included. Those who had already been treated for ROP were excluded.ExposuresAll participants had WDFI images and from 1 of 2 SBFI devices, the Make-In-India (MII) Retcam or Keeler Monocular Indirect Ophthalmoscope (MIO) devices. Two masked readers evaluated zone, stage, plus, and vascular severity scores (VSS, from 1-9) in all images. Smartphone images were then stratified by patient into training (70%), validation (10%), and test (20%) data sets and used to train a ResNet18 deep learning architecture for binary classification of normal vs preplus or plus disease, which was then used for patient-level predictions of referral warranted (RW)– and treatment requiring (TR)–ROP.Main Outcome and MeasuresSensitivity and specificity of detection of RW-ROP, and TR-ROP by both human graders and an AI system and area under the receiver operating characteristic curve (AUC) of grader-assigned VSS. Sensitivity and specificity were compared between the 2 SBFI systems using Pearson χ2testing.ResultsA total of 156 infants (312 eyes; mean [SD] gestational age, 33.0 [3.0] weeks; 75 [48%] female) were included with paired examinations. Sensitivity and specificity were not found to be statistically different between the 2 SBFI systems. Human graders were effective with SBFI at detecting TR-ROP with a sensitivity of 100% and specificity of 83.49%. The AUCs with grader-assigned VSS only were 0.95 (95% CI, 0.91-0.99) and 0.96 (95% CI, 0.93-0.99) for RW-ROP and TR-ROP, respectively. For the AI system, the sensitivity of detecting TR-ROP sensitivity was 100% with specificity of 58.6%, and RW-ROP sensitivity was 80.0% with specificity of 59.3%.Conclusions and RelevanceIn this cross-sectional study, 2 different SBFI systems used by technicians in an ROP screening program were highly sensitive for TR-ROP. SBFI systems with AI may be a cost-effective method to improve the global capacity for ROP screening.

show abstract

External Validation of a Retinopathy of Prematurity Screening Model Using Artificial Intelligence in 3 Low- and Middle-Income Populations

Cited by 27 publications

References 39 publications

Research Partnerships and Guidance in Academic Global Ophthalmology

Research Partnerships and Guidance in Academic Global Ophthalmology

The Pursuit of Generalizability and Equity Through Artificial Intelligence-Based Risk Prediction Models

Efficacy of Smartphone-Based Telescreening for Retinopathy of Prematurity With and Without Artificial Intelligence in India

Contact Info

Product

Resources

About