Protocol for a systematic review and meta-analysis of the diagnostic accuracy of artificial intelligence for grading of ophthalmology imaging modalities

Abstract

BackgroundWith the rise of artificial intelligence (AI) in ophthalmology, the need to define its diagnostic accuracy is increasingly important. The review aims to elucidate the diagnostic accuracy of AI algorithms in screening for all ophthalmic conditions in patient care settings that involve digital imaging modalities, using the reference standard of human graders.MethodsThis is a systematic review and meta-analysis. A literature search will be conducted on Ovid Medline, Ovid EMBASE, and Wiley Cochrane CENTRAL from January 1, 2000 to December 20, 2021. Studies will be selected via screening titles and abstracts, followed by full-text screening. Articles that compare the results of AI-graded ophthalmic images with results from human graders as a reference standard will be included. Systematic review software DistillerSR will be used to automate part of the screening process as an adjunct to human reviewers. After full text screening, data will be extracted from each study via the categories of study characteristics, patient information, AI methods, intervention, and outcomes. Risk of bias will be scored using Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) by two trained independent reviewers. Disagreements at any step will be addressed by a third adjudicator. Study results will include summary receiver operating characteristic (sROC) curve plots as well as pooled sensitivity and specificity of artificial intelligence for detection of any ophthalmic conditions based on imaging modalities compared to the reference standard. Statistics will be calculated in R statistical software.DiscussionThis study will provide novel insights on diagnostic accuracy of AI in new domains of ophthalmology that have not been previously studied. The protocol also outlines the use of an AI-based software to assist article screening, which may serve as a reference for improving efficiency and accuracy of future large systematic reviews.Systematic Review Registration: This study is registered on PROSPERO (CRD42021274441).