A Call For Better Methodological Quality Of Reviews On Using Artificial Intelligence For COVID-19 Detection In Medical Imaging – An Umbrella Systematic Review

Abstract

Objective: In this umbrella systematic review, we screen existing reviews on using artificial intelligence(AI) techniques to diagnose COVID-19 in patients of any age and sex (both hospitalised and ambulatory) using medical images and assess their methodological quality. Methods: We searched seven databases (MEDLINE, EMBASE, Web of Science, Scopus, dblp, CochraneLibrary, IEEE Xplore) and two preprint services (arXiv, OSF Preprints) up to September 1, 2020. Eligible studies were identified as reviews or surveys where any metric of classification of detection of COVID-19 using AI was provided. Two independent reviewers did all steps of identification of records (titles and abstracts screening, full texts assessment, essential data extraction, and quality assessment). Any discrepancies were resolved by discussion. We qualitatively analyse methodological credibility ofthe reviews using AMSTAR 2 and evaluate reporting using PRISMA-DTA tools, leaving quantitative analysis for further publications. Results: We included 22 reviews out of 725 records covering 165 primary studies. This review covers 416,254 participants in total, including 50,022 diagnosed with COVID-19. The methodological quality of all eligible studies was rated as critically low. 91% of papers had significant flaws in reporting quality. More than half of the reviews did not comment on the results of previously published reviews at all. Almost three fourth of the studies included less than 10% of available studies. Discussion: In this umbrella review, we focus on the descriptive summary of included papers. Much wasting time and resources could be avoided if referring to previous reviews and following methodological guidelines. Due to the low credibility of evidence and flawed reporting, any recommendation about automated COVID-19 clinical diagnosis from medical images using AI at this point cannot be provided. Funding: PO was supported by NIH grant AI116794 (the funding body had no role in the design, in any stage of the review, or in writing the manuscript); PJ and DS did not receive any funding. Registration:The protocol of this review was registered on the OSF platform at osf.io/kxrmh