Scoping review of COVID-19-related systematic reviews and meta-analyses: can we really have confidence in their results?

Abstract

AimThe aim of this study was to systematically appraise the quality of a sample of COVID-19-related systematic reviews (SRs) and discuss internal validity threats affecting the COVID-19 body of evidence.DesignWe conducted a scoping review of the literature. SRs with or without meta-analysis (MA) that evaluated clinical data, outcomes or treatments for patients with COVID-19 were included.Main outcome measuresWe extracted quality characteristics guided by A Measurement Tool to Assess Systematic Reviews-2 to calculate a qualitative score. Complementary evaluation of the most prominent published limitations affecting the COVID-19 body of evidence was performed.ResultsA total of 63 SRs were included. The majority were judged as a critically low methodological quality. Most of the studies were not guided by a pre-established protocol (39, 62%). More than half (39, 62%) failed to address risk of bias when interpreting their results. A comprehensive literature search strategy was reported in most SRs (54, 86%). Appropriate use of statistical methods was evident in nearly all SRs with MAs (39, 95%). Only 16 (33%) studies recognised heterogeneity in the definition of severe COVID-19 as a limitation of the study, and 15 (24%) recognised repeated patient populations as a limitation.ConclusionThe methodological and reporting quality of current COVID-19 SR is far from optimal. In addition, most of the current SRs fail to address relevant threats to their internal validity, including repeated patients and heterogeneity in the definition of severe COVID-19. Adherence to proper study design and peer-review practices must remain to mitigate current limitations.

Potential limitations in systematic review studies assessing the effect of the main intervention for treatment/therapy of COVID-19 patients: An overview

“…In addition, Differences in disease definition and heterogeneity in studies are important factors influencing the results of these studies. Following existing guidelines and proper study design can be one of the factors reducing the limitations of these studies ( 66 , 67 ). Taken together, poor designs and various limitations of the studies render them ineffective in gaging the full extent of its safety and efficacy and thus warrant further research into the use of interventions in COVID-19 patient treatment.…”

Síndrome multisistémico inflamatorio pediátrico (MIS-C/PIMS): bases inmunológicas que sustentan el tratamiento

“…Si bien, algunas publicaciones tempranas consideraban pocos los beneficios cuantitativos, otros defienden los beneficios observados, y finalmente, aún más, se ha puesto en duda la validez de los resultados de revisiones sistemáticas y metaanálisis, por lo que la postura de algunos autores, incluyéndonos, los consideramos aún en proceso de investigación para definir el mejor escenario para considerar su uso. 24 Hasta el momento, en pocos pacientes con MIS-C /PIMS se ha utilizado el TCZ; la serie más grande reportada fue en Nueva York por Kaushik y colaboradores, quienes la aplicaron a 12 de 33 pacientes con niveles elevados de IL-6, los cuales fueron egresados por mejoría 32. 25 La experiencia en Latinoamerica reportada por Niño-Taravilla y cols, fue un caso con buena respuesta.…”

Are COVID-19 systematic reviews up to date and can we tell? A cross-sectional study

“…By May 2022, at least 4000 had been added to PubMed and over 7000 had been registered in PROSPERO. Several studies have identified deficiencies in the reliability of systematic reviews relevant to COVID-19 and highlighted less than optimal reporting [ 2 – 6 ].…”