Utilization of the evidence from studies with no events in meta-analyses of adverse events: an empirical investigation

Abstract

Backgrounds Zero-events studies frequently occur in systematic reviews of adverse events, which consist of an important source of evidence. We aimed to examine how evidence of zero-events studies was utilized in the meta-analyses of systematic reviews of adverse events. Methods We conducted a survey of systematic reviews published in two periods: January 1, 2015, to January 1, 2020, and January 1, 2008, to April 25, 2011. Databases were searched for systematic reviews that conducted at least one meta-analysis of any healthcare intervention and used adverse events as the exclusive outcome. An adverse event was defined as any untoward medical occurrence in a patient or subject in healthcare practice. We summarized the frequency of occurrence of zero-events studies in eligible systematic reviews and how these studies were dealt with in the meta-analyses of these systematic reviews. Results We included 640 eligible systematic reviews. There were 406 (63.45%) systematic reviews involving zero-events studies in their meta-analyses, among which 389 (95.11%) involved single-arm-zero-events studies and 223 (54.93%) involved double-arm-zero-events studies. The majority (98.71%) of these systematic reviews incorporated single-arm-zero-events studies into the meta-analyses. On the other hand, the majority (76.23%) of them excluded double-arm-zero-events studies from the meta-analyses, of which the majority (87.06%) did not discuss the potential impact of excluding such studies. Systematic reviews published at present (2015-2020) tended to incorporate zero-events studies in meta-analyses than those published in the past (2008-2011), but the difference was not significant (proportion difference=−0.09, 95% CI −0.21 to 0.03, p = 0.12). Conclusion Systematic review authors routinely treated studies with zero-events in both arms as “non-informative” carriers and excluded them from their reviews. Whether studies with no events are “informative” or not largely depends on the methods and assumptions applied, thus sensitivity analyses using different methods should be considered in future meta-analyses.

Validity of data extraction in evidence synthesis practice of adverse events: reproducibility study

“…Overall, we screened 18 636 records, and initially identified 456 systematic reviews of adverse events. 9 After a further screening of the full texts, 102 were excluded for having non-randomised studies of intervention and 153 were excluded for not having a pairwise meta-analysis, having fewer than five studies in all meta-analyses, or not reporting 2×2 table data used in meta-analyses (supplementary table 3). As such, 201 systematic reviews were included in the current study (fig 1).…”

“…This article is an extension of our previous work describing methods to deal with double-zero-event studies. 9 A protocol was drafted on 11 April 2021 by a group of core authors (CX, TY, LL, LFK), which was then revised after expert feedback (SV, LZ, RQ, and JZ; see supplementary file). We also record the detailed implementation of this study (supplementary table 1).…”

“…The search strategy was developed by an information specialist (supplementary box 1), and the literature search was conducted on 28 July 2020, and has been recorded elsewhere. 9 inclusion criteria and screening We included systematic reviews of randomised controlled trials for healthcare interventions, with adverse events as the exclusive outcome. The term adverse event was defined as "any untoward medical occurrence in a patient or subject in clinical practice," 10 which could be a side effect, adverse effect, adverse reaction, harm, or complication associated with any healthcare intervention.…”

Methodological quality for systematic reviews of adverse events with surgical interventions: a cross-sectional survey

“…The agreement rate was 0.74 in the full-text screening process. The details of excluded studies during the full-text screen process have been documented in our previous work [ 25 ]. According to the minimal requirement, the sample size of the current study could be sufficient.…”

“…The current study is one of the series of a project that aimed to improve the evidence-based practice of systematic reviews of adverse events. A protocol of the project was developed in advance ( supplementary file ), which also involves some context referred to the current study [ 25 ]. This study was designed, conducted, and reported follows the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement [ 26 ]; the following domains are not feasible for the current study: 1) Describe any sensitivity analyses (Reason: no need); 2) Indicate the number of participants with missing data for each variable of interest (Reason: no missing data).…”