Living network meta-analysis compared with pairwise meta-analysis in comparative effectiveness research: empirical study

Abstract

ObjectiveTo examine whether the continuous updating of networks of prospectively planned randomised controlled trials (RCTs) (“living” network meta-analysis) provides strong evidence against the null hypothesis in comparative effectiveness of medical interventions earlier than the updating of conventional, pairwise meta-analysis.DesignEmpirical study of the accumulating evidence about the comparative effectiveness of clinical interventions.Data sourcesDatabase of network meta-analyses of RCTs identified through searches of Medline, Embase, and the Cochrane Database of Systematic Reviews until 14 April 2015.Eligibility criteria for study selectionNetwork meta-analyses published after January 2012 that compared at least five treatments and included at least 20 RCTs. Clinical experts were asked to identify in each network the treatment comparison of greatest clinical interest. Comparisons were excluded for which direct and indirect evidence disagreed, based on side, or node, splitting test (P<0.10).Outcomes and analysisCumulative pairwise and network meta-analyses were performed for each selected comparison. Monitoring boundaries of statistical significance were constructed and the evidence against the null hypothesis was considered to be strong when the monitoring boundaries were crossed. A significance level was defined as α=5%, power of 90% (β=10%), and an anticipated treatment effect to detect equal to the final estimate from the network meta-analysis. The frequency and time to strong evidence was compared against the null hypothesis between pairwise and network meta-analyses.Results49 comparisons of interest from 44 networks were included; most (n=39, 80%) were between active drugs, mainly from the specialties of cardiology, endocrinology, psychiatry, and rheumatology. 29 comparisons were informed by both direct and indirect evidence (59%), 13 by indirect evidence (27%), and 7 by direct evidence (14%). Both network and pairwise meta-analysis provided strong evidence against the null hypothesis for seven comparisons, but for an additional 10 comparisons only network meta-analysis provided strong evidence against the null hypothesis (P=0.002). The median time to strong evidence against the null hypothesis was 19 years with living network meta-analysis and 23 years with living pairwise meta-analysis (hazard ratio 2.78, 95% confidence interval 1.00 to 7.72, P=0.05). Studies directly comparing the treatments of interest continued to be published for eight comparisons after strong evidence had become evident in network meta-analysis.ConclusionsIn comparative effectiveness research, prospectively planned living network meta-analyses produced strong evidence against the null hypothesis more often and earlier than conventional, pairwise meta-analyses.

Systemic immunomodulatory treatments for atopic dermatitis: protocol for a systematic review with network meta-analysis

“…A recent study concluded that living network meta-analyses with continuous updating produce strong, timely evidence of comparative effectiveness 37. The research questions in this systematic review are in line with proposed criteria for continuing a living systematic review, namely: (1) the systematic review is a priority for decision making; (2) new information will change decision making; and (3) there is likely to be, on an ongoing basis, new research relevant to decision making 38 39.…”

Preventive Treatments for Psychosis: Umbrella Review (Just the Evidence)

“…Cumulative meta-analysis, defined as updating a metaanalysis whenever a new eligible RCT becomes available, can be used to address this (39). However, the problem is that the median time taken for a primary study to be incorporated into a meta-analysis ranges from 2.5 to 6.5 years (40).…”

“…The feasibility and sustainability of living systematic reviews can be supported by the synergic interaction of human effort and machine automation (135). A recent empirical study has demonstrated that prospectively planned living network meta-analyses produced strong evidence against the null hypothesis more often-and earlier-than conventional pairwise meta-analyses (39).…”

A Markov Chain approach for ranking treatments in network meta-analysis

“…It should be noted that both ( ) and are dependent on the characteristics being considered, the medical field and the research question and therefore the criteria for their definition are specific to every NMA. Although such an approach seems to require additional time and resources than current practices in NMA, recent directions in evidence synthesis that encourage living systematic reviews and live cumulative network meta-analyses within the context of a research community are well compatible to our method [28][29][30][31][32] .…”