The Impact of Excluding Trials from Network Meta-Analyses – An Empirical Study

Zhang, Jing; Yuan, Yiping; Chu, Haitao

doi:10.1371/journal.pone.0165889

Cited by 19 publications

(13 citation statements)

References 46 publications

(109 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Contrast-based models currently represent the most popular methodology in the network meta-analysis literature. However, another approach, called armbased, has also been recently advanced (see [35][36][37][38]). In contrast-based models, a baseline treatment is defined for each study and the focus of the analysis is on the estimation of the relative treatment effects (for example using log odds ratios, or another suitable metric).…”

Section: Introductionmentioning

confidence: 99%

A Bayesian approach to discrete multiple outcome network meta-analysis

Graziani

Venturini

2020

PLoS ONE

View full text Add to dashboard Cite

In this paper we suggest a new Bayesian approach to network meta-analysis for the case of discrete multiple outcomes. The joint distribution of the discrete outcomes is modeled through a Gaussian copula with binomial marginals. The remaining elements of the hierarchial random effects model are specified in a standard way, with the logit of the success probabilities given by the sum of a baseline log-odds and random effects comparing the logodds of each treatment against the reference and having a Gaussian distribution centered at the vector of pooled effects. An adaptive Markov Chain Monte Carlo algorithm is devised for running posterior inference. The model is applied to two datasets from Cochrane reviews, already analysed in two papers so to assess and compare its performance. We implemented the model in a freely available R package called netcopula.

show abstract

Section: Introductionmentioning

confidence: 99%

A Bayesian approach to discrete multiple outcome network meta-analysis

Graziani

Venturini

2020

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…Further, treatment ranks that are based on the probability of ranking best may be biased and influenced by the removal of treatments from an NMA 16 17. In addition, the removal of a study can impact ranking probabilities and ranks based on the probability of ranking best 18 19. Since ranking probabilities contribute to the calculation of SUCRA, which is often estimated with large uncertainty,12 yet is increasingly being used in published NMAs,20 it is of interest to examine the robustness of SUCRA-based treatment ranks and to quantify sensitivity with respect to evidence contributed by individual studies.…”

Section: Introductionmentioning

confidence: 99%

Empirical evaluation of SUCRA-based treatment ranks in network meta-analysis: quantifying robustness using Cohen’s kappa

et al. 2019

View full text Add to dashboard Cite

ObjectiveTo provide a framework for quantifying the robustness of treatment ranks based on Surface Under the Cumulative RAnking curve (SUCRA) in network meta-analysis (NMA) and investigating potential factors associated with lack of robustness.MethodsWe propose the use of Cohen’s kappa to quantify the agreement between SUCRA-based treatment ranks estimated through NMA of a complete data set and a subset of it. We illustrate our approach using five published NMA data sets, where robustness was assessed by removing studies one at a time.ResultsOverall, SUCRA-based treatment ranks were robust to individual studies in the five data sets we considered. We observed more incidences of disagreement between ranks in the networks with larger numbers of treatments. Most treatments moved only one or two ranks up or down. The lowest quadratic weighted kappa estimate observed across all networks was in the network with the smallest number of treatments (4), where weighted kappa=40%. In the network with the largest number of treatments (12), the lowest observed quadratic weighted kappa=89%, reflecting a small shift in this network's treatment ranks overall. Preliminary observations suggest that a study’s size, the number of studies making a treatment comparison, and the agreement of a study’s estimated treatment effect(s) with those estimated by other studies making the same comparison(s) may explain the overall robustness of treatment ranks to studies.ConclusionsInvestigating robustness or sensitivity in an NMA may reveal outlying rank changes that are clinically or policy-relevant. Cohen’s kappa is a useful measure that permits investigation into study characteristics that may explain varying sensitivity to individual studies. However, this study presents a framework as a proof of concept and further investigation is required to identify potential factors associated with the robustness of treatment ranks using more extensive empirical evaluations.

show abstract

“…However, the AB framework does offer advantages in ease of model fitting and interpretation when single-arm studies are present 15 . The pros and cons of the arm-based and contrast-based frameworks have been compared in published works 17181920162122 and will also be discussed in this paper. All the three models proposed are expected to gain precision by including single-arm studies compared with a meta-analysis of solely RCTs.…”

Section: Introductionmentioning

confidence: 99%

Bayesian hierarchical methods for meta-analysis combining randomized-controlled and single-arm studies

Zhang

Chen

et al. 2018

Stat Methods Med Res

Self Cite

View full text Add to dashboard Cite

Meta-analysis of interventions usually relies on randomized controlled trials. However, when the dominant source of information comes from single-arm studies, or when the results from randomized controlled trials lack generalization due to strict inclusion and exclusion criteria, it is vital to synthesize both sources of evidence. One challenge of synthesizing both sources is that single-arm studies are usually less reliable than randomized controlled trials due to selection bias and confounding factors. In this paper, we propose a Bayesian hierarchical framework for the purpose of bias reduction and efficiency gain. Under this framework, three methods are proposed: bivariate generalized linear mixed effects models, hierarchical power prior model and hierarchical commensurate prior model. Design difference and potential biases are considered in all models, within which the hierarchical power prior and hierarchical commensurate prior models further offer to downweight single-arm studies flexibly. The hierarchical commensurate prior model is recommended as the primary method for evidence synthesis because of its accuracy and robustness. We illustrate our methods by applying all models to two motivating datasets and evaluate their performance through simulation studies. We finish with a discussion of the advantages and limitations of our methods, as well as directions for future research in this area.

show abstract

The Impact of Excluding Trials from Network Meta-Analyses – An Empirical Study

Cited by 19 publications

References 46 publications

A Bayesian approach to discrete multiple outcome network meta-analysis

A Bayesian approach to discrete multiple outcome network meta-analysis

Empirical evaluation of SUCRA-based treatment ranks in network meta-analysis: quantifying robustness using Cohen’s kappa

Bayesian hierarchical methods for meta-analysis combining randomized-controlled and single-arm studies

Contact Info

Product

Resources

About