Ranking treatments in frequentist network meta-analysis works without resampling methods

Abstract

BackgroundNetwork meta-analysis is used to compare three or more treatments for the same condition. Within a Bayesian framework, for each treatment the probability of being best, or, more general, the probability that it has a certain rank can be derived from the posterior distributions of all treatments. The treatments can then be ranked by the surface under the cumulative ranking curve (SUCRA). For comparing treatments in a network meta-analysis, we propose a frequentist analogue to SUCRA which we call P-score that works without resampling.MethodsP-scores are based solely on the point estimates and standard errors of the frequentist network meta-analysis estimates under normality assumption and can easily be calculated as means of one-sided p-values. They measure the mean extent of certainty that a treatment is better than the competing treatments.ResultsUsing case studies of network meta-analysis in diabetes and depression, we demonstrate that the numerical values of SUCRA and P-Score are nearly identical.ConclusionsRanking treatments in frequentist network meta-analysis works without resampling. Like the SUCRA values, P-scores induce a ranking of all treatments that mostly follows that of the point estimates, but takes precision into account. However, neither SUCRA nor P-score offer a major advantage compared to looking at credible or confidence intervals.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-015-0060-8) contains supplementary material, which is available to authorized users.

Network meta-analysis of randomized trials in multiple myeloma: efficacy and safety in relapsed/refractory patients

“…All regimens were ranked by using the P-score according to the method recently described by Rücker et al 20 We observed similar results in terms of all variables (supplemental Figure 9B) and the daratumumab 1 lenalidomidedexamethasone regimen was again found to be the highest scoring regimen (average P-score, 0.733).…”

“…The Bayesian NMA was performed with STATA software by using the mvmeta package, and the frequentist NMA was performed with R software by using the netmeta package. 20 NMA synthesizes data from a network of trials that involve multiple interventions and therefore has the potential to rank the treatments according to the outcome. This method integrates direct and indirect comparisons.…”

“…The frequentist equivalent of the SUCRA, the P-score, was further used to confirm the results of the NMA. 20 Figure 1 shows a PRISMA chart of RCT selection and search strategy. In the time frame covered by this systematic review, a total of 1679 studies, including full articles or meeting abstracts, were retrieved.…”

Bibliographic study showed improving statistical methodology of network meta-analyses published between 1999 and 2015

“…We recorded whether all possible relative effects were presented (in the main text or as supplementary material) or only a subset of them. We also examined whether the authors presented the estimated hierarchy of the included interventions and which measure was used for this purpose (such as probability of being the best or surface under the cumulative ranking (SUCRA) curve [11,43,44] [46]. We also investigated any associations between methods employed and the complexity of the evidence-base such as the number of nodes and the M A N U S C R I P T…”

Additional considerations are required when preparing a protocol for a systematic review with multiple interventions

“…In such cases, unlike standard systematic review protocols, protocols including NMA should specify a priori the measure that will be used to rank the competing interventions (e.g. cumulative ranking curves, SUCRAs, mean ranks, or median ranks) as well as how the uncertainty of ranking will be reflected in the conclusions (47,48). For example, graphical tools such as the rankograms reflect visually the uncertainty in the ranking probabilities (38,49,50).…”