Empirical comparison of univariate and multivariate meta‐analyses in Cochrane Pregnancy and Childbirth reviews with multiple binary outcomes

Abstract

Background Multivariate meta‐analysis (MVMA) jointly synthesizes effects for multiple correlated outcomes. The MVMA model is potentially more difficult and time‐consuming to apply than univariate models, so if its use makes little difference to parameter estimates, it could be argued that it is redundant. Methods We assessed the applicability and impact of MVMA in Cochrane Pregnancy and Childbirth (CPCB) systematic reviews. We applied MVMA to CPCB reviews published between 2011 and 2013 with two or more binary outcomes with at least three studies and compared findings with results of univariate meta‐analyses. Univariate random effects meta‐analysis models were fitted using restricted maximum likelihood estimation (REML). Results Eighty CPCB reviews were published. MVMA could not be applied in 70 of these reviews. MVMA was not feasible in three of the remaining 10 reviews because the appropriate models failed to converge. Estimates from MVMA agreed with those of univariate analyses in most of the other seven reviews. Statistical significance changed in two reviews: In one, this was due to a very small change in P value; in the other, the MVMA result for one outcome suggested that previous univariate results may be vulnerable to small‐study effects and that the certainty of clinical conclusions needs consideration. Conclusions MVMA methods can be applied only in a minority of reviews of interventions in pregnancy and childbirth and can be difficult to apply because of missing correlations or lack of convergence. Nevertheless, clinical and/or statistical conclusions from MVMA may occasionally differ from those from univariate analyses.

Mindfulness-based programmes for mental health promotion in adults in nonclinical settings: A systematic review and meta-analysis of randomised controlled trials

“…Multivariate meta–analyses for secondary outcome domains also included all available time point ranges, but data from our IPD were less suitable to estimate between–outcome within–study correlations and no IPD were available, so meta–analyses were outcome–specific. Within–study correlations were bypassed using Riley’s method as our IPD were unsuitable [ 66 ]; for cognitive functioning, this method had to be rejected due to extreme correlations, so a within–study correlation of 0.75 was imputed (high given the outcome–specific analysis) with a sensitivity analysis testing 0.5 [ 67 ]. Results of meta–analyses containing few studies were interpreted cautiously, including multivariate meta–analyses with outcomes derived from a single trial.…”

Multivariate meta-analysis of critical care meta-analyses: a meta-epidemiological study

“…The differences between the results of the current study and those referenced above [8,14,[89][90][91][92][93] needs some further explication with regard to data structure. The bi-and tri-variate meta-analyses under consideration were relatively conventional; a primary mortality outcome and second and third recorded outcomes which were not direct extensions of the primary outcome.…”

“…Frequentist and Bayesian empirical comparisons between univariate meta-analyses and MVMA have appeared in the literature [8,14,[89][90][91][92][93] with results demonstrating similar (pooled) parameter estimates between the two analytic forms. However, papers by Riley and co-workers [1,15,24,94], which included formal simulation studies, found advantage; a smaller standard error and mean-square error of pooled estimates, predicated upon the presence of missing data; again, assuming missing at random.…”

“…MVMA in such meta-analyses would allow joint inference upon multiple outcomes and be of relevance from a methodological and clinical viewpoint. Price et al suggested that where multiple outcomes routinely occur, MVMA would be “…more likely to have an impact” [ 8 ]. From a previous study which reported mortality outcome of a series of meta-analyses in the critically ill [ 9 ] utilising only randomised controlled trials, a meta-analytic cohort was identified where secondary outcomes were reported in such detail as to yield bivariate or tri-variate data structures.…”

“…From a previous study which reported mortality outcome of a series of meta-analyses in the critically ill [ 9 ] utilising only randomised controlled trials, a meta-analytic cohort was identified where secondary outcomes were reported in such detail as to yield bivariate or tri-variate data structures. Tri-variate data structures have been rarely subjected to MVMA; in the Price et al analysis [ 8 ], only one such MVMA was reported. Univariate and multivariate analyses were undertaken and compared with respect to differences between estimated outcome variable coefficients, their standard errors (SE) and 95% confidence interval (CI) width and statistical significance, with no selection of meta-analyses based upon the number of RCTs per meta-analysis.…”