Multivariate meta‐analysis: a robust approach based on the theory of U‐statistic

Abstract

Meta-analysis is the methodology for combining findings from similar research studies asking the same question. When the question of interest involves multiple outcomes, multivariate meta-analysis is used to synthesize the outcomes simultaneously taking into account the correlation between the outcomes. Likelihood-based approaches, in particular restricted maximum likelihood (REML) method, are commonly utilized in this context. REML assumes a multivariate normal distribution for the random-effects model. This assumption is difficult to verify, especially for meta-analysis with small number of component studies. The use of REML also requires iterative estimation between parameters, needing moderately high computation time, especially when the dimension of outcomes is large. A multivariate method of moments (MMM) is available and is shown to perform equally well to REML. However, there is a lack of information on the performance of these two methods when the true data distribution is far from normality. In this paper, we propose a new nonparametric and non-iterative method for multivariate meta-analysis on the basis of the theory of U-statistic and compare the properties of these three procedures under both normal and skewed data through simulation studies. It is shown that the effect on estimates from REML because of non-normal data distribution is marginal and that the estimates from MMM and U-statistic-based approaches are very similar. Therefore, we conclude that for performing multivariate meta-analysis, the U-statistic estimation procedure is a viable alternative to REML and MMM. Easy implementation of all three methods are illustrated by their application to data from two published meta-analysis from the fields of hip fracture and periodontal disease. We discuss ideas for future research based on U-statistic for testing significance of between-study heterogeneity and for extending the work to meta-regression setting.

A matrix‐based method of moments for fitting the multivariate random effects model for meta‐analysis and meta‐regression

“…However there is no evidence of bias in the pooled estimates, even when data are missing. REML performed well when the random effects model is misspecified using a t -distribution; Ma and Mazumdar 2011 found that this was also the case for other random effects distributions. Finally, the two methods of moments generally provided very similar rates of requiring truncation to ensure a positive semi-definite estimated between-study covariance matrix, but the proposed method required truncating more often when covariate effects were included in the final simulation study where a multivariate meta-regression model was used.…”

“…The more general validity of the non-likelihood-based methods may be considered advantageous because we can only invoke the Central Limit Theorem to justify this assumption by the notion that the unobserved random effects are the sum of several different factors. Despite this lack of optimality, the simulation studies performed by Ma and Mazumdar 2011, Jackson et al 2010 and Chen et al (2012) suggest that the semi-parametric methods perform well compared with likelihood-based methods when making inferences about the treatment effect. However, the method proposed by Jackson et al 2010 is not invariant to linear transformations and the procedure described by Chen et al (2012) cannot handle covariates or missing outcome data.…”

The choice of prior distribution for a covariance matrix in multivariate meta‐analysis: a simulation study

“…Each simulated meta‐analysis is a set of hypothetical clinical trials comparing a treatment arm with a control. Data sets representing small ( n = 10), medium ( n = 30) and large meta‐analyses ( n = 50) were generated following . Equation is used to generate each meta‐analysis data set, assuming that Y i follows a bivariate normal distribution with given marginal mean θ ∗ and given variance D ∗ + Σ i .…”

“…Several frequentist methods exist for estimating the parameters θ and D in model . Maximum likelihood (ML) , restricted ML (REML) , the method of moments (MM) , the generalized least squares method (GLS) , and the nonparametric U‐statistics method (UM) are among the frequentist options. These differ primarily in the estimation of the between‐study covariance matrix, D ; hence, inferences about the between‐study variances and correlations can be expected to vary from method to method.…”

Bayesian Inference for Multivariate Meta-Regression With a Partially Observed Within-Study Sample Covariance Matrix

“…Wei and Higgins (2013b) discuss Bayesian multivariate meta-analysis with multiple outcomes with a known within-study covariance matrix, where they decompose the between-study covariance matrix into a product of variances and correlations as in Barnard et al (2000), carry out a Cholesky decomposition of the between-study correlation matrix, and specify uniform priors on the Cholesky elements while at the same time ensuring positive definiteness. Ma and Mazumdar (2011) examine robust methods based on U-statistics for a multivariate meta-analysis random effects model assuming that the within-study sample covariance matrix is known. Hamza et al (2009) examine multivariate random effects meta-analysis models with applications to diagnostic tests, where again, the within-study covariance matrix is assumed known.…”