Permutation inference methods for multivariate meta‐analysis

Abstract

Multivariate meta‐analysis is gaining prominence in evidence synthesis research because it enables simultaneous synthesis of multiple correlated outcome data, and random‐effects models have generally been used for addressing between‐studies heterogeneities. However, coverage probabilities of confidence regions or intervals for standard inference methods for random‐effects models (eg, restricted maximum likelihood estimation) cannot retain their nominal confidence levels in general, especially when the number of synthesized studies is small because their validities depend on large sample approximations. In this article, we provide permutation‐based inference methods that enable exact joint inferences for average outcome measures without large sample approximations. We also provide accurate marginal inference methods under general settings of multivariate meta‐analyses. We propose effective approaches for permutation inferences using optimal weighting based on the efficient score statistic. The effectiveness of the proposed methods is illustrated via applications to bivariate meta‐analyses of diagnostic accuracy studies for airway eosinophilia in asthma and a network meta‐analysis for antihypertensive drugs on incident diabetes, as well as through simulation experiments. In numerical evaluations performed via simulations, our methods generally provided accurate confidence regions or intervals under a broad range of settings, whereas the current standard inference methods exhibited serious undercoverage properties.

Elevated Calprotectin and Abnormal Myeloid Cell Subsets Discriminate Severe from Mild COVID-19

“…The regularization parameter was selected from 10-folds cross-validation using the glmnet R package ( Friedman et al, 2010 ). The final AUC estimate was corrected for optimism using the Harrell’s method ( Harrell et al, 1996 ), and its confidence interval was computed using the two-stage approach proposed by Noma and colleagues ( Noma et al, 2020 ) with 2000 bootstrap samples for each stage. In this analysis, we included age, sex and comorbidities together with biological parameters.…”

A D-vine copula mixed model for joint meta-analysis and comparison of diagnostic tests

“…The purpose of a meta-analysis of diagnostic test accuracy studies is to combine information over different studies, and provide an integrated analysis that will have more statistical power to detect an accurate diagnostic test than an analysis based on a single study. As the accuracy of a diagnostic test is commonly measured by a pair of indices such as sensitivity and specificity, synthesis of diagnostic test accuracy studies is the most common medical application of multivariate meta-analysis (e.g., Noma et al 2020). Most of the existing literature has mainly focused on a single test as acknowledged in a recent review by Ma et al (2016).…”

A permutation‐based approach for heterogeneous meta‐analyses of rare events

“…Liu et al 34 and Wang and Tian 35 build on this idea and propose methods to construct exact confidence intervals that are less computationally intensive than Follmann and Proschan 33 and that can perform well for data sets with few studies. Noma et al 36 extend the work of Follmann and Proschan 33 and Liu et al 34 with applications to multivariate, or network, meta-analysis. While our proposed method is also permutation-based, the interesting distinction lies in how these permutations are implemented.…”