Individual Participant Data Meta-Analysis for a Binary Outcome: One-Stage or Two-Stage?

Abstract

BackgroundA fundamental aspect of epidemiological studies concerns the estimation of factor-outcome associations to identify risk factors, prognostic factors and potential causal factors. Because reliable estimates for these associations are important, there is a growing interest in methods for combining the results from multiple studies in individual participant data meta-analyses (IPD-MA). When there is substantial heterogeneity across studies, various random-effects meta-analysis models are possible that employ a one-stage or two-stage method. These are generally thought to produce similar results, but empirical comparisons are few.ObjectiveWe describe and compare several one- and two-stage random-effects IPD-MA methods for estimating factor-outcome associations from multiple risk-factor or predictor finding studies with a binary outcome. One-stage methods use the IPD of each study and meta-analyse using the exact binomial distribution, whereas two-stage methods reduce evidence to the aggregated level (e.g. odds ratios) and then meta-analyse assuming approximate normality. We compare the methods in an empirical dataset for unadjusted and adjusted risk-factor estimates.ResultsThough often similar, on occasion the one-stage and two-stage methods provide different parameter estimates and different conclusions. For example, the effect of erythema and its statistical significance was different for a one-stage (OR = 1.35, ) and univariate two-stage (OR = 1.55, ). Estimation issues can also arise: two-stage models suffer unstable estimates when zero cell counts occur and one-stage models do not always converge.ConclusionWhen planning an IPD-MA, the choice and implementation (e.g. univariate or multivariate) of a one-stage or two-stage method should be prespecified in the protocol as occasionally they lead to different conclusions about which factors are associated with outcome. Though both approaches can suffer from estimation challenges, we recommend employing the one-stage method, as it uses a more exact statistical approach and accounts for parameter correlation.

Meta-analysis using individual participant data: one-stage and two-stage approaches, and why they may differ

“…An individual participant data (IPD) meta‐analysis can help overcome many of these issues, by obtaining and then synthesising the raw, participant‐level data from each study. For example, IPD allows the meta‐analyst to standardise the inclusion criteria and analyses across studies, to obtain study results that had not been provided by the trial publications and to check modelling assumptions 3. An important advantage is being able to model individual‐level interactions directly within studies, which has substantially greater power and avoids ecological bias compared with a meta‐regression of aggregate data across studies 4, 5.…”

“…The two‐stage approach is often preferred 2, 9 because in the second stage it uses standard meta‐analysis methods that are well documented, for example, in the Cochrane Handbook 10. However, one‐stage methods have also been recommended because they use a more exact likelihood specification 3, 11, which avoids the assumptions of within‐study normality and known within‐study variances, which are especially problematic in meta‐analyses with small studies and/or rare events. Yet, one‐stage methods are also criticised for being computationally intensive and prone to convergence problems 3, 12.…”

“…This is especially important as statistical and/or clinical conclusions may depend on the chosen approach. For example, Debray et al found that erythema was a statistically significant predictor of deep vein thrombosis (DVT) in a one‐stage analysis ( p  = 0.03), but not a two‐stage analysis ( p  = 0.12) 3. For this reason, Tierney et al advise, ‘It is important, therefore, that the choice of one or two‐stage analysis is specified in advance or that results for both approaches are reported’ 13.…”

“…This is potentially a strong assumption, and will usually be unnecessary, but illustrates the so‐called flexibility of the one‐stage approach 3. It is perhaps more realistic and most helpful when the average baseline response α is also of interest, or when needing to reduce (perhaps due to convergence problems) the number of parameters to estimate (that is, rather than estimating K intercepts in equation (13), we now estimate just α and δ in equation (15)).…”

“…Several authors have investigated the difference between one‐stage and two‐stage IPD meta‐analysis results (for example, see the following 1, 2, 3, 9, 48, 49, 50), either empirically, theoretically or via simulation. Most authors conclude that they give very similar results.…”