Statistical approaches to identify subgroups in meta-analysis of individual participant data: a simulation study

Abstract

Background Individual participant data meta-analysis (IPD-MA) is considered the gold standard for investigating subgroup effects. Frequently used regression-based approaches to detect subgroups in IPD-MA are: meta-regression, per-subgroup meta-analysis (PS-MA), meta-analysis of interaction terms (MA-IT), naive one-stage IPD-MA (ignoring potential study-level confounding), and centred one-stage IPD-MA (accounting for potential study-level confounding). Clear guidance on the analyses is lacking and clinical researchers may use approaches with suboptimal efficiency to investigate subgroup effects in an IPD setting. Therefore, our aim is to overview and compare the aforementioned methods, and provide recommendations over which should be preferred. Methods We conducted a simulation study where we generated IPD of randomised trials and varied the magnitude of subgroup effect (0, 25, 50% relative reduction), between-study treatment effect heterogeneity (none, medium, large), ecological bias (none, quantitative, qualitative), sample size (50,100,200), and number of trials (5,10) for binary, continuous and time-to-event outcomes. For each scenario, we assessed the power, false positive rate (FPR) and bias of aforementioned five approaches. Results Naive and centred IPD-MA yielded the highest power, whilst preserving acceptable FPR around the nominal 5% in all scenarios. Centred IPD-MA showed slightly less biased estimates than naïve IPD-MA. Similar results were obtained for MA-IT, except when analysing binary outcomes (where it yielded less power and FPR < 5%). PS-MA showed similar power as MA-IT in non-heterogeneous scenarios, but power collapsed as heterogeneity increased, and decreased even more in the presence of ecological bias. PS-MA suffered from too high FPRs in non-heterogeneous settings and showed biased estimates in all scenarios. Meta-regression showed poor power (< 20%) in all scenarios and completely biased results in settings with qualitative ecological bias. Conclusions Our results indicate that subgroup detection in IPD-MA requires careful modelling. Naive and centred IPD-MA performed equally well, but due to less bias of the estimates in the presence of ecological bias, we recommend the latter. Electronic supplementary material The online version of this article (10.1186/s12874-019-0817-6) contains supplementary material, which is available to authorized users.

Individual participant data meta‐analysis to examine interactions between treatment effect and participant‐level covariates: Statistical recommendations for conduct and planning

“…For example, an IPD meta‐analysis might be conducted for males and females separately, to obtain the summary treatment effect for males and females. However, Fisher et al and Belias et al show that it is dangerous to use the subsequent results to make inferences about whether an interaction exists . In particular, a common mistake is to conclude a treatment‐covariate interaction exists if the summary treatment effect estimate is statistically significant in one subgroup but not the other.…”

Mega-Analysis Methods in ENIGMA: The Experience of the Generalized Anxiety Disorder Working Group

“…Multiple studies have compared meta-and mega-anlyises (Belias, Rovers, Reitsma, Debray, & IntHout, 2019;Riley et al, 2010;Simmonds, Stewart, & Stewart, 2015), suggesting superiority of mega-analyses with IPD when compared to meta-analyses in terms of higher statistical power and acceptable false positive rates. In the context of ENIGMA, comparisons have likewise tended to favor mega-analyses (Boedhoe et al, 2019;Kochunov et al, 2014;Koshiyama et al, 2019).…”

Predicting the treatment response of certolizumab for individual adult patients with rheumatoid arthritis: protocol for an individual participant data meta-analysis

“…Individual participant data meta-analysis (IPD-MA) has been previously employed to develop prediction models for treatment effects [3][4][5][6]. Previous treatment response prediction models for RA were mainly based on observational studies [7][8][9][10][11].…”

“…On the other hand, though the population in RCTs is highly restricted hence may be less representative, data from RCTs are more rigorously collected and more likely to provide an unbiased estimate of the relative treatment effects [ 12 ]. The synthesis of RCT data via IPD-MA can increase the statistical power [ 13 ] and have been used to predict treatment response [ 6 , 14 – 17 ]. To the best of the authors’ knowledge, such an approach has not been taken to predict treatment response in RA to date.…”