Improving ecological inference using individual‐level data

Jackson, Chris; Best, Nicky; Richardson, Sylvia

doi:10.1002/sim.2370

Cited by 131 publications

(159 citation statements)

References 19 publications

Supporting

Mentioning

158

Contrasting

Order By: Relevance

“…At best, meta‐regression analyses using the trial‐level should only be viewed as exploratory when the aim is to identify individual‐level associations and should not inform clinical recommendations. However, we recognize that others may not agree with these recommendations; for example, models have been proposed for combining across‐study and within‐study interactions when a mixture of IPD and AD are available 56, 57.…”

Section: Discussionmentioning

confidence: 99%

One-stage individual participant data meta-analysis models: estimation of treatment-covariate interactions must avoid ecological bias by separating out within-trial and across-trial information

et al. 2016

View full text Add to dashboard Cite

Stratified medicine utilizes individual‐level covariates that are associated with a differential treatment effect, also known as treatment‐covariate interactions. When multiple trials are available, meta‐analysis is used to help detect true treatment‐covariate interactions by combining their data. Meta‐regression of trial‐level information is prone to low power and ecological bias, and therefore, individual participant data (IPD) meta‐analyses are preferable to examine interactions utilizing individual‐level information. However, one‐stage IPD models are often wrongly specified, such that interactions are based on amalgamating within‐ and across‐trial information. We compare, through simulations and an applied example, fixed‐effect and random‐effects models for a one‐stage IPD meta‐analysis of time‐to‐event data where the goal is to estimate a treatment‐covariate interaction. We show that it is crucial to centre patient‐level covariates by their mean value in each trial, in order to separate out within‐trial and across‐trial information. Otherwise, bias and coverage of interaction estimates may be adversely affected, leading to potentially erroneous conclusions driven by ecological bias. We revisit an IPD meta‐analysis of five epilepsy trials and examine age as a treatment effect modifier. The interaction is −0.011 (95% CI: −0.019 to −0.003; p = 0.004), and thus highly significant, when amalgamating within‐trial and across‐trial information. However, when separating within‐trial from across‐trial information, the interaction is −0.007 (95% CI: −0.019 to 0.005; p = 0.22), and thus its magnitude and statistical significance are greatly reduced. We recommend that meta‐analysts should only use within‐trial information to examine individual predictors of treatment effect and that one‐stage IPD models should separate within‐trial from across‐trial information to avoid ecological bias. © 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

show abstract

Section: Discussionmentioning

confidence: 99%

One-stage individual participant data meta-analysis models: estimation of treatment-covariate interactions must avoid ecological bias by separating out within-trial and across-trial information

et al. 2016

View full text Add to dashboard Cite

show abstract

“…This uses Markov chain Monte Carlo methods to simultaneously estimate IPD-trial and AD-trial models linked by common parameters. The approach has been developed in the context of ecological studies, where individual-level data supplement the aggregate information across different groups [21], such as geographical areas. Sutton et al [25] have also applied the method to clinical studies varying in design and whether they provided IPD or AD.…”

Section: Discussionmentioning

confidence: 99%

Meta‐analysis of continuous outcomes combining individual patient data and aggregate data

Riley

Lambert

Staessen

et al. 2007

Statistics in Medicine

237

340

View full text Add to dashboard Cite

SUMMARYMeta-analysis of individual patient data (IPD) is the gold-standard for synthesizing evidence across clinical studies. However, for some studies IPD may not be available and only aggregate data (AD), such as a treatment effect estimate and its standard error, may be obtained. In this situation, methods for combining IPD and AD are important to utilize all the available evidence. In this paper, we develop and assess a range of statistical methods for combining IPD and AD in meta-analysis of continuous outcomes from randomized controlled trials.The methods take either a one-step or a two-step approach. The latter is simple, with IPD reduced to AD so that standard AD meta-analysis techniques can be employed. The one-step approach is more complex but offers a flexible framework to include both patient-level and trial-level parameters. It uses a dummy variable to distinguish IPD trials from AD trials and to constrain which parameters the AD trials estimate. We show that this is important when assessing how patient-level covariates modify treatment effect, as aggregate-level relationships across trials are subject to ecological bias and confounding. We thus develop models to separate within-trial and across-trials treatment-covariate interactions; this ensures that only IPD trials estimate the former, whilst both IPD and AD trials estimate the latter in addition to the pooled treatment effect and any between-study heterogeneity. Extension to multiple correlated outcomes is also considered. Ten IPD trials in hypertension, with blood pressure the continuous outcome of interest,

show abstract

“…[44][45][46][47][48][49][50][51][52][53] These articles included two comprehensive reviews 44,45 as well as individual articles, all of which were identified in the review articles. Many of the techniques described in Verde and Ohmann 44 and Doi, 45 however, have limited applicability to regenerative medicine (i.e.…”

Section: Data Sharing Statementmentioning

confidence: 99%

The assessment and appraisal of regenerative medicines and cell therapy products: an exploration of methods for review, economic evaluation and appraisal

Hettle

Corbett

Hinde

et al. 2017

Health Technol Assess

104

143

View full text Add to dashboard Cite

BackgroundThe National Institute for Health and Care Excellence (NICE) commissioned a ‘mock technology appraisal’ to assess whether changes to its methods and processes are needed. This report presents the findings of independent research commissioned to inform this appraisal and the deliberations of a panel convened by NICE to evaluate the mock appraisal.MethodsOur research included reviews to identify issues, analysis methods and conceptual differences and the relevance of alternative decision frameworks, alongside the development of an exemplar case study of chimeric antigen receptor (CAR) T-cell therapy for treating acute lymphoblastic leukaemia.ResultsAn assessment of previous evaluations of regenerative medicines found that, although there were a number of evidential challenges, none was unique to regenerative medicines or was beyond the scope of existing methods used to conceptualise decision uncertainty. Regarding the clinical evidence for regenerative medicines, the issues were those associated with a limited evidence base but were not unique to regenerative medicines: small non-randomised studies, high variation in response and the intervention subject to continuing development. The relative treatment effects generated from single-arm trials are likely to be optimistic unless it is certain that the historical data have accurately estimated the efficacy of the control agent. Pivotal trials may use surrogate end points, which, on average, overestimate treatment effects. To reduce overall uncertainty, multivariate meta-analysis of all available data should be considered. Incorporating indirectly relevant but more reliable (more mature) data into the analysis can also be considered; such data may become available as a result of the evolving regulatory pathways being developed by the European Medicines Agency. For the exemplar case of CAR T-cell therapy, target product profiles (TPPs) were developed, which considered the ‘curative’ and ‘bridging to stem-cell transplantation’ treatment approaches separately. Within each TPP, three ‘hypothetical’ evidence sets (minimum, intermediate and mature) were generated to simulate the impact of alternative levels of precision and maturity in the clinical evidence. Subsequent assessments of cost-effectiveness were undertaken, employing the existing NICE reference case alongside additional analyses suggested within alternative frameworks. The additional exploratory analyses were undertaken to demonstrate how assessments of cost-effectiveness and uncertainty could be impacted by alternative managed entry agreements (MEAs), including price discounts, performance-related schemes and technology leasing. The panel deliberated on the range of TPPs, evidence sets and MEAs, commenting on the likely recommendations for each scenario. The panel discussed the challenges associated with the exemplar and regenerative medicines more broadly, focusing on the need for a robust quantification of the level of uncertainty in the cost-effective estimates and the potential value of MEAs in limiting the exposure of the NHS to high upfront costs and loss associated with a wrong decision.ConclusionsIt is to be expected that there will be a significant level of uncertainty in determining the clinical effectiveness of regenerative medicines and their long-term costs and benefits, but the existing methods available to estimate the implications of this uncertainty are sufficient. The use of risk sharing and MEAs between the NHS and manufacturers of regenerative medicines should be investigated further.FundingThe National Institute for Health Research Health Technology Assessment programme.

show abstract

Improving ecological inference using individual‐level data

Cited by 131 publications

References 19 publications

One-stage individual participant data meta-analysis models: estimation of treatment-covariate interactions must avoid ecological bias by separating out within-trial and across-trial information

One-stage individual participant data meta-analysis models: estimation of treatment-covariate interactions must avoid ecological bias by separating out within-trial and across-trial information

Meta‐analysis of continuous outcomes combining individual patient data and aggregate data

The assessment and appraisal of regenerative medicines and cell therapy products: an exploration of methods for review, economic evaluation and appraisal

Contact Info

Product

Resources

About