Simulated Treatment Comparison of Time-To-Event (And Other Non-Linear) Outcomes

Population‐adjusted indirect comparisons estimate treatment effects when access to individual patient data is limited and there are cross‐trial differences in effect modifiers. Popular methods include matching‐adjusted indirect comparison (MAIC) and simulated treatment comparison (STC). There is limited formal evaluation of these methods and whether they can be used to accurately compare treatments. Thus, we undertake a comprehensive simulation study to compare standard unadjusted indirect comparisons, MAIC and STC across 162 scenarios. This simulation study assumes that the trials are investigating survival outcomes and measure continuous covariates, with the log hazard ratio as the measure of effect. MAIC yields unbiased treatment effect estimates under no failures of assumptions. The typical usage of STC produces bias because it targets a conditional treatment effect where the target estimand should be a marginal treatment effect. The incompatibility of estimates in the indirect comparison leads to bias as the measure of effect is non‐collapsible. Standard indirect comparisons are systematically biased, particularly under stronger covariate imbalance and interaction effects. Standard errors and coverage rates are often valid in MAIC but the robust sandwich variance estimator underestimates variability where effective sample sizes are small. Interval estimates for the standard indirect comparison are too narrow and STC suffers from bias‐induced undercoverage. MAIC provides the most accurate estimates and, with lower degrees of covariate overlap, its bias reduction outweighs the loss in precision under no failures of assumptions. An important future objective is the development of an alternative formulation to STC that targets a marginal treatment effect.

show abstract

“…Alternative “simulation‐based” formulations to STC have been proposed 16,49 . These are outlined as follows.…”

Section: Methodsmentioning

confidence: 99%

Methods for population adjustment with limited access to individual patient data: A review and simulation study

Remiro‐Azócar

Heath

Baio

2021

Research Synthesis Methods

View full text Add to dashboard Cite

show abstract

“…where t = A, B, denotes the treatment arms of the trial AB and N t(AB) the sample size of treatment arm t. The STC introduced by Ishak et al 6 is based on a regression model for the IPD, which is substituted in mean covariate values. This substitution is done by including covariate adjustment that centers all effect modifiers by the mean values of the aggregated data.…”

Section: Abmentioning

confidence: 99%

“…To supply unbiased estimates, the regression model needs to be specified correctly. Ishak et al 6 propose to simulate the missing arm, but we do not follow this suggestion as it introduces additional variation. Instead, we proceed by substituting in mean covariate values directly.…”

Section: Abmentioning

confidence: 99%

“…Instead, we proceed by substituting in mean covariate values directly. 9 More details are provided by Ishak et al 6 and Phillippo et al 9…”

Section: Abmentioning

confidence: 99%

“…We are interested to find a treatment effect between A and C (AC) under the assumption that the population of interest is given by the population considered in the CB trial. The method of Bucher, 5 the matching-adjusted indirect comparison (MAIC), 4 and the simulated treatment comparison (STC) 6 address this setting of an anchored indirect comparison. In early benefit assessment, the acceptance of population-adjusted indirect comparisons with a common comparator differs between health regulatory authorities.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Comparison of Methods for Estimating Therapy Effects by Indirect Comparisons: A Simulation Study

2020

View full text Add to dashboard Cite

Objective. In evidence synthesis, therapeutic options have to be compared despite the lack of head-to-head trials. Indirect comparisons are then widely used, although little is known about their performance in situations where cross-trial differences or effect modification are present. Methods. We contrast the matching adjusted indirect comparison (MAIC), simulated treatment comparison (STC), and the method of Bucher using a simulation study. The different methods are evaluated according to their power and type I error rate as well as with respect to the coverage, bias, and the root mean squared error (RMSE) of the effect estimate for practically relevant scenarios using binary and time-to-event endpoints. In addition, we investigate how the power planned for the head-to-head trials influences the actual power of the indirect comparison. Results. Indirect comparisons are considerably underpowered. None of the methods had substantially superior performance. In situations without cross-trial differences and effect modification, MAIC and Bucher led to similar results, while Bucher has the advantage of preserving the within-study randomization. MAIC and STC could enhance power in some scenarios but at the cost of a potential type I error inflation. Adjusting MAIC and STC for confounders that did not modify the effect led to higher bias and RMSE. Conclusion. The choice of effect modifiers in MAIC and STC influences the precision of the indirect comparison. Therefore, a careful selection of effect modifiers is warranted. In addition, missed differences between trials may lead to low power and partly high bias for all considered methods, and thus, results of indirect comparisons should be interpreted with caution.

show abstract

Parametric G‐computation for compatible indirect treatment comparisons with limited individual patient data

Remiro‐Azócar

Heath

Baio

2022

Research Synthesis Methods

View full text Add to dashboard Cite

Population adjustment methods such as matching‐adjusted indirect comparison (MAIC) are increasingly used to compare marginal treatment effects when there are cross‐trial differences in effect modifiers and limited patient‐level data. MAIC is based on propensity score weighting, which is sensitive to poor covariate overlap and cannot extrapolate beyond the observed covariate space. Current outcome regression‐based alternatives can extrapolate but target a conditional treatment effect that is incompatible in the indirect comparison. When adjusting for covariates, one must integrate or average the conditional estimate over the relevant population to recover a compatible marginal treatment effect. We propose a marginalization method based on parametric G‐computation that can be easily applied where the outcome regression is a generalized linear model or a Cox model. The approach views the covariate adjustment regression as a nuisance model and separates its estimation from the evaluation of the marginal treatment effect of interest. The method can accommodate a Bayesian statistical framework, which naturally integrates the analysis into a probabilistic framework. A simulation study provides proof‐of‐principle and benchmarks the method's performance against MAIC and the conventional outcome regression. Parametric G‐computation achieves more precise and more accurate estimates than MAIC, particularly when covariate overlap is poor, and yields unbiased marginal treatment effect estimates under no failures of assumptions. Furthermore, the marginalized regression‐adjusted estimates provide greater precision and accuracy than the conditional estimates produced by the conventional outcome regression, which are systematically biased because the measure of effect is non‐collapsible.

show abstract

Simulated Treatment Comparison of Time-To-Event (And Other Non-Linear) Outcomes

Cited by 7 publications

References 0 publications

Methods for population adjustment with limited access to individual patient data: A review and simulation study

Methods for population adjustment with limited access to individual patient data: A review and simulation study

Comparison of Methods for Estimating Therapy Effects by Indirect Comparisons: A Simulation Study

Parametric G‐computation for compatible indirect treatment comparisons with limited individual patient data

Contact Info

Product

Resources

About