Assessment of heterogeneous treatment effect estimation accuracy via matching

Gao, Zijun; Hastie, Trevor; Tibshirani, Robert

doi:10.1002/sim.9010

Cited by 7 publications

(17 citation statements)

References 37 publications

(49 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…8 it is possible to encapsulate non-parametric functions in the logit. Such decomposition is present in the literature (Gao and Hastie, 2021) (and see Section D, and in particular Lemma 11 for details). But interpretation of all other metrics than the conditional odds ratio is clearly not obvious (see Lemma 12 in Section D) In other words, it seems that such logistic models are rather forcing all covariates and treatment to interact through the link function, preventing any simple interpretation of the causal measure (except conditional OR).…”

Section: Why Not a Logistic Model?mentioning

confidence: 95%

Risk ratio, odds ratio, risk difference... Which causal measure is easier to generalize?

Colnet¹,

Josse²,

Varoquaux³

et al. 2023

Preprint

View full text Add to dashboard Cite

There are many measures to report so-called treatment or causal effect: absolute difference, ratio, odds ratio, number needed to treat, and so on. The choice of a measure, eg absolute versus relative, is often debated because it leads to different appreciations of the same phenomenon; but it also implies different heterogeneity of treatment effect. In addition some measures -but not all -have appealing properties such as collapsibility, matching the intuition of a population summary. We review common measures and their pros and cons typically brought forward. Doing so, we clarify notions of collapsibility and treatment effect heterogeneity, unifying different existing definitions. Our main contribution is to propose to reverse the thinking: rather than starting from the measure, we start from a non-parametric generative model of the outcome. Depending on the nature of the outcome, some causal measures disentangle treatment modulations from baseline risk. Therefore, our analysis outlines an understanding of what heterogeneity and homogeneity of treatment effect mean, not through the lens of the measure, but through the lens of the covariates. Our goal is the generalization of causal measures. We show that different sets of covariates are needed to generalize an effect to a different target population depending on (i) the causal measure of interest, (ii) the nature of the outcome, and (iii) the generalization's method itself (generalizing either conditional outcome or local effects).

show abstract

Section: Why Not a Logistic Model?mentioning

confidence: 95%

Risk ratio, odds ratio, risk difference... Which causal measure is easier to generalize?

Colnet¹,

Josse²,

Varoquaux³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…For each sub-challenge, the primary metric (DSS) was the difference in squared scaled BM between the nivolumab arm and chemotherapy arm, where scaled ( BM ) = 2 × ( BM − 0.5) (Table 1, Supplementary Figure 1). 32,33 Models that performed well in the nivolumab arm and randomly in the chemotherapy arm had positive primary scores. Models that performed well in the chemotherapy arm but randomly in the nivolumab arm had negative primary scores.…”

Section: Methodsmentioning

confidence: 99%

“…For each sub-challenge, the primary metric (DSS) was the difference in squared scaled BM between the nivolumab arm and chemotherapy arm, where ‫݈݀݁ܽܿݏ‬ ሺ‫ܯܤ‬ሻ ൌ 2 ൈ ‫ܯܤ‪ሺ‬‬ െ 0.5ሻ (Table 1, Supplementary Figure 1). 32,33 Models that performed well in the nivolumab arm and randomly in the chemotherapy arm had positive primary scores.…”

Section: Training and Validation Datasetsmentioning

confidence: 99%

A Community Challenge to Predict Clinical Outcomes After Immune Checkpoint Blockade in Non-Small Cell Lung Cancer

Mason

Lapuente-Santana

Halkola

et al. 2022

Preprint

View full text Add to dashboard Cite

Purpose: Predictive biomarkers of immune checkpoint inhibitors (ICIs) efficacy are currently lacking for non-small cell lung cancer (NSCLC). Here, we describe the results from the Anti-PD-1 Response Prediction DREAM Challenge, a crowdsourced initiative that enabled the assessment of predictive models by using data from two randomized controlled clinical trials (RCTs) of ICIs in first-line metastatic NSCLC. Methods: Participants developed and trained models using public resources. These were evaluated with data from the CheckMate 026 trial (NCT02041533), according to the model-to-data paradigm to maintain patient confidentiality. The generalizability of the models with the best predictive performance was assessed using data from the CheckMate 227 trial (NCT02477826). Both trials were phase III RCTs with a chemotherapy control arm, which supported the differentiation between predictive and prognostic models. Isolated model containers were evaluated using a bespoke strategy that considered the challenges of handling transcriptome data from clinical trials. Results: A total of 59 teams participated, with 417 models submitted. Multiple predictive models, as opposed to a prognostic model, were generated for predicting overall survival, progression-free survival, and progressive disease status with ICIs. Variables within the models submitted by participants included tumor mutational burden (TMB), programmed death ligand 1 (PD-L1) expression, and gene-expression-based signatures. The best-performing models showed improved predictive power over reference variables, including TMB or PD-L1. Conclusion: This DREAM Challenge is the first successful attempt to use protected phase III clinical data for a crowdsourced effort towards generating predictive models for ICIs clinical outcomes and could serve as a blueprint for similar efforts in other tumor types and disease states, setting a benchmark for future studies aiming to identify biomarkers predictive of ICIs efficacy.

show abstract

“…The orthogonalization strategy has the distinct advantage over other methods against confounding-such as inverse propensity weighting and matching-that it is stable for extreme propensity scores and forgoes stratification. 35 Robinson 34 and Chernozhukov et al 15 use parametric models to estimate treatment effects based on residualized W and Y , but these models could be replaced by non-parametric or local parametric models 16,49 -such as model-based forests. For mean regression, Dandl et al 36 adapted the orthogonalization strategy to model-based forests.…”

Section: Model-based Forests For Observational Studiesmentioning

confidence: 99%

“…As stated above, our main goal is to assess how the orthogonalization strategy proposed for continuous outcomes could be extended to models beyond mean regression, specifically generalized linear models and transformation models. Based on Dandl et al 36 and Gao and Hastie 35 we propose two different versions of model-based forests, which should be more robust against confounding. Following Dandl et al, 36 we formulate research questions for these versions, which we aim to answer empirically in Section 4.…”

Section: Novel Model-based Forests For Observational Datamentioning

confidence: 99%

Heterogeneous treatment effect estimation for observational data using model-based forests

Dandl,

Bender,

Hothorn

2024

Stat Methods Med Res

View full text Add to dashboard Cite

The estimation of heterogeneous treatment effects has attracted considerable interest in many disciplines, most prominently in medicine and economics. Contemporary research has so far primarily focused on continuous and binary responses where heterogeneous treatment effects are traditionally estimated by a linear model, which allows the estimation of constant or heterogeneous effects even under certain model misspecifications. More complex models for survival, count, or ordinal outcomes require stricter assumptions to reliably estimate the treatment effect. Most importantly, the noncollapsibility issue necessitates the joint estimation of treatment and prognostic effects. Model-based forests allow simultaneous estimation of covariate-dependent treatment and prognostic effects, but only for randomized trials. In this paper, we propose modifications to model-based forests to address the confounding issue in observational data. In particular, we evaluate an orthogonalization strategy originally proposed by Robinson (1988, Econometrica) in the context of model-based forests targeting heterogeneous treatment effect estimation in generalized linear models and transformation models. We found that this strategy reduces confounding effects in a simulated study with various outcome distributions. We demonstrate the practical aspects of heterogeneous treatment effect estimation for survival and ordinal outcomes by an assessment of the potentially heterogeneous effect of Riluzole on the progress of Amyotrophic Lateral Sclerosis.

show abstract

Assessment of heterogeneous treatment effect estimation accuracy via matching

Cited by 7 publications

References 37 publications

Risk ratio, odds ratio, risk difference... Which causal measure is easier to generalize?

Risk ratio, odds ratio, risk difference... Which causal measure is easier to generalize?

A Community Challenge to Predict Clinical Outcomes After Immune Checkpoint Blockade in Non-Small Cell Lung Cancer

Heterogeneous treatment effect estimation for observational data using model-based forests

Contact Info

Product

Resources

About