A Within-Group Approach to Ensemble Machine Learning Methods for Causal Inference in Multilevel Studies

Suk, Youmi

doi:10.3102/10769986231162096

Cited by 3 publications

(2 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fifth, the effectiveness of the proposed modifications may be diminished in small data samples, such as the (25, 25) condition in our study. In such cases, one can consider employing a grouping strategy that combines clusters based on treatment prevalence (Lee et al, 2021;Suk, 2023), or Bayesian OTR estimation methods (Murray et al, 2018).…”

Section: Resultsmentioning

confidence: 99%

Designing Optimal, Data-Driven Policies from Multisite Randomized Trials

Suk,

Park

2023

Psychometrika

Self Cite

View full text Add to dashboard Cite

Optimal treatment regimes (OTRs) have been widely employed in computer science and personalized medicine to provide data-driven, optimal recommendations to individuals. However, previous research on OTRs has primarily focused on settings that are independent and identically distributed, with little attention given to the unique characteristics of educational settings, where students are nested within schools and there are hierarchical dependencies. The goal of this study is to propose a framework for designing OTRs from multisite randomized trials, a commonly used experimental design in education and psychology to evaluate educational programs. We investigate modifications to popular OTR methods, specifically Q-learning and weighting methods, in order to improve their performance in multisite randomized trials. A total of 12 modifications, 6 for Q-learning and 6 for weighting, are proposed by utilizing different multilevel models, moderators, and augmentations. Simulation studies reveal that all Q-learning modifications improve performance in multisite randomized trials and the modifications that incorporate random treatment effects show the most promise in handling cluster-level moderators. Among weighting methods, the modification that incorporates cluster dummies into moderator variables and augmentation terms performs best across simulation conditions. The proposed modifications are demonstrated through an application to estimate an OTR of conditional cash transfer programs using a multisite randomized trial in Colombia to maximize educational attainment.

show abstract

Section: Resultsmentioning

confidence: 99%

Designing Optimal, Data-Driven Policies from Multisite Randomized Trials

Suk,

Park

2023

Psychometrika

Self Cite

View full text Add to dashboard Cite

show abstract

“…Regarding the method of incorporating clusters, the method we consider here (i.e., including cluster indicators as part of the covariates) has been similarly used in previous literature that uses machine learning in propensity score model estimation with clustered data (e.g., . Alternative methods exist (e.g., Salditt and Nestler, 2023;Suk, 2024;Suk and Kang, 2023), and it could be important future research topics to explore them for estimating the interventional (in)direct effects with clustered data. In the next section, we conduct simulation studies to examine the performance of the methods we proposed in this section; We also provide an R package "MediationClustered" that implements all methods we examined (available at https://github.com/xliu12/multiM).…”

Section: Multiply-robust Estimationmentioning

confidence: 99%

Estimating Causal Mediation Effects in Multiple-Mediator Analyses with Clustered Data

Liu

2024

Preprint

View full text Add to dashboard Cite

In multiple-mediator analyses in behavioral sciences, data are often clustered. Despite the growing literature on causal mediation analysis, little research exists on methods for assessing causal mediation effects in multiple-mediator analyses with clustered data, especially when the multiple mediators may be dependent. In this paper, we develop methods for estimating causal mediation effects (particularly the interventional indirect and direct effects) in the multiple-mediator analyses with clustered data, taking into account (possibly) unmeasured cluster-level confounders. Extending the causal mediation literature, we develop three estimators—the regression-based, the weighting-based, and the multiply-robust estimators—for clustered data, with incorporating clusters in estimating the models involved. The simulation results show that our developed methods (including the three estimators with multilevel random-effects or fixed-effects models, and the multiply-robust estimator with machine learning methods to estimate the models incorporating clusters) can yield satisfactory estimates of the causal mediation effects for the multiple-mediator analysis with clustered data. We illustrate our methods in an example using data from the Educational Longitudinal Study.

show abstract

Parametric and nonparametric propensity score estimation in multilevel observational studies

Salditt

Nestler

2023

Statistics in Medicine

View full text Add to dashboard Cite

There has been growing interest in using nonparametric machine learning approaches for propensity score estimation in order to foster robustness against misspecification of the propensity score model. However, the vast majority of studies focused on single‐level data settings, and research on nonparametric propensity score estimation in clustered data settings is scarce. In this article, we extend existing research by describing a general algorithm for incorporating random effects into a machine learning model, which we implemented for generalized boosted modeling (GBM). In a simulation study, we investigated the performance of logistic regression, GBM, and Bayesian additive regression trees for inverse probability of treatment weighting (IPW) when the data are clustered, the treatment exposure mechanism is nonlinear, and unmeasured cluster‐level confounding is present. For each approach, we compared fixed and random effects propensity score models to single‐level models and evaluated their use in both marginal and clustered IPW. We additionally investigated the performance of the standard Super Learner and the balance Super Learner. The results showed that when there was no unmeasured confounding, logistic regression resulted in moderate bias in both marginal and clustered IPW, whereas the nonparametric approaches were unbiased. In presence of cluster‐level confounding, fixed and random effects models greatly reduced bias compared to single‐level models in marginal IPW, with fixed effects GBM and fixed effects logistic regression performing best. Finally, clustered IPW was overall preferable to marginal IPW and the balance Super Learner outperformed the standard Super Learner, though neither worked as well as their best candidate model.

show abstract

A Within-Group Approach to Ensemble Machine Learning Methods for Causal Inference in Multilevel Studies

Cited by 3 publications

References 56 publications

Designing Optimal, Data-Driven Policies from Multisite Randomized Trials

Designing Optimal, Data-Driven Policies from Multisite Randomized Trials

Estimating Causal Mediation Effects in Multiple-Mediator Analyses with Clustered Data

Parametric and nonparametric propensity score estimation in multilevel observational studies

Contact Info

Product

Resources

About