Bayesian Regression Tree Models for Causal Inference: Regularization, Confounding, and Heterogeneous Effects

Abstract

This paper develops a semi-parametric Bayesian regression model for estimating heterogeneous treatment effects from observational data. Standard nonlinear regression models, which may work quite well for prediction, can yield badly biased estimates of treatment effects when fit to data with strong confounding. Our Bayesian causal forest model avoids this problem by directly incorporating an estimate of the propensity function in the specification of the response model, implicitly inducing a covariate-dependent prior on the regression function. This new parametrization also allows treatment heterogeneity to be regularized separately from the prognostic effect of control variables, making it possible to informatively "shrink to homogeneity", in contrast to existing Bayesian non-and semi-parametric approaches. We illustrate the benefits of this approach via the reanalysis of an observational study assessing the causal effects of smoking on medical expenditures as well as extensive simulation studies.

Semiparametric mixed‐scale models using shared Bayesian forests

“…A special case of such a construction is given by Hahn et al . () to estimate heterogeneous causal effects; in our terminology, their model consists of a shared forest, which captures the prognostic features of covariates which are shared across treatment levels and and an innovation forest which is specific to the treatment .…”

“…Hahn et al . () consider a related structure in the context of causal inference; given a binary treatment , they model potential outcomes as , with both and modeled using BART priors. This is referred to as a Bayesian causal forest (BCF).…”

“…(). The BART framework has been successfully applied to a diverse set of problems including survival analysis (Sparapani et al ., ), causal inference (Hill, ; Hahn et al ., ), analysis of log linear models (Murray, ), imputation of missing predictors (Xu et al ., ), and high‐dimensional prediction and variable selection (Linero, ).…”

“…The Bayesian causal forest (BCF) model of Hahn et al . () uses two separate forests to model the distribution of potential outcomes: the first forest accounts for the direct effect of confounders on the potential outcomes, and is identical for both the treatment and controls, while the second forest is unique to the distribution of the treated samples. Another related work by Starling et al .…”

Approaches to treatment effect heterogeneity in the presence of confounding

“…Many methods for subgroup detection in observational studies have grown out of the genetics and bioinformatics literature but are not designed for comparative evaluation or causal inference . In light of these issues, there have been a number of proposed applications of modern machine learning methods such as regression trees to TEH, including the development of nonparametric causal forests comprised of “honest” trees, the use of trees to identify members of a subgroup with an “enhanced” TE, and a weighted ensemble of estimators …”