Uncertainty in Propensity Score Estimation: Bayesian Methods for Variable Selection and Model-Averaged Causal Effects

Abstract

Causal inference with observational data frequently relies on the notion of the propensity score (PS) to adjust treatment comparisons for observed confounding factors. As decisions in the era of “big data” are increasingly reliant on large and complex collections of digital data, researchers are frequently confronted with decisions regarding which of a high-dimensional covariate set to include in the PS model in order to satisfy the assumptions necessary for estimating average causal effects. Typically, simple or ad-hoc methods are employed to arrive at a single PS model, without acknowledging the uncertainty associated with the model selection. We propose three Bayesian methods for PS variable selection and model averaging that 1) select relevant variables from a set of candidate variables to include in the PS model and 2) estimate causal treatment effects as weighted averages of estimates under different PS models. The associated weight for each PS model reflects the data-driven support for that model’s ability to adjust for the necessary variables. We illustrate features of our proposed approaches with a simulation study, and ultimately use our methods to compare the effectiveness of surgical vs. nonsurgical treatment for brain tumors among 2,606 Medicare beneficiaries. Supplementary materials are available online.

The Central Role of Bayes’ Theorem for Joint Estimation of Causal Effects and Propensity Scores

“…This strategy implies a joint distribution for ( Y , X , C ), and also benefits from accurate characterization of e i ( C i , θ X|C ) for estimating causal effects. Augmenting the outcome model to include both e i ( C i , θ X|C ) and individual covariate adjustment is advocated for a variety of reasons (Stuart, 2010), and has been employed in Bayesian propensity score methods in Zigler and Dominici (2014).…”

“…Various methods described as “quasi-Bayesian,” “approximately-Bayesian,” or “two-step Bayesian” have been recently proposed to combine estimation of the propensity score with that of causal effects without appealing to Bayes theorem to unify these two stages with a joint likelihood (Hoshino, 2008; McCandless et al, 2010; Kaplan and Chen, 2012; Zigler and Dominici, 2014). These methods have been described as “cutting the feedback” between the propensity score and outcome stages (Lunn et al, 2009), and represent a special case of so-called “modularization” in Bayesian inference (Liu et al, 2009).…”

“…While the usefulness of propensity scores for Bayesian inference was motivated decades ago (Rubin, 1985), the promise of modern Bayesian analysis tools has spawned renewed interest in Bayesian estimation of causal effects with propensity scores (Hoshino, 2008; McCandless et al, 2009; An, 2010; McCandless et al, 2010; Kaplan and Chen, 2012; Zigler et al, 2013; McCandless et al, 2012; Zigler and Dominici, 2014; Saarela et al, 2015). …”

“…More recently, Saarela et al (2015) proposed a Bayesian approach that uses propensity scores to estimate the parameters of a marginal structural model, which incited interesting discussions about whether the approach could be truly “Bayesian” (Robins et al, 2015). A variety of other methods have been proposed and described with terms such as “quasi-Bayesian” or “two-step Bayesian” to employ the Bayesian inferential paradigm in propensity score and causal effect estimation while maintaining the conceptual separation of design and analysis stages (McCandless et al, 2010; Kaplan and Chen, 2012; Zigler and Dominici, 2014). The purpose of this article is to clarify, against the backdrop of existing literature on “Bayesian propensity score analysis,” the fundamental issues at play when attempting to conduct Bayesian inference amid the two-stage nature of estimating causal effects with propensity scores.…”

“…Improved understanding of the fundamental interplay between propensity scores and Bayesian inference is an avenue to unite the established virtues of propensity scores with the benefits of modern Bayesian methodology. Such a unification can motivate new methods targeting new research questions, with important progress recently appearing for using propensity scores in Bayesian estimation of parameters in latent variable models (Hoshino, 2008), in Bayesian models that use propensity scores for combining data from multiple sources (McCandless et al, 2012), and in methods for Bayesian model averaging to address uncertainty in the propensity score model specification (Zigler and Dominici, 2014). Improved understanding of the complexities of Bayesian inference with propensity scores will permit continued progress towards these and other methodological goals.…”