Model Feedback in Bayesian Propensity Score Estimation

Abstract

Summary Methods based on the propensity score comprise one set of valuable tools for comparative effectiveness research and for estimating causal effects more generally. These methods typically consist of two distinct stages: 1) a propensity score stage where a model is fit to predict the propensity to receive treatment (the propensity score), and 2) an outcome stage where responses are compared in treated and untreated units having similar values of the estimated propensity score. Traditional techniques conduct estimation in these two stages separately; estimates from the first stage are treated as fixed and known for use in the second stage. Bayesian methods have natural appeal in these settings because separate likelihoods for the two stages can be combined into a single joint likelihood, with estimation of the two stages carried out simultaneously. One key feature of joint estimation in this context is “feedback” between the outcome stage and the propensity score stage, meaning that quantities in a model for the outcome contribute information to posterior distributions of quantities in the model for the propensity score. We provide a rigorous assessment of Bayesian propensity score estimation to show that model feedback can produce poor estimates of causal effects absent strategies that augment propensity score adjustment with adjustment for individual covariates. We illustrate this phenomenon with a simulation study and with a comparative effectiveness investigation of carotid artery stenting vs. carotid endarterectomy among 123,286 Medicare beneficiaries hospitlized for stroke in

The Central Role of Bayes’ Theorem for Joint Estimation of Causal Effects and Propensity Scores

“…Bayesian estimation of causal effects with (4) was proposed in McCandless et al (2009), but shown in Zigler et al (2013) to yield biased estimates of causal effects without further adjustments. That is, the posterior distribution in (3) cannot, in general, be recovered with a likelihood specified as in (4).…”

“…While the usefulness of propensity scores for Bayesian inference was motivated decades ago (Rubin, 1985), the promise of modern Bayesian analysis tools has spawned renewed interest in Bayesian estimation of causal effects with propensity scores (Hoshino, 2008; McCandless et al, 2009; An, 2010; McCandless et al, 2010; Kaplan and Chen, 2012; Zigler et al, 2013; McCandless et al, 2012; Zigler and Dominici, 2014; Saarela et al, 2015). …”

“…In essence, the use of Bayesian methods in either the design stage or the analysis stage is straightforward, but debate remains about how to combine these two stages into a single Bayesian analysis. In McCandless et al (2009) a “Bayesian propensity score analysis” is proposed to use the mechanics of Bayesian estimation to unite the design and analysis stages into a single procedure, but Zigler et al (2013) showed that “feedback” of information between the two stages could render the procedure unable to reliably estimate causal effects. More recently, Saarela et al (2015) proposed a Bayesian approach that uses propensity scores to estimate the parameters of a marginal structural model, which incited interesting discussions about whether the approach could be truly “Bayesian” (Robins et al, 2015).…”

Bayesian Approach for Addressing Differential Covariate Measurement Error in Propensity Score Methods

“…This negates an advantage of propensity scores, which is that propensity score model fit is typically optimized without looking at the outcome–thus reflecting their role in the design stage of the study and preventing researchers from modifying their analysis to get desired results in terms of the effect estimates (Stuart, 2010). Recently, Zigler and others (2013) point out that the propensity score adjustment approach can result in model feedback that biases ATE estimates. Our joint models use a propensity score weighting approach and perform better in imputing the true covariate than two-step models.…”

Covariate Balance in Bayesian Propensity Score Approaches for Observational Studies

“…In fact, Zigler et al (2013) assessed the joint estimation of the Bayesian propensity score and the treatment effect and found that the feedback between propensity score model and outcome model can lead to poor treatment effect estimates. This model feedback is especially problematic if the relationship between the outcome and the propensity score is misspecified (McCandless, Douglas, Evans, & Smeeth, 2010 (2010) included the posterior distribution of the propensity score parameters as covariate input in the outcome model so that the flow of information between the propensity score and the outcome is restricted.…”