Covariate Balance in Bayesian Propensity Score Approaches for Observational Studies

Abstract

Bayesian alternatives to frequentist propensity score approaches have recently been proposed. However, few studies have investigated their covariate balancing properties. This article compares a recently developed two-step Bayesian propensity score approach to the frequentist approach with respect to covariate balance. The effects of different priors on covariate balance are evaluated and the differences between frequentist and Bayesian covariate balance are discussed. Results of the case study reveal that both the Bayesian and frequentist propensity score approaches achieve good covariate balance. The frequentist propensity score approach performs slightly better on covariate balance for stratification and weighting methods, whereas the two-step Bayesian approach offers slightly better covariate balance in the optimal full matching method. Results of a comprehensive simulation study reveal that accuracy and precision of prior information on propensity score model parameters do not greatly influence balance performance. Results of the simulation study also show that overall, the optimal full matching method provides the best covariate balance and treatment effect estimates compared to the stratification and weighting methods. A unique feature of covariate balance within Bayesian propensity score analysis is that we can obtain a distribution of balance indices in addition to the point estimates so that the variation in balance indices can be naturally captured to assist in covariate balance checking.

Bayesian Model Averaging for Propensity Score Analysis

“…A further study of the covariate balance properties of the Kaplan and Chen (2012) approach was given in a case study by Chen and Kaplan (2014). Their results revealed that both Bayesian and frequentist propensity score approaches substantially reduced initial imbalance as expected, and their performance on covariate balance was similar with regard to the standardized mean/proportion differences and variance ratios in the treatment group and control group.…”

“…Chen and Kaplan (2014) argued that a benefit of conducting Bayesian propensity score analysis is that one can obtain the posterior distribution of the propensity score and thus the posterior distribution of corresponding balance indices (e.g., Cohen’s d and variance ratio) so that the variation in balance indices can be studied in addition to the point estimates to assist in balance checking. Good balance is achieved if both the point estimates and the posterior probability intervals of the balance indices fall into the desirable range.…”

STEM Pathways: Examining Persistence in Rigorous Math and Science Course Taking

“…Several authors (see Austin 2011; Chen and Kaplan 2015) note the fact that the use of propensity score adjustment as a conditioning method can be hindered by possible model misspecification. The credibility of propensity score analysis is based on the assumption of ''strongly ignorable treatment assignment'' where it is assumed that all of the relevant covariates are included in the treatment assignment and that the bias due to unmeasured covariates is ignorable (see Lanehart et al 2012, p. 8).…”

Causal inference with large-scale assessments in education from a Bayesian perspective: a review and synthesis

“…Propensity score stratification involves forming strata directly on the basis of the observed propensity score. Subclassification into five strata on continuous distributions such as the propensity score has been shown to remove approximately 90 % of the bias due to non-random selection effects ( Cochran 1968 , see also Rosenbaum and Rubin 1983 ). However, for stratification on the propensity score to achieve the desired effect, the assumption of no hidden biases must hold.…”