Treatment effect estimation with Multilevel Regression and Poststratification

Abstract

Multilevel regression and poststratification (MRP) is a flexible modeling technique that has been used in a broad range of small-area estimation problems. Traditionally, MRP studies have been focused on non-causal settings, where estimating a single population value using a nonrepresentative sample was of primary interest. In this manuscript, MRP-style estimators will be evaluated in an experimental causal inference setting. We simulate a large-scale randomized control trial with a stratified cluster sampling design, and compare traditional and nonparametric treatment effect estimation methods with MRP methodology. Using MRP-style estimators, treatment effect estimates for areas as small as 1.3% of the population have lower bias and variance than standard causal inference methods, even in the presence of treatment effect heterogeneity. The design of our simulation studies also requires us to build upon a MRP variant that allows for non-census covariates to be incorporated into poststratification.

Small Area Estimation for Demographic Analysis

“…For each level in each county, each joint posterior estimate is then multiplied by the inclusion weight previously calculated using the poststratification data from the census to create a distribution of weighted posterior estimates. At this point, the median 7 of each of these distributions is taken to produce the median weighted probability for each poststratification level in [21]). The direct estimates are slightly higher in certain areas, whereas the indirect estimates appear to be virtually identical in this format.…”

“…Figure 2. Comparison of number of vaccinated individuals: Direct survey estimates and indirect estimates (classic MRP and MRP with a spatial term) for ZCTAs in Los Angeles County, California, September 2020.The direct survey estimates (left) represent the most common approach to generating population estimates of an outcome (in this case, the number of individuals vaccinated in Los Angeles County, California in September 2020 according to survey results from Carnegie Mellon University's Delphi Group[21]). The direct estimates are slightly higher in certain areas, whereas the indirect estimates appear to be virtually identical in this format.…”