Inference on finite-population treatment effects under limited overlap

Abstract

Summary This paper studies inference on finite-population average and local average treatment effects under limited overlap, meaning that some strata have a small proportion of treated or untreated units. We model limited overlap in an asymptotic framework, sending the propensity score to zero (or one) with the sample size. We derive the asymptotic distribution of analogue estimators of the treatment effects under two common randomization schemes: conditionally independent and stratified block randomization. Under either scheme, the limit distribution is the same and conventional standard error formulas remain asymptotically valid, but the rate of convergence is slower the faster the propensity score degenerates. The practical import of these results is two-fold. When overlap is limited, standard methods can perform poorly in smaller samples, as asymptotic approximations are inadequate owing to the slower rate of convergence. However, in larger samples, standard methods can work quite well even when the propensity score is small.

Estimation and Inference for Moments of Ratios With Robustness Against Large Trimming Bias

“…In order to provide a complete empirical procedure, we develop an inference method for E[B/A] based on θ(h n ). 5 There is an extensive body of literature on inference in a related framework (e.g., Khan and Tamer, 2010;Yang, 2014;Khan and Nekipelov, 2015;Chaudhuri and Hill, 2016a;Rothe, 2017;Ma and Wang, 2020;Heiler and Kazak, 2020;Hong et al, 2020). All these papers use distributional restrictions in different ways.…”

“…For the latter, see Andrews and Schafgans (1998); Romano and Wolf (1999); Peng (2001Peng ( , 2004; Peña et al (2008); Antoine and Renault (2009); Hill and Renault (2010); Antoine and Renault (2012); Chen and Liao (2014); Chen and Pouzo (2015) in addition to the references in the main text that are more closely related to this paper in terms of the ratio structure. Hong et al (2020) consider a finite support with drifting sequence. We do not make any of these distributional assumptions.…”

Auction Throttling and Causal Inference of Online Advertising Effects

“…We take a designed-based inference perspective to quantify the uncertainty of the sample local average treatment effect. Our inference objective is related to Hong et al (2020) that measures the uncertainty of the population local average treatment effect with a stratified experiment. Our setting can be understood as a nonstandard stratified experiment, where the assignment probability is not a priori fixed.…”

“…This variance estimate is expected to be conservative because 1) bootstrap is generally known to be valid for asymptotic inference under the superpopulation model (Hong et al (2020)), and 2) the standard error estimated under a superpopulation model can be used as a conservative standard error for our sample LATE under a finite sample model (Imbens (2003) and Abadie et al (2020)).…”

Bounding Treatment Effects by Pooling Limited Information across Observations

“…This problem is referred to as limited overlap (e.g., Crump, Hotz, Imbens and Mitnik, 2009). The existence of limited overlap may change the asymptotic behavior of the estimators (e.g., Khan and Tamer, 2010;Hong, Leung and Li, 2019) and may necessitate using a more robust inference method (e.g., Rothe, 2017;Sasaki and Ura, 2021). D'Amour, Ding, Feller, Lei and Sekhon (2021) provide a cautionary tale on the overlap condition when high-dimensional covariates are adopted to make unconfoundedness more plausible.…”