Comparing Inference Approaches for RD Designs: A Reexamination of the Effect of Head Start on Child Mortality

Abstract

The regression discontinuity (RD) design is a popular quasi‐experimental design for causal inference and policy evaluation. The most common inference approaches in RD designs employ “flexible” parametric and nonparametric local polynomial methods, which rely on extrapolation and large‐sample approximations of conditional expectations using observations somewhat near the cutoff that determines treatment assignment. An alternative inference approach employs the idea of local randomization, where the very few units closest to the cutoff are regarded as randomly assigned to treatment and finite‐sample exact inference methods are used. In this paper, we contrast these approaches empirically by re‐analyzing the influential findings of Ludwig and Miller (), who studied the effect of Head Start assistance on child mortality employing parametric RD methods. We first review methods based on approximations of conditional expectations, which are relatively well developed in the literature, and then present new methods based on randomization inference. In particular, we extend the local randomization framework to allow for parametric adjustments of the potential outcomes; our extended framework substantially relaxes strong assumptions in prior literature and better resembles other RD inference methods. We compare all these methods formally, focusing on both estimands and inference properties. In addition, we develop new approaches for randomization‐based sensitivity analysis specifically tailored to RD designs. Applying all these methods to the Head Start data, we find that the original RD treatment effect reported in the literature is quite stable and robust, an empirical finding that enhances the credibility of the original result. All the empirical methods we discuss are readily available in general purpose software in R and Stata; we also provide the dataset and software code needed to replicate all our results.

The Internal and External Validity of the Regression Discontinuity Design: A Meta‐analysis of 15 Within‐study Comparisons

“…While we cannot rule out that less sophisticated RD researchers might have obtained more average bias, this possibility has to be weighed against the fact that researchers can learn how to do technically better RDs. There is no shortage of papers and software offering documented guidance (e.g., Cattaneo, Titiunik, & Vazquez‐Bare, ).…”

Cost‐Effectiveness of Early Childhood Interventions to Enhance Preschool: Evidence from a Randomized Experiment in Head Start Centers Enrolling Historically Underserved Populations

“…Most cost analyses of early childhood education compare entire preschool programs, such as Head Start, Abecedarian, or Perry Preschool, to either no preschool at all, or a business-as-usual condition in which families may or may not have access to early childhood education (Barnett, 1985;Belfield et al, 2006;Cattaneo, Titiunik, & Vazquez-Bare, 2017;Ludwig & Phillips, 2008). These types of studies are particularly amenable to cost-benefit analysis, which assesses whether the monetary social benefits of a policy warrant the social cost (Levin et al, 2017;Miller et al, 2018).…”

Simultaneous selection of optimal bandwidths for the sharp regression discontinuity estimator

“…28 They assess the effect of HS assistance on numerous measures such as HS participation, HS spending, other social spending, health, mortality, and education. The effect of HS on child mortality is extensively reexamined by Cattaneo, Titiunik, and Vazquez-Bare (2017) using a local randomization framework as well as a nonparametric framework.…”

“…The numbers of observations with nonzero weight on the left and right of the cut-off are shown in curly brackets. "Robust p-value" is obtained by the CCT-type robust t value based on the bias correction and robust standard error as in Cattaneo, Titiunik, and Vazquez-Bare (2017). For IK and CCT, the "Robust 95% CI" is computed as in CCT.…”