Selection into Identification in Fixed Effects Models, with Application to Head Start

Abstract

Many papers use fixed effects (FE) to identify causal impacts of an intervention. In this paper we show that when the treatment status only varies within some groups, this design can induce nonrandom selection of groups into the identifying sample, which we term selection into identification (SI). We begin by illustrating SI in the context of several family fixed effects (FFE) applications with a binary treatment variable. We document that the FFE identifying sample differs from the overall sample along many dimensions, including having larger families. Further, when treatment effects are heterogeneous, the FFE estimate is biased relative to the average treatment effect (ATE). For the general FE model, we then develop a reweighting-on-observables estimator to recover the unbiased ATE from the FE estimate for policy-relevant populations. We apply these insights to examine the long-term effects of Head Start in the PSID and the CNLSY. Using our reweighting methods, we estimate that Head Start leads to a 2.6 percentage point (p.p.) increase (s.e. = 6.2 p.p.) in the likelihood of attending some college for white Head Start participants in the PSID. This ATE is 78% smaller than the traditional FFE estimate (12 p.p). Reweighting the CNLSY FE estimates to obtain the ATE produces similar attenuation in the estimated impacts of Head Start.

Elusive Longer-Run Impacts of Head Start: Replications Within and Across Cohorts

“…However, recent research calls into question the sibling comparison design both in terms of its external validity (D. L. Miller et al, 2019) and potential to produce biased estimates from sibling spillover effects (Heckman & Karapakula, 2019).…”

Who benefits from privileged peers? Evidence from siblings in schools

“…3 For the sake of convenience, we use the expressions "average earnings between age 31 and 40" and "lifetime earnings" as synonymous in the rest of the paper. the estimation and the full sample, as well as the lack of evidence of selection into identification (see Miller et al, 2019), suggest that our causal estimates perform well in terms of external validity.…”

“…Yet, with only 6,737 immigrants in our sample, we lack the statistical power to do so. In addition, less than 25% of school-by-cohort groups are characterized by a positive share of immigrants, raising serious issues in terms of selection into identification and external validity (see Miller et al, 2019).…”

The effect of physical education on children's body weight and human capital: New evidence from the ECLS‐K:2011

“…The FE estimates rely on identification from the subset of students who change PE from year to year. If these students differ from the larger group of students, the effect identified may not represent the ATE (Miller, Shenhav, & Grosz, ). Of course, mixed results across the two approaches may be due to biased coefficients.…”