Treatment effect heterogeneity in the head start impact study: A systematic review of study characteristics and findings

Abstract

There have been consistent efforts to assess treatment effect heterogeneity (TEH) of Head Start using the data from the Head Start Impact Study (HSIS), a randomized controlled trial of a federally funded child development program for a nationally representative sample of low-income parents and their 3- and 4-year-old children in the United States. Including 28 studies on TEH of Head Start, this review found that multiple high-risk subgroups (e.g., children with lower cognitive abilities, Spanish-speaking dual language learners) experienced larger gains across a range of developmental and parental outcomes, but mixed results for several subgroups. Most studies focused on subgroup analyses, cognitive and social-emotional outcomes, and short-term effects. Further studies on distributional effects, health and parental outcomes, and long-term effects are warranted. Finally, suggestions for future research on TEH of Head Start are discussed, which are applicable to other child development programs and policy evaluations.

Assessment of heterogeneous Head Start treatment effects on cognitive and social-emotional outcomes

“…They further examined for which subgroups of children Head Start was effective and found that the program in general had compensatory effects, or greater benefits for those with greater needs. 11 Head Start benefitted several subgroups with more disadvantages, such as children with Spanish as a primary language, 12 , 13 those who had lower cognitive skills at baseline, 12 , 13 and those with home-based or non-parental care. 14 – 16 However, further examination of HTE in Head Start is needed because findings on the treatment effects were mixed for many other disadvantaged subgroups, such as children with low parental education level, 17 , 18 special needs, 19 single parents, 20 or caregivers with depressive symptoms.…”

“…A common methodological approach of the previous studies on HTE was a subgroup analysis which restricts the analysis to a subgroup or tests for statistical interactions between the treatment and covariates of interest (e.g., gender, race/ethnicity). 11 However, such an approach has been shown to be insufficient in capturing HTE because it still relies on ATE. 22 While one can test whether ATE estimates are heterogeneous across selected subgroups, heterogeneity around those estimates remains masked.…”

Distributional effects on children's cognitive and social-emotional outcomes in the Head Start Impact Study: A quantile regression approach

“…Since the conclusion of the HSIS, studies have examined the potential heterogeneity of treatment effects of the Head Start, and there have been reviews summarizing the findings (S. Y. Lee et al, 2021 ; Morris et al, 2018 ). Most attempts focused on subgroup or interaction analyses, investigating children's individual or environmental characteristics as sources of effect heterogeneity.…”

“…However, a number of follow-up studies using the same data have revealed substantial variation in the effects of Head Start, with one study reporting a pattern of the Head Start having larger positive effects for systematically more excluded subgroups of children such as Hispanics, those with low maternal education level, and those who had low cognitive measures at baseline (S. Y. Lee et al, 2021 ).…”

“…Other cognitive and social-emotional outcomes at later follow-ups remained unexplored in terms of effect heterogeneity along the outcome distributions (S. Y. Lee et al, 2021 ).…”