Likelihood-Based Analysis of Causal Effects of Job-Training Programs Using Principal Stratification

Abstract

Government-sponsored job-training programs must be subject to evaluation to assess whether their effectiveness justifies their cost to the public. The evaluation usually focuses on employment and total earnings, although the effect on wages is also of interest, because this effect reflects the increase in human capital due to the training program, whereas the effect on total earnings may be simply reflecting the increased likelihood of employment without any effect on wage rates. Estimating the effects of training programs on wages is complicated by the fact that, even in a randomized experiment, wages are ''truncated'' (or less accurately ''censored'') by nonemployment, that is, they are only observed and well-defined for individuals who are employed. In this article, we develop a likelihood-based approach to estimate the wage effect of the US federally-funded Job Corps training program using ''Principal Stratification''. Our estimands are formulated in terms of: (1) the effect of the training program on wages for those who would be employed whether they were trained or not, also called the survivor average causal effect (SACE), and the proportion of people in this category; (2) the wages when trained for those who would be employed only when trained, and the proportion of people in this category; (3) the wages when not trained for those who would be employed only when not trained, and the proportion of people in this category; (4) the proportion of people who would be not employed whether trained or not. We conduct likelihood-based analysis using the EM algorithm, and investigate the plausibility of important submodels with scaled loglikelihood ratio statistics. We also conduct a sensitivity analysis with respect to specific parametric assumptions. Our results suggest that all four types of people [(1)-(4) previously] exist, which is impossible under the usual monotonicity assumptions made in traditional econometric evaluation methods.

Compared to what? Variation in the impacts of early childhood education by alternative care type

“…Some previous education examples include Barnard et al (2003) on the effect a randomized lottery for private school voucher use in New York City with complex noncompliance patterns (see also, Jin and Rubin, 2009); Page (2012) on the relative importance of student exposure to the labor market in career academy high schools; and Schochet (2013) on student mobility in school-based randomized trials. Outside of education, several studies have used principal stratification to analyze the JobCorps evaluation (e.g., Zhang et al, 2009;Frumento et al, 2012) and JOBS II evaluation (Mattei et al, 2013). Finally, a separate series of papers use a principal score approach, rather than model-based inference, to estimate similar quantities of interest.…”

“…Zhang et al (2009) point to two main functions. First, covariates can be predictive of the outcome and stratum membership.…”

“…We provide a brief sketch of the idea here. The Appendix provides additional discussion of identification in principal stratification models (see also Zhang et al, 2009). …”

“…In practice, we could estimate the full parametric model from either a likelihood or Bayesian perspective. Indeed, some prominent applications of modelbased principal stratification utilize a direct likelihood approach (e.g., Zhang et al, 2009;Frumento et al, 2012). This approach is quite flexible and allows for straightforward comparisons between different models.…”

Nordic Economic Policy Review, 2014 no 1

“…They use data from a quasi-randomized-out control group to investigate programme effects and find positive effects on postprogramme employment rates. Flores et al (2012) and Zhang et al (2012) also investigate educational programmes (Job Corps) for uneducated youths in the US and find positive effects on future earnings.…”