The National Evaluation of the Food Stamp Employment and Training Program

We overview the use of field experiments in labor economics. We showcase studies that highlight the central advantages of this methodology, which include: (i) using economic theory to design the null and alternative hypotheses; (ii) engineering exogenous variation in real world economic environments to establish causal relations and learn about the underlying mechanisms; and (iii) engaging in primary data collection and often working closely with practitioners. To highlight the potential for field experiments to inform issues in labor economics, we organize our discussion around the individual life cycle. We therefore consider field experiments related to the accumulation of human capital, the demand and supply of labor, and behavior within firms, and close with a brief discussion of the nascent literature of field experiments related to household decision making. JEL classification: C93; J01

show abstract

“…control group. The practical importance of substitution bias is provided in Puma et al (1990) and Heckman and Smith (1995).…”

Section: Potential Shortcomings Of Social Experimentsmentioning

confidence: 99%

Field Experiments in Labor Economics

List

Rasul

2011

Handbook of Labor Economics

143

103

View full text Add to dashboard Cite

show abstract

“…In contrast, random site selection is rare. The Digest clearly identifies only two social experiments as having selected sites randomly: (1) the Food Stamp Employment and Training Program Evaluation (Puma et al, ), and (2) the Pennsylvania Re‐employment Bonus Demonstration (Corson et al, ). We know of three other social experiments that selected sites randomly, were included in the Digest , but were not clearly identified by the Digest as having selected sites randomly: (1) the National Evaluation of the Upward Bound Program (Seftor, Mamun, & Schirm, ), (2) the Evaluation of Ohio's Learning, Earning, and Parenting Program (Bos & Fellerath, ), and (3) the Evaluation of Florida's Project Independence (Kemple, Friedlander, & Fellerath, ).…”

Section: Introductionmentioning

confidence: 99%

External Validity in Policy Evaluations That Choose Sites Purposively

Olsen¹,

Orr²,

Bell³

et al. 2012

J Policy Anal Manage

105

121

View full text Add to dashboard Cite

Evaluations of the impact of social programs are often carried out in multiple “sites,” such as school districts, housing authorities, local TANF offices, or One-Stop Career Centers. Most evaluations select sites purposively following a process that is nonrandom. Unfortunately, purposive site selection can produce a sample of sites that is not representative of the population of interest for the program. In this paper, we propose a conceptual model of purposive site selection. We begin with the proposition that a purposive sample of sites can usefully be conceptualized as a random sample of sites from some well-defined population, for which the sampling probabilities are unknown and vary across sites. This proposition allows us to derive a formal, yet intuitive, mathematical expression for the bias in the pooled impact estimate when sites are selected purposively. This formula helps us to better understand the consequences of selecting sites purposively, and the factors that contribute to the bias. Additional research is needed to obtain evidence on how large the bias tends to be in actual studies that select sites purposively, and to develop methods to increase the external validity of these studies.

show abstract

“…To take a rather extreme, but important, example, in the Food Stamp Employment and Training Program demonstration, 12-month survey data were successfully collected for only 50% of the research sample of 13,086. This was attributable to difficulties in obtaining addresses from local Food Stamp offices, high mobility among the sample population, and the large number of homeless persons who were randomly assigned (Puma, Burstein, Merrell, & Silverstein, 1990).…”

Section: Sample Attritionmentioning

confidence: 99%