Widening the Disparity Gap: Differences in Outcomes for Racial and Ethnic Groups in Youth Substance Abuse Prevention Programming

Abstract

The retrospective pretest grew out of work by Donald Campbell and Julian Stanley (1963). Campbell and Stanley (1963) outline nine threats to internal validity, the extent to which an evaluator can determine a cause-effect relationship by adequately ruling out alternative explanations. They argue that random assignment of participants to treatment and control conditions is the best method of controlling each of the nine threats. In 1979, George Howard (Howard & Daily, 1979; Howard, Ralph, Gulanick, Maxwell, Nance, & Gerber, 1979) proposed a threat to internal validity that Campbell and Stanley had not considered and that random assignment of participants to control and treatment conditions could not attenuate. This threat is based on instrument effects (Howard, 1980; Howard & Dailey, 1979; Howard, Ralph, et al. 1979), or results produced by the measurement instrument rather than the treatment. Howard called this threat to internal validity "response shift bias" and described it as a change in the participant's metric for answering questions from pretest to posttest due to a new understanding of the concept being investigated (Howard, 1980; Howard, Ralph, et al. 1979). Response shift occurs when participants, rating themselves on self-report measures, use a different internal standard between ratings. To reduce response shift bias, Howard proposed using the retrospective pretest (RPT). Although Campbell and Stanley (1967) had discussed the possible contributions of a retrospective pretest to experimental designs, the notion of response shift was new and created fresh interest in the retrospective design. With the increasing demand for accountability and measurement of change, the retrospective pretest design has gained prominence as a convenient, valid method for measuring self-reported change. It has been shown to reduce response-shift bias providing more accurate assessments of actual effect, is convenient to implement, provides comparison data in the absence of "pre" data, and may be more appropriate in given situations. This review was undertaken to examine the published literature relative to the retrospective pretest design to see what we know about its use, relevance and validity. It covers 49 articles representing sources from educational measurement, psychology, sociology, health, agricultural education, evaluation, extension, management, training, and social work.