Criterion Measures for Youth‐work Training Programs: The Development of Relevant Performance Dimensions

Abstract

An attempt to develop relevant criterion measures for work-training programs serving disadvantaged adolescents involved the longitudinal application of 32 trainee performance criteria at program completion and 40 longer-term outcome measures approximately six months following training. Factor analyses of those variables yielded four program completion (prOXimal) criterion dimensions of work motivation and planning ability, personal-social adjustments and training program competency. Post-program (distal) variables resulted in five dimensions encompassing areas of social and community adjustments, skill in job planning, ability to perform on jobs and to seek or find employment. Loading magnitudes and factor patterns provided a basis for identifying those variables with the greatest concurrent relevance; whereas, correlations between proximal and distal criteria pinpointed those end-ofprogram outcomes of greatest predictive relevance (e.g., proficiency ratings by counselors, work supervisors and peers; family adjustments, favorable self and social perceptions). This empirical approach to criterion development was shown to provide better understanding of the multidimensional aspects of a criterion donmin and information for setting priorities in the choice of criterion variables. training, the roots of that uncertainty have been attributed directly to "imprecise gOc;l;l statements" and lack of "properly designed systems for planning, executing and using evaluation studies" (Hholey, Scanlon, D,~ffy, Fukumoto, & Vogt, 1970). Criterion MeasuresMoves to fill the void have stimulated a spate of 1iteratur~aimed at clarifying re~earch strategies and analytical methods that would allow for more systematic evaluation of the programs (Barth, 1972;Borus, 1972;Borus & Tash, 1970; National League of Cities, 1973; Rossi, 1969, pp. 217-234; U.S. , 1974; Walther, Magnusson, & Cherkasky, 1969). Unfortunately, there remains a major shortcoming in those "how-to" mat~rials, as well as in much broader theoretical models that attempt to define approaches to social pro~:am evaluation (Wortman, 1975). The deficiency is~ssentia1ly a methodologic~l one~spawned by avoidance of basic issues and t7~~niques Department of Laborneeded for the derivation of criterion tools that serve as critica.l elements in any evaluative and research efforts. Criterion Measures 3Instead of verifiable procedures, the prescriptions prOVided for specifying training objectives and associated measures appear to begin, and end, with purely consensual or rational decisions, usually leading to a laundry list of recommended performance outcomes that somehow appeal to the particular writer. The conceptual framework and procedures for formulating criterion constructs--long espoused in the research literature from Thorndike's (1949, chap. 5) and Cureton's (1951, pp. 621-694) early presentations to the more recent work of Dunnette (1967), Guion (196l), Ghise11i (1956), Cronbach (1971, and Wherry (1957)--seem to have gone unrecognized as a crucial aspect of the evalu...

Predicting Job Performance of Youth Trainees Under a Job Training Partnership Act Program (Jtpa): Criterion Validation of a Behavior‐based Measure of Work Maturity

“…The issue of predicting the job performance of low-income workers has also been addressed by Freeberg (1976), although the primary intent of his study was to provide an empirical method for developing criteria for program evaluation and test validation. He found that measures taken immediately after program completion were significantly related to work-related variables measured six months later.…”