The criterion problem: 1917–1992.

Abstract

Individuals differ on multiple aspects of their job-role behavior; criteria are measures that attempt to capture these differences. Measures of criteria are used by several constituencies within applied psychology. Among them, researchers used criteria for the evaluation of theories of work behavior, the effective administration of human resources and the provision of feedback to individuals. One index of the importance of criteria is the observation that most, if not all, of the pioneers of industrial-organizational psychology addressed this issue during their careers. This article reviews conceptual and methodological developments pertaining to the criterion problem since 1917, using as an organizing device dimensions, methods of measurement and analysis, and categorizing frameworks. A shift away from an emphasis on brute prediction toward a balanced treatment of both empirical and conceptual issues is highlighted by calls for the validation of criteria and by increased attention to modeling performance, as well as a recognition of multiple perspectives and competing values from which to view performance and criterion measurement.People in organizations behave according to their role perceptions and the roles assigned to them by others. The resulting behaviors and associated outputs, which are often limited by situations (Viteles, 1925(Viteles, -1926b and may vary widely across individuals (Tiffin, 1942;Viteles, 1932), form a large part of the subject matter of industrial-organizational (I-O) psychology. Individual variability on criterion measures interests researchers and managers, who attempt to measure and influence it. Because criteria are essential for evaluation of individuals, programs, and organizational interventions (Schmitt & Klimoski, 1991), the practical significance of criterion measurement has been recognized since the beginning of industrial psychology (Munsterberg, 1913;Scott, 1917). However, the systematic study of criteria per se to expand scientific knowledge This article is dedicated to Robert L. Thorndike, 1910Thorndike, -1990, who made monumental contributions to the study of criteria during his long career at Teachers College, Columbia University. Both authors contributed equally to the preparation of this article.We thank the following individuals, who critically and constructively commented on earlier versions of this article:

Work Flow Policy and Within-Worker and Between-Workers Variability in Performance.

“…It seems that worker performance is dynamic: Workers' performance relative to each other on a given performance criterion may change over time (the dynamic criterion problem; Austin & Villanova, 1992;Ployhart & Hakel, 1998). Moreover, even at a given point in time, substantial within-worker variability in performance times has been shown to exist (Doerr et al, 2002;Knott & Sury, 1987).…”

Getting Rid of Performance Ratings: Genius or Folly? A Debate

“…Nearly a century of research on performance appraisal (see, for example, Austin & Villanova, 1992) suggests that there is a longstanding history of problems with performance rating and little reason to believe that these problems will be solved in the foreseeable future. The g e t t i ng r i d of pe r f or m a nc e r at i ng s conclusion that performance rating is not working is not solely an academic one; there is evidence of widespread dissatisfaction with performance rating and related techniques in organizational settings; many large organizations (e.g., Accenture, Deloitte, Microsoft, Gap, Inc., Eli Lilly) have abandoned or substantially curtailed their use of performance appraisal (Culbert & Rout, 2010;Cunningham, 2014).…”

The Role-Based Performance Scale: Validity Analysis of a Theory-Based Measure.

“…Sophisticated models for understanding performance appraisal, some that elaborated upon the underlying cognitive process, have been developed and tested throughout the past 30 years (see Cardy & Dobbins, 1994;Murphy & Cleveland, organizations tend to use customized performance measures, which results in utilizing measures that do not typically allow for comparisons among jobs or across companies. Many researchers have noted that this lack of generalizability of the performance criterion hinders the validity of many predictors of performance (Austin & Villanova, 1992).…”