THE FUTURE OF CRITERION‐RELATED VALIDITY¹

Abstract

As a result of rejection by personnel psychologists of the erroneous law of small numbers and of the adoption of correct inferential procedures, the future of criterion‐related validity promises to be bright. Probable future developments include: (a) widespread appreciation of the low statistical power characteristic of small samples; (b) rejection of the traditional belief that validities are situationally specific; (c) widespread validity generalization based on new methods; (d) research demonstrations that rational estimates of validity are often superior in accuracy to empirical estimates; (e) widespread acceptance of research findings showing that single‐group and differential validity by race are artifactual rather than substantive in origin and that test unfairness by race is a nonproblem; (f) increased application of decision‐theoretic dollar utility analyses to selection programs; and (g) progress in the establishment of general principles and theories about trait‐performance relations in the world of work.

Meta Analysis of Aircraft Pilot Selection Measures

“…The measures are approximately equally predictive of performance across services and nationalities and (to the extent data are available) in both fixed-wing and rotary-wing aircraft. These findings, therefore support the notions of validity generalization advanced by Schmidt & Hunter (1977Schmidt, 1988).…”

Managerial Applications of Judgmental Biases and Heuristics

“…To achieve the appropriate statistical power, much larger sample sizes (200-300) are needed. Schmidt and Hunter (1980) conclude that personnel psychologists have been believers in the law of small numbers. They have grossly overestimated the amount of information contained in small samples.…”

“…As Schmidt and Hunter (1980) note, psychologists have believed for years that sample sizes of between 30-50 persons are large enough to validate tests. Unfortunately, this assumption is erroneous.…”

Validity

“…It is argued that if this variance component is small. then situational specificity is negligible and the validity in any specific situation is best characterized by the overall average validity (Schmidt, 1987). Tnis extreme position is diametrically opposite to the equally extreme position that test validity is situation-bound, that validity in one situation provides no information bearing on validity in other, even similar situations.…”