Forming Composites of Cognitive Ability and Alternative Measures to Predict Job Performance and Reduce Adverse Impact: Corrected Estimates and Realistic Expectations

Abstract

Although there has been empirical attention paid to the criterion-related validity of predictor composites, there has been much less attention paid to the standardized ethnic group differences associated with these composites. One important area of inquiry in predictor composite research is the influence of adding predictors to a test of general mental ability. The limited empirical literature on this practice is mixed, but the prevailing expectation is that there is likely to be higher validity and less adverse impact. Unfortunately, much of the previous work is limited by the presence of inaccurate validity and standardized ethnic group difference values. In this analysis we formed meta-analytic matrices to more accurately estimate the validity and standardized ethnic group differences of several composites that combine a measure of cognitive ability with measures of conscientiousness, a structured interview, or biodata. While results were somewhat complex, we found that adding alternative predictors does not result in a situation in which validity automatically goes up and adverse impact potential automatically goes down. In fact, the reductions in adverse impact (if any) from adding ''non-cognitive'' predictors were more modest than much of the literature suggests.

Criterion-Focused Approach to Reducing Adverse Impact in College Admissions

“…Some of these studies have attempted to reduce adverse impact by using noncognitive measures to supplement tests of cognitive ability (De Corte, 1999;Potosky, Bobko, & Roth, 2005;Schmitt, Rogers, Chan, Sheppard, & Jennings, 1997). A concern about low selection rates for minority students has led to an increased interest in the inclusion of standardized noncognitive predictors in admission decisions (Thomas, Kuncel, & Crede, 2004).…”

Meta-Analysis of the Relationship Between the Big Five and Academic Success at University

“…Kyllonen, Walters, and Kaufman (2005) hypothesize that using noncognitive factors in addition to cognitive factors could increase fairness in admission decisions for graduate education, because gender and race differences tend to be smaller in noncognitive factors than in cognitive factors (for an opposing view see Potosky, Bobko, & Roth, 2005). Goldberg (2001) noted that including measures of personality in university admission procedures could increase diversity, which is diminished by using primarily cognitive tasks.…”

Practical Impact of Predictor Reliability for Personnel Selection Decisions

“…Our results will differ from Roth et al because we are also considering predictors corrected for measurement error variance, and we are examining a different level of unreliability for the performance criterion. (Hunter, 1986) and .52 (Salgado, Anderson, Moscoso, Bertuna, & de Fruyt, 2003a;Salgado et al, 2003b) for cognitive ability, .60 for structured interview (Potosky et al, 2005), .59 for conscientiousness (Hurtz & Donovan, 2000), and .64 for biodata (Rothstein, Schmidt, Erwin, Owens, & Sparks, 1990). The average of these values for criterion unreliability across all predictors were used for correction (.59).…”

“…Both cases are considered compensatory models, because higher scores on some predictors will compensate for lower scores in others (Potosky, Bobko, & Roth, 2005). For example, if scores on cognitive ability and conscientiousness measures are standardized, regression weighted, and combined for use in top-down selection, then higher cognitive ability scores will compensate for lower conscientiousness scores, and conversely, higher conscientiousness scores will compensate for lower ability scores.…”