Reviewing, Categorizing, and Analyzing the Literature on Black–White Mean Differences for Predictors of Job Performance: Verifying Some perceptions and Updating/Correcting Others

Abstract

In both theoretical and applied literatures, there is confusion regarding accurate values for expected Black-White subgroup differences in personnel selection test scores. Much confusion arises because empirical estimates of standardized subgroup differences (d) are subject to many of the same biasing factors associated with validity coefficients (i.e., d is functionally related to a point-biserial r). To address such issues, we review/cumulate, categorize, and analyze a systematic set of many predictor-specific meta-analyses in the literature. We focus on confounds due to general use of concurrent, versus applicant, samples in the literature on Black-White d. We also focus on potential confusion due to different constructs being assessed within the same selection test method, as well as the influence of those constructs on d. It is shown that many types of predictors (such as biodata inventories or assessment centers) can have magnitudes of d that are much larger than previously thought. Indeed, some predictors (such as work samples) can have ds similar to that associated with paper-and-pencil tests of cognitive ability. We present more realistic values of d for both researcher and practitioner use. Implications for practice and future research are noted.In the literature on personnel selection, as well as organizational application, much attention is given to both (a) the level of validity of individual predictors and (b) the level of adverse impact associated with such predictors. Organizations often focus on selection validity in order to increase the expected performance levels of their hires, increase organizational utility, and defend test use when adverse impact is present. Organizations

Gender Differences in Work Sample Assessments: Not All Tests Are Created Equal

“…For example, cognitive ability is among the most robust predictors of job performance across occupations and criteria (cf. Schmidt & Hunter, 1998); however, there are subgroup differences in cognitive ability scores such that minority group members generally score lower than majority group members (Bobko & Roth, 2013;Roth, Bevier, Bobko, Switzer, & Tyler, 2001). Indeed, a number of predictors traditionally used in personnel selection show group differences that result in adverse impact for minority members of various racial, ethnic, and gender subgroups (Ployhart & Holtz, 2008).…”

“…While revised estimates (i.e., accounting for range restriction) of ethnic group differences in work sample scores appear in the literature (Bobko & Roth, 2013;Roth et al, 2008), much less research has considered this issue for gender differences. Indeed, the few studies that have investigated gender differences in work samples (e.g., Hattrup & Schmitt, 1990;Pulakos, Schmitt, & Chan, 1996) have been conducted using samples of incumbents and as a result, their findings are biased due to range restriction .…”

The effects of predictor method factors on selection outcomes: A modular approach to personnel selection procedures.

“…Minor Black-White differences for self-reports of personality inventories (Bobko & Roth, 2013) Mediocre perceptions for self-reports of personality: metaanalysis (Hausknecht et al, 2004) This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.…”

“…For instance, the bulk of subgroup differences research focused on the stimulus format factor (see review of Schmitt & Quinn, 2010). Therefore, Bobko and Roth (2013) recently advocated that we should parse out selection procedures by other relevant method factors. Widening the scope of factors for reducing subgroup differences is exactly what a modular approach does.…”

The Psychology of Assessment Centers, edited by Duncan J. R. Jackson, Charles E. Lance and Brian J. Hoffman, New York: Routledge, 2012, 360 pp., ISBN: 978-0-4158-7814-2.

“…Several of the authors acknowledge that gender and race have been linked with assessment centre ratings but this issue was not thoroughly addressed, despite people being one of the major sources of variance within measurement models of the type underlying the book's structure. This is no small issue, either with respect to the size of the effects cited in passing within the book, nor with respect to the consequences for non-discriminatory practice (see Bobko and Roth, 2013 for a recent review of one aspect of this).…”