Does the use of alternative predictor methods reduce subgroup differences? It depends on the construct

Abstract

Using a bare-bones meta-analysis, the present study examined the effectiveness of the use of commonly considered alternative predictor methods as a means to reduce subgroup differences (i.e., the method-change approach), taking into account the constructs assessed. With a focus on assessment centers, interviews, situational judgment tests, and work samples as alternative methods, the results indicated that consonant with the construct/method distinction, the effectiveness of a method in reducing subgroup differences is a function of the constructs assessed. Specifically, there are larger White-African American subgroup differences that favor Whites for cognitive constructs on paper-and-pencil tests compared to the alternative methods; and most notably, the opposite result was obtained for noncognitive constructs such that, compared to paper-and-pencil assessments, substantially larger White-African American subgroup differences were observed for alternative methods. A similar pattern of results was obtained for White-Hispanic comparisons, albeit based on a smaller number of data points. In summary, the study's results indicate that the ubiquitously asserted effectiveness of the method-change approach for reducing subgroup differences is overstated, with said effectiveness depending on the construct assessed.

Using natural language processing to increase prediction and reduce subgroup differences in personnel selection decisions.

“…We then show how this might translate into reduced adverse impact based on realistic hypothetical selection rates. This research demonstrates the construct-change approach to reducing subgroup differences in selection (Arthur et al, 2021). Aside from measuring additional constructs, such reductions may in part be because text scores are not subjected to human evaluation and the potential for intentional discrimination.…”

“…Moreover, we demonstrate that the inclusion of this information reduces subgroup differences in a composite score based on the idea that greater content coverage in predictors reduces differences among subgroups, depending on the relative subgroup differences and intercorrelations (Sackett & Ellingson, 1997). We compare the variables captured through NLP to mental ability to assess this reduction in subgroup differences through the construct-change approach (Arthur & Villado, 2008; Arthur et al, 2013, 2021).…”