Implementing evidence-based assessment and selection in organizations: A review and an agenda for future research

Abstract

In personnel- and educational selection, a substantial gap exists between research and practice, since evidence-based assessment instruments and decision-making procedures are underutilized. We provide an overview of studies that investigated interventions to encourage the use of evidence-based assessment methods, or factors related to their use. The most promising studies were grounded in self-determination theory. Training and autonomy in the design of evidence-based assessment methods were positively related to their use, while negative stakeholder perceptions decreased practitioners’ intentions to use evidence-based assessment methods. Use of evidence-based decision-making procedures was positively related to access to such procedures, information to use it, and autonomy over the procedure, but negatively related to receiving outcome feedback. A review of the professional selection literature showed that the implementation of evidence-based assessment was hardly discussed. We conclude with an agenda for future research on encouraging evidence-based assessment practice.

Horse-sized ducks or duck-sized horses? Oddball Personality Questions are likable (but useless) for organizational recruitment

“…In doing so, we hope managers will be less inclined to ask OPQs. Still, there is considerable research documenting the wide gap between science and practice, particularly in the context of employee selection (Neumann et al, 2020;Rynes, 2009). In a survey of over 400 human resource professionals, Fisher et al (2020) found that they were worse at identifying personnel selection myths than they were almost two decades ago.…”

The Autonomy-validity Dilemma in Mechanical Prediction Procedures: The Quest for a Compromise

“…Furthermore, decision makers are overly confident in their holistic predictions (Kausel et al, 2016;Sieck & Arkes, 2005), have the desire to take an individual's uniqueness and context into account (Longoni et al, 2019;Newman et al, 2020), and wrongly believe that they can account for valid exceptions to a mechanical rule (Dietvorst et al, 2018;Guay & Parent, 2018;Hoffman et al, 2017;Meehl, 1954b). Overall, a main reason seems to be that holistic prediction better satisfies decisionmakers' fundamental needs (Neumann et al, 2020). According to self-determination theory, people have three fundamental needs: autonomy, competence, and relatedness (Deci & Ryan, 2000).…”

“…All three needs may play a role in the implementation of mechanical prediction (Nolan, 2013). Yet, among existing studies that have focused on interventions to implement mechanical prediction procedures, the most promising interventions increased decision-makers' autonomy in mechanical prediction in some way (Kaplan et al, 2001;Neumann et al, 2020). Given these prior findings and its solid theoretical basis, we focused on autonomy as the main variable of interest in this paper.…”

“…Although studies demonstrating the superiority of mechanical prediction have been published for decades (Grove et al, 2000;Meehl, 1954a;Sarbin, 1943;Sawyer, 1966), there still exists a large gap between science and practice. Even in highly consequential contexts such as admission to higher education, personnel selection, and clinical treatment selection, decision makers underutilize mechanical prediction (Arkes, 2008;Highhouse, 2008;Kirch, 2012;Neumann et al, 2020;Ryan et al, 2015;Ryan & Sackett, 1987;Vrieze & Grove, 2009). This phenomenon has also been referred to as algorithm aversion (Dietvorst et al, 2015).…”

“…A major reason why decision makers underutilize mechanical rules is that it restricts their autonomy (Neumann et al, 2020;Nolan & Highhouse, 2014). Some studies found that providing decision makers with autonomy increased their (intended) use of mechanical rules (Dietvorst et al, 2018;Nolan & Highhouse, 2014).…”