The Effect of AI-generated Advice on Decision-Making in Personnel Selection

Abstract

Despite the rise of decision support systems enabled by artificial intelligence (AI) in personnel selection, their impact on decision-making processes is largely unknown. Consequently, we conducted five experiments (N = 1,403) investigating how people interact with AI-generated advice in a personnel selection task. In all pre-registered experiments, we presented correct and incorrect advice. In Experiments 1a and 1b, we manipulated the source of the advice (human vs. AI). In Experiments 2a, 2b, and 2c, we further manipulated the type of explainability of AI advice (2a and 2b: heatmaps and 2c: charts) to test if explainable advice improves decision-making. The independent variables were regressed on task performance, perceived advice quality and confidence ratings. The results consistently showed that incorrect advice negatively impacted performance, as people failed to dismiss it (i.e., overreliance). Additionally, we found that the effects of source and explainability of advice on the dependent variables were limited.

A signal detection perspective on error and unfairness detection as a critical aspect of human oversight of AI-based systems

“…We are not aware of research in AI-based decision-making that has explicitly considered an SDT perspective for the detection of AI-based system errors although there are studies that could be analyzed and interpreted from an SDT perspective (studies with several rounds of AI-supported decision-making, rounds with correct and incorrect outputs, as well as decisions regarding whether to follow or reject system outputs; see e.g., Cecil et al, 2023;Gaube et al, 2021;Green & Chen, 2019;Schoeffer et al, 2023). We propose that SDT provides a promising general framework for studying human error detection in the context of AI-based systems, as it a) provides measures to quantify the effectiveness of error detection b) distinguishes sensitivity and response bias as two crucial aspects of human decision-making in error detection, and c) can be used to study the factors that influence sensitivity and response bias.…”