Automatic identification of storytelling responses to past‐behavior interview questions via machine learning

Abstract

Structured interviews often feature past‐behavior questions, where applicants are asked to tell a story about past work experience. Applicants often experience difficulties producing such stories. Automatic analyses of applicant behavior in responding to past‐behavior questions may constitute a basis for delivering feedback and thus helping them improve their performance. We used machine learning algorithms to predict storytelling in transcribed speech of participants responding to past‐behavior questions in a simulated selection interview. Responses were coded as to whether they featured a story or not. For each story, utterances were also manually coded as to whether they described the situation, the task/action performed, or results obtained. The algorithms predicted whether a response features a story or not (best accuracy: 78%), as well as the count of situation, task/action, and response utterances. These findings contribute to better automatic identification of verbal responses to past‐behavior questions and may support automatic provision of feedback to applicants about their interview performance.

Diversity and technology—Challenges for the next decade in personnel selection

“…The third study investigates the effectiveness of countermeasures against faking in interviews and finds that none of the countermeasures tested could reduce faking intentions or faking (Bill & Melchers, 2023). The fourth study uses machine learning algorithms to predict storytelling in responses to pastbehavior questions in a simulated selection interview, with potential implications for automatic provision of feedback to applicants (Bangerter et al, 2023). The fifth study compares trust, trustworthiness, and trusting behavior for different types of decision‐support (automated, human, and hybrid) across two assessment contexts (personnel selection and bonus payments) and examines trust violations (Kares et al, 2023).…”

“…How do different stakeholders react to the use of novel technology? The papers by Bill and Melchers (2023), Bangerter et al (2023), Kares et al (2023), and Köchling and Wehner (2022) included in this Special Issue address such questions.…”

“…Bangerter et al (2023) investigate to what extent it is possible to automatically identify storytelling behavior in responses to job interview questions. The findings show an accuracy of up to 78% for classifying whether a response includes a story.…”