AI avatar tells you what happened: The first test of using AI-operated children in simulated interviews to train investigative interviewers

Abstract

Previous research has shown that simulated child sexual abuse (CSA) interview training using avatars paired with feedback and modeling improves interview quality. However, to make this approach scalable, the classification of interviewer questions needs to be automated. We tested an automated question classification system for these avatar interviews while also providing automated interventions (feedback and modeling) to improve interview quality. Forty-two professionals conducted two simulated CSA interviews online and were randomly provided with no intervention, feedback, or modeling after the first interview. Feedback consisted of the outcome of the alleged case and comments on the quality of the interviewer’s questions. Modeling consisted of learning points and videos illustrating good and bad questioning methods. The total percentage of agreement in question coding between human operators and the automated classification was 72% for the main categories (recommended vs. not recommended) and 52% when 11 subcategories were considered. The intervention groups improved from first to second interview while this was not the case in the no intervention group (intervention x time: p = 0.007, ηp2 = 0.28). Automated question classification worked well for classifying the interviewers’ questions allowing interventions to improve interview quality.

Effects of combining feedback and hypothesis-testing on the quality of simulated child sexual abuse interviews with avatars among Chinese university students

“…The current study was the first to examine the efficacy of Avatar Training on improving interview quality in the Chinese context. Consistent with previous findings [ 26 – 28 , 30 , 33 ], feedback (combined on question types and case outcomes) showed positive effects on both the questioning skills of the interviewers and the amount of elicited information from avatars, this latter finding being a function of the algorithms driving the avatars. Apart from this, our main focus was to look at the effects of hypothesis-testing on interviewing behavior.…”

“…This result was inconsistent with previous Avatar Training studies. Previous studies have demonstrated that feedback positively influenced the probability of correct conclusions [ 26 – 30 , 33 ]. For example, Pompedda et al [ 30 ] found that the percentage of groups receiving both feedback (outcome and process) was higher (29%) than the group receiving feedback on process (21%), the group receiving feedback on the outcome (15%), and the control group (17%).…”

“…Importantly, this training effect transferred into interviews with real children about a mock event sharing features with sexual abuse [ 31 ] as well as to actual investigative interviews in criminal cases [ 32 ]. Regarding the applicability of Avatar Training, previous studies conducted in both Western contexts [ 26 , 29 , 30 , 31 ] as well as in Japan [ 27 , 28 , 33 ], have supported its effectiveness.…”

Enhancing questioning skills through child avatar chatbot training with feedback

“…The trainees formulated questions orally, but until now, the system required an operator to be present to manually code into the software the types of questions asked and thereby to activate a pre-recorded child response ( Haginoya et al, 2021 ). Lately, the group has undertaken a study using an automatized, simulated avatar ( Haginoya et al, 2023 ). In this study, professionals were asked to conduct two child sexual abuse interviews with an online avatar in which the two conditions were with pedagogical intervention (modeling or feedback) or without.…”

“…The feedback system, which was tested for its accuracy in classifying interviewer questions (as either recommended or non-recommended), showed an overall agreement with manual coders of 72% (chance level being 33%), with a Cohen’s kappa (κ) of 0.49. The results showed improvement in the use of recommended question types (broad and focused invitations, facilitators, directives, and clarifications) in the pedagogical intervention (both modeling and feedback) condition ( Haginoya et al, 2023 ).…”