Advancing Motivational Interviewing Training with Artificial Intelligence: ReadMI

Abstract

Background Motivational interviewing (MI) is an evidence-based, brief interventional approach that has been demonstrated to be highly effective in triggering change in high-risk lifestyle behaviors. MI tends to be underutilized in clinical settings, in part because of limited and ineffective training. To implement MI more widely, there is a critical need to improve the MI training process in a manner that can provide prompt and efficient feedback. Our team has developed and tested a training tool, Real-time Assessment of Dialogue in Motivational Interviewing (ReadMI), that uses natural language processing (NLP) to provide immediate MI metrics and thereby address the need for more effective MI training. Methods Metrics produced by the ReadMI tool from transcripts of 48 interviews conducted by medical residents with a simulated patient were examined to identify relationships between physician-speaking time and other MI metrics, including the number of open- and closed-ended questions. In addition, interrater reliability statistics were conducted to determine the accuracy of the ReadMI’s analysis of physician responses. Results The more time the physician spent talking, the less likely the physician was engaging in MI-consistent interview behaviors ( r = −0.403, p = 0.007), including open-ended questions, reflective statements, or use of a change ruler. Conclusion ReadMI produces specific metrics that a trainer can share with a student, resident, or clinician for immediate feedback. Given the time constraints on targeted skill development in health professions training, ReadMI decreases the need to rely on subjective feedback and/or more time-consuming video review to illustrate important teaching points.

Motivational interviewing skills practice enhanced with artificial intelligence: ReadMI

“…The Motivational Interviewing Knowledge Test (MIKT) was completed by students prior to the MI practice sessions and was used to statistically control for MI knowledge in our analyses [ 31 ]. The MIKT contains 22 questions, and the number of questions correct was summed to create a score.…”

“…ReadMI has been found to analyze and categorize the type of question/statement for interview transcripts nearly as well as human raters; the intraclass correlation coefficient was 0.828 [ 31 ]. With respect to correlations between the conversational metrics produced by ReadMI, higher percentages of physician speaking time is significantly related to lower numbers of open-ended questions, fewer reflections, and less use of 0–10 scales [ 31 ]. ReadMI metrics served as data for comparing student interview performance from Role-Play #1 to Role-Play #2.…”

“…We have developed a software-based MI skills-measuring program, “Real-time Assessment of Dialogue in Motivational Interviewing” (ReadMI), that aims to advance the skill acquisition of medical students as they learn the MI approach to address the behaviors that are crucial in effective chronic disease management [ 30 – 31 ]. ReadMI makes use of deep-learning-based speech recognition and natural language processing, implemented through mobile-cloud computing technologies to produce a spectrum of MI-relevant metrics.…”

“…Additionally, a tool such as ReadMI can help make necessary follow-up coaching and feedback sessions more realistic as feedback is less cumbersome to produce [ 32 ]. We previously demonstrated that ReadMI had similar accuracy as human raters in identifying the types of questions and statements spoken by MI trainees [ 31 ]. The purpose of this study was to employ ReadMI as a means to both assess skill development and evaluate the extent to which its use affected the acquisition of MI-relevant skills.…”