Self-Report After Randomly Assigned Supervision Does not Predict Ability to Practice Motivational Interviewing

Abstract

The objective of this study was to investigate the relation between self-report and objective assessment of Motivational Interviewing (MI) skills following training and supervision. After an MI workshop, 96 clinicians from 26 community programs (age 21–68, 65% female, 40.8% Black, 29.6% Caucasian, 24.5% Hispanic, 2.0% Asian, 3.1% other) were randomized to supervision (tele-conferencing or tape-based), or workshop only. At four time points, trainees completed a self-report of MI skill, using items from the MI Understanding questionnaire (MIU), and were objectively assessed by raters using the Motivational Interviewing Treatment Integrity (MITI) system. Correlations were calculated between MIU and MITI scores. A generalized linear mixed model was tested on MIU scores, with MITI scores, supervision condition and time as independent variables. MIU scores increased from pre-workshop (Mean = 4.74, SD = 1.79) to post-workshop (Mean = 6.31, SD = 1.03) (t = 8.69, p < .0001). With supervision, scores continued to increase, from post-workshop to week 8 (Mean = 7.07, SD = 0.91, t = 5.60, p < .0001) and from week 8 to week 20 (Mean = 7.28, SD = 0.94, t = 2.43, p = .02). However, MIU scores did not significantly correlate with MITI scores, with or without supervision. Self- reported ability increased with supervision, but self-report was not an indicator of objectively measured skill. This suggests that training does not increase correspondence between self-report and objective assessment, so community treatment programs should not rely on clinician self- report to assess the need for ongoing training and supervision and it may be necessary to train clinicians to accurately assess their own skill.

Effectiveness of Motivational Interviewing Training for Primary Care Dentists and Dental Health Technicians: Results from a Community Clinical Trial

“…29 In that study, the latter group had greater and increasing results over time, reinforcing the importance of contemplating mechanisms for review, follow-up, and reinforcement of the main training. A community trial compared physicians who received only one training with another group that received additional teleconference supervision or recordings.…”

“…A community trial compared physicians who received only one training with another group that received additional teleconference supervision or recordings. 29 In that study, the latter group had greater and increasing results over time, reinforcing the importance of contemplating mechanisms for review, follow-up, and reinforcement of the main training.…”

“…These indings agree with other studies on using MI in other professional categories. 27,29 The quality of training and veriication of the assessment tools are extremely important for maintaining long-term results. In our study, training was performed with a highly qualiied and experienced MI practitioner.…”

Qualitative and quantitative feedback following workshop training in evidence-based practices: A dissemination study.

“…First, this initial dissemination effort included no objective assessments of skill acquisition. Research has consistently demonstrated a tendency for trainees to overestimate their skills (Kruger & Dunning, 1999), an effect that has been repeatedly demonstrated in both medicine (Davis et al, 2006; Violato & Lockyer, 2006) and psychotherapy (Mathieson, Barnfield, & Beaumont, 2009; Wain et al, 2015). The relationship between the perceived and actual benefits of training is unknown in the present sample of workshop attendees.…”

Student-Perceived Quality of Motivational Interviewing Training: A Factor-Analytic Study

“…Many substance-use treatment providers indicate that they are using MI principles in practice (Gifford et al, 2012), and it seems unlikely that practice-as-usual settings provide the MI intervention in its entirety and with high fidelity (Santa Ana et al, 2008). For example, professionals in the field overestimate their MI skills when their self-reported MI skills are compared to neutral expert ratings of their skills (Wain et al, 2015). Similarly, some research studies nominally of MI have included interventions not recognizable as MI (Miller & Rollnick, 2013), potentially skewing findings from metaanalyses on the effectiveness of MI studies.…”