A unique application of the Many-Facet Rasch Model (MFRM) is introduced as the preferred method for evaluating the psychometric properties of a measure of therapist adherence to Contingency Management (CM) treatment of adolescent substance use. The utility of psychometric methods based in Classical Test Theory was limited by complexities of the data, including: (a) ratings provided by multiple informants (i.e., youth, caregivers, and therapists), (b) data from separate research studies, (c) repeated measurements, (d) multiple versions of the questionnaire, and (e) missing data. Two dimensions of CM adherence were supported: adherence to Cognitive Behavioral components and adherence to Monitoring components. The rating scale performed differently for items in these subscales, and of 11 items evaluated, eight were found to perform well. The MFRM is presented as a highly flexible approach that can be used to overcome the limitations of traditional methods in the development of adherence measures for evidence-based practices.
Keywords
Contingency Management; therapist adherence; Rasch modelThe Contingency Management (CM) treatment model developed by Azrin and his colleagues (Donohue & Azrin, 2001) has shown promise in treating adolescent substance use in conjunction with family therapy. Results from randomized trials demonstrated significantly better outcomes for youths in the CM conditions, compared with supportive counseling, for drug use abstinence, mental health and conduct problems, and employment/school attendance (Azrin, Acierno et al., 1996;. To ensure the integrity of CM implementation in these clinical trials, Azrin and colleagues provided intensive and ongoing oversight of the therapy sessions. This oversight included (1) extensive training, modeling, and role-playing with corrective feedback to therapists; (2) ongoing audiotape coding with corrective feedback provided to therapists; (3) observation of therapy sessions with corrective feedback; (4) detailed session checklists prompting use of techniques, reviewed weekly with therapists; and (5) weekly written documentation by therapists of techniques used, youth and family participation, and progress toward treatment goals. Notably, fidelity data from session checklists and ratings of audiotapes showed greater than 95% adherence , 1997;Henggeler, Pickrel, & Brondino, 1999;Schoenwald, Sheidow, Letourneau, & Liao, 2003). Likewise, favorable outcomes for other evidence-based practices (e.g., Functional Family Therapy, Program for Assertive Community Treatment) have been attenuated by low therapist fidelity to the corresponding treatment protocols (Weisz, Weersing, & Henggeler, 2005).If achieving desired clinical outcomes relies, in part, on the fidelity of therapist implementation of the treatment model, then attempts to transport evidence-based practices to community settings requires effective measures of treatment fidelity. Although the observational methods used for monitoring and measuring fidelity in the CM trials noted previously were highly...