Measures of treatment integrity are needed to advance clinical research in general and are viewed as particularly relevant for dissemination and implementation research. Although some efforts to develop such measures are underway, a conceptual and methodological framework will help guide these efforts. The purpose of this article is to demonstrate how frameworks adapted from the psychosocial treatment, therapy process, healthcare, and business literatures can be used to address this gap. We propose that components of treatment integrity (i.e., adherence, differentiation, competence, alliance, client involvement) pulled from the treatment technology and process literatures can be used as quality indicators of treatment implementation and thereby guide quality improvement efforts in practice settings. Further, we discuss how treatment integrity indices can be used in feedback systems that utilize benchmarking to expedite the process of translating evidence-based practices to service settings.

No abstract

Observational measures to assess implementation integrity (the extent to which components of an evidence-based treatment are delivered as intended) are needed. We evaluated the reliability of the scores and the validity of the score interpretations for the Therapy Process Observational Coding System for Child Psychotherapy – Revised Strategies scale (TPOCS-RS; McLeod, 2010) and assessed the potential of the TPOCS-RS to assess treatment differentiation, a component of implementation integrity. The TPOCS-RS includes five theory-based subscales (Cognitive, Behavioral, Psychodynamic, Client-Centered, Family). Using the TPOCS-RS, coders independently rated 954 sessions conducted with 89 children (M age = 10.56, SD = 2.00; aged 7–15 years; 65.20% Caucasian) diagnosed with a primary anxiety disorder who received different treatments (manual-based vs. non-manualized) across settings (research vs. practice). Coders produced reliable ratings at the item level (M ICC = .76, SD = .18). Analyses support the construct validity of the Cognitive and Behavioral subscale scores and, to a lesser extent, the Psychodynamic, Family, and Client-Centered subscale scores. Correlations among the TPOCS-RS subscale scores and between the TPOCS-RS subscale scores and observational ratings of the alliance and client involvement were moderate suggesting independence of the subscale scores. Moreover, the TPOCS-RS showed promise for assessing implementation integrity as the TPOCS-RS subscale scores, as hypothesized, discriminated between manual-guided treatment delivered across research and practice settings and non-manualized usual care. The findings support the potential of the TPOCS-RS Cognitive and Behavioral subscales to assess treatment differentiation in implementation research. Results for the remaining subscales are promising, although further research is needed.

Does delivery of the same manual-based individual cognitive-behavioral treatment (ICBT) program for youth anxiety differ across research and practice settings? We examined this question in a sample of eighty-nine youths (M age = 10.56, SD = 1.99; 63.70% Caucasian; 52.80% male) diagnosed with a primary anxiety disorder. The youths received (a) ICBT in a research setting, (b) ICBT in practice settings, or (c) non-manual-based usual care (UC) in practice settings. Treatment delivery was assessed using four theory-based subscales (Cognitive-behavioral, Psychodynamic, Client-Centered, Family) from the Therapy Process Observational Coding System for Child Psychotherapy–Revised Strategies scale (TPOCS-RS). Reliable independent coders, using the TPOCS-RS, rated 954 treatment sessions from two randomized controlled trials (1 efficacy and 1 effectiveness trial). In both settings, therapists trained and supervised in ICBT delivered comparable levels of cognitive-behavioral interventions at the beginning of treatment. However, therapists trained in ICBT in the research setting increased their use of cognitive-behavioral interventions as treatment progressed whereas their practice setting counterparts waned over time. Relative to the two ICBT groups, the UC therapists delivered a significantly higher dose of psychodynamic and family interventions and a significantly lower dose of cognitive-behavioral interventions. Overall, results indicate that there were more similarities than differences in manual-based ICBT delivery across research and practice settings. Future research should explore why the delivery of cognitive-behavioral interventions in the ICBT program changed over time and across settings, and whether the answers to these questions could inform implementation of ICBT programs.

Objective Little is known about the nature of the relationship between the alliance and client involvement in child psychotherapy. To address this gap, we examined the relationship between these therapy processes over the course of cognitive-behavioral therapy (CBT) for child anxiety disorders. Method The sample was 31 child participants (Mage = 9.58 years, SD = 2.17, range 6–13 years, 67.7% boys; 67.7% Caucasian, 6.5% Latino, 3.2% Asian/Pacific Islander, and 22.6% mixed/other) diagnosed with a primary anxiety disorder. The participants received a manual-based individual CBT program for child anxiety or a manual-based family CBT program for child anxiety. Ratings of alliance and client involvement were collected on early (session two) and late (session eight) treatment phases. Two independent coding teams rated alliance and client involvement. Results Change in alliance positively predicted late client involvement after controlling for initial levels of client involvement. In addition, change in client involvement positively predicted late alliance after controlling for initial levels of the alliance. The findings were robust after controlling for potentially confounding variables. Conclusions In CBT for child anxiety disorders, change in the alliance appears to predict client involvement; however, client involvement also appears to predict the quality of the alliance. Our findings suggest that the nature of the relationship between alliance and client involvement may be more complex than previously hypothesized. In clinical practice, tracking alliance and level of client involvement could help optimize the impact and delivery of CBT for child anxiety.