Validity of Treatment Target Progress Ratings as Indicators of Youth Improvement

Abstract

We examined whether clinical progress ratings on the Monthly Treatment and Progress Summary form (MTPS), an idiographic treatment progress measure, were meaningfully related to changes measured by two separate standardized instruments; the Child and Adolescent Functional Assessment Scale (CAFAS) and the Child and Adolescent Level of Care Utilization System (CALOCUS). Validity coefficients were examined at intake and three-, six-, and nine-month follow-up periods. Samples were selected for each measure and follow-up period from the population of youth receiving services through the Hawaii Child and Adolescent Mental Health Division's (CAMHD) system of care. Significant youth improvement was evident on all three measures across all follow-up intervals. The type of changes measured by the MTPS and CAFAS were more alike over longer follow-up intervals than changes measured by the CALOCUS. The MTPS captures distinct aspects of client change that overlaps somewhat with CAFAS and CALOCUS measures. The MTPS is a brief client-tailored measure that seems to provide valid, sensitive, and nonredundant client specific treatment outcome information that can be collected on a frequent basis within a complex system of care.

Longitudinal Predictors of Youth Functional Improvement in a Public Mental Health System

“…A previous study suggested that care coordinator-reported CAFAS scores converged significantly with therapistreported progress ratings, supporting the validity of care coordinator-reported CAFAS scores. 36 …”

“…While clearly more work is needed in this area, the fact that CAFAS scores were moderately correlated with therapist reports of progress on specific treatment targets in this system of care suggests the utility of the measure. 36 Second, the CAFAS is a measure of overall functional impairment while most randomized control trials use diagnosis-specific outcomes making comparisons difficult. More work needs to be done looking at specific symptom improvement among youth in systems of care so that such comparisons can be made more readily.…”

Exploring the Black Box: Measuring Youth Treatment Process and Progress in Usual Care

“…The researchers, along with a panel of practitioners, intervention developers, and other experts, identified 55 practice elements, defined as single strategies that can be identified and shared across EBTs, as the most common techniques used in EBT or UC. From this work, the Monthly Treatment and Progress Summary (MTPS) was created to record the techniques employed during treatment with each client on a monthly basis (Nakamura et al 2007). …”

Outcomes from Multisystemic Therapy in a Statewide System of Care

“…Care coordinators, certified for administration on an annual basis, served as the primary raters for the CAFAS in the current study. A previous study conducted at CAMHD suggested that care coordinator-reported CAFAS scores converged significantly with therapist-reported progress ratings, bolstering the validity of care coordinatorreported CAFAS scores (Nakamura et al 2007). …”