Notes from the youth mental health field: Using movement towards goals as a potential indicator of service change and quality improvement

Abstract

The aim of this paper is to report our notes from the field on using movement toward goals at an aggregate level as an inference of service effectiveness. Analysis of routinely collected data from UK youth mental health services was conducted (N = 8,172, age M = 13.8, 67% female, 32% male) to explore the impact of including goal‐based outcome data in combined calculations of standardized measures based on the principles of reliable change (“measurable change”). Due to the broad nature of standardized measures, inferred validity becomes diluted in any team or service level aggregate analysis. To make inferences that are closer to the person's interpretation of their difficulties, we argue that Idiographic Patient Reported Outcome Measures (I‐PROMs) counterbalance these limitations. This is supported by our findings. The measurable change metric is the first step towards enabling national analysis of aggregated I‐PROMs. I‐PROMs, supplemented by standardized measures should be used to consider service evaluation.

The Session Wants and Need Outcome Measure: The Development of a Brief Outcome Measure for Single-Sessions of Web-Based Support

“…The use of an I-PROM for single-sessions will also facilitate routinely data collection. So aggregated data from the measure across service users can be collected for monitoring data on the analysis of this I-PROM by aggregating the scores of the common goals that you can find in the instrument, while providing the option to develop individual goals (Jacob et al, 2021). I-PROMs have increased currency especially for personcentred support service provision within children and young people population, while other measures can be used, no idiographic measure exists or has been designed specifically for single-session work.…”

Special section: Idiographic tools for routine outcome monitoring in psychotherapy

“…Duncan et al (2023) draw on data from a randomized controlled to examine the psychometric properties of the Goal-Based Outcome tool (GBO tool, Law & Jacob, 2015)-the most widely used I-PROM for children and young people-including sensitivity to intervention effects. A second paper on the GBO tool assesses its current roll out by England's National Health Service as a potential indicator of service-level change (Jacob et al, 2023). Jacob et al examine how I-PROMs might be used at this level, and the added value it brings to nomothetic ROM.…”

Idiographic patient reported outcome measures (I‐PROMs) for routine outcome monitoring in psychological therapies: Position paper

“…In addition, thematic analysis of I‐PROM free‐text items indicates that more than 50% of I‐PROM contents are not featured in the pre‐set items of their nomothetic PROMs counterparts, creating considerable nonoverlap between these two modalities of outcome measurement (Alves et al, 2020; Ashworth et al, 2007; Sales et al, 2018). This may be why, when measuring change on items that are of key importance to the patient, I‐PROMs change scores have consistently been found to exceed nomothetic PROMs change scores for both adults and youth (Ashworth et al, 2005; Edbrooke‐Childs et al, 2015; Elliott et al, 2016; Jacob et al, 2021; Karpenko & Owens, 2013; Krause et al, 2020).…”

“…Alternative methods have also been developed to accommodate the specific nature of I‐PROM data. Currently, for instance, in Child and Adolescent Mental Health Services in England, NHS England and NHS Improvement are trialing an outcomes metric which incorporates change on at least one goal on the GBO tool, utilizing the principles of the reliable change index (Jacob et al, 2021; Jacobson & Truax, 1991). This outcome metric is in its “shadow year” and its findings will be watched with interest.…”

“…In analysis of service outcomes based on this principle, the combining of data from an I‐PROM with data from nomothetic measures gave higher levels of reliable improvement compared to nomothetic measures alone. This suggests that important improvements may be missed without the use of I‐PROMs data at the service level (Jacob et al, 2021).…”