Routine outcome measurement in specialist non‐government alcohol and other drug treatment services: Establishing effectiveness indicators for the NADAbase

Abstract

Introduction and Aims The collection of routine outcome measurement (ROM) data provides an opportunity for service providers to conduct benchmarking to inform quality assurance practices. To conduct comparative benchmarking, it is important that services have access to comparative data. This paper aims to establish effectiveness indicators for ROM data collected within the alcohol and other drug (AOD) sector. Design and Methods ROM data were collected by specialist non‐government AOD treatment services within the Network of Alcohol and other Drugs Agencies online database (i.e. NADAbase). All participants were attending treatment within New South Wales, Australia (N = 21 572). Effectiveness indicators were calculated by using effect sizes, standard error of measurement, and rates of reliable and clinically significant change. The study focused on quality of life (EUROHIS Quality of Life Scale), psychological distress (Kessler‐10) and substance dependence (Substance Dependence Scale). Results Since 2010, 21 572 unique people have completed at least one NADAbase Client Outcome Measure. Amphetamines (36%) and alcohol (32%) were the most commonly reported primary substances of concern. Effectiveness indicators were established for the total sample, as well as for people attending residential rehabilitation (n = 8161) and community‐based (n = 10 306) treatment services. Standard error of measurement was the least stringent effectiveness indicator (i.e. a higher proportion of people demonstrated improvement), while the clinically significant change was the most stringent approach. Discussion and Conclusions The study demonstrated the utility of the NADAbase to establish effectiveness indicators for benchmarking purposes. Recommendations are provided for the use of benchmarking to inform quality assurance activities in the sector.

Telehealth Services for Substance Use Disorders During the COVID-19 Pandemic: Longitudinal Assessment of Intensive Outpatient Programming and Data Collection Practices

“…The HBFF has an established infrastructure and process for collection of ROM data that has been used and refined since 1974. ROM data provide an understanding of real-world conditions, offering applied generalizability to community-based treatment settings where the majority of care is provided [ 21 , 22 ]. These data can be designed as a feedback loop, intended to quickly translate findings into treatment implementation [ 23 , 24 ].…”

Evaluating an Aboriginal community controlled residential alcohol and other drug services: Use of benchmarking to examine within treatment changes in wellbeing

“…Various methods have been recommended for calculating effectiveness indicators in the AOD treatment sector. As outlined by Kelly et al [14], the main difference between these approaches is the stringency in which these approaches demonstrate rates of improvement. To inform future replication of the study procedures for other ACCOs, all four effectiveness indicators recommended by Kelly et al [14] were calculated in the present study:…”

“…Benchmarks were established from a relatively large pool of participants who were accessing a representative range of residential services across NSW. Likewise, the study presented a series of clinical indicators specifically recommended for the sector [14]. Historically, there has been limited research examining specialist AOD treatment for Indigenous people internationally [e.g.…”

“…psychological distress and quality of life) during participants stay at The Glen. Using procedures established by Kelly et al [14], effectiveness indicators were calculated including effect sizes, standard error of measurement (SEM) and rates of reliable and clinically significant change. The proportion of participants who demonstrated meaningful change was then benchmarked with other non-ACCO residential services in NSW.…”

“…While benchmarking approaches do not replace the need for randomised controlled trials, they provide opportunities for organisations to more systemically evaluate their service activities and make meaningful comparisons to other service providers in the sector. Within the AOD treatment field, recommended benchmarking procedures using routine outcome measures have been established [14]. This involves using statistical procedures (e.g.…”