Achievable benchmarks of care: the ABC<SUP>TM</SUP>s of benchmarking

Abstract

Benchmarking is generally considered to be an important tool for quality improvement. Traditional approaches to benchmarking have relied on subjective identification of 'leaders in the field'. We derive an objective, reproducible and attainable Achievable Benchmark of Care (ABC) by measuring and analysing performance on process-of-care indicators. Three characteristics of the ABC that we deem essential are: (1) benchmarks represent a measurable level of excellence; (2) benchmarks are demonstrably attainable; (3) benchmarks are derived from data in an objective, reproducible and predetermined fashion. From these characteristics it follows that (4) providers with high performance are selected to define a level of excellence in a predetermined fashion, but (5) providers with high performance on small numbers of cases do not influence unduly benchmark levels. We use the 'pared mean' to operationalize the ABC. Roughly, the pared mean summarizes the performance of top-ranked providers whereby at least 10% of the patient pool across all providers is included. Bayesian estimators for adjustment of performance of providers with small sample sizes are used to rank providers. Randomized controlled trials to assess the independent effect of the ABC in quality improvement projects are under way. We have developed a methodology objectively and reproducibly to derive a level of excellent, attainable performance, based on measured performance by a group of providers. The ABC can be applied to groups of providers in communities, to institutions and departments within them, or to individual practitioners.

Effectiveness of Quality Improvement in Hospitalization for Bronchiolitis: A Systematic Review

“…19 ABCs were calculated as follows: (1) an adjusted performance fraction was calculated for each study by using published methods 19 ; (2) studies were ranked based on the adjusted performance fraction; (3) a second sample of studies was created by moving down this ranked list until a sample approximating 10% of the total sample size was reached; and (4) the achievable benchmark was calculated as the average performance of the second sample. 19 …”

“…Achievable benchmarks of care (ABC) is a benchmarking method that overcomes this problem by deriving benchmarks from a Bayesian adjusted top 10%. 19 Despite the frequency of bronchiolitis and the well-documented problem of unnecessary care, there have been no previous attempts to review QI interventions for the disease. For this systematic review, our specific research question was: in children hospitalized for acute viral bronchiolitis, how well do QI strategies reduce unnecessary care?…”

Results from Colombia’s 2014 Report Card on Physical Activity for Children and Youth

“…The benchmark for the Colombian grading scale used the model of Saskatchewan (2009), 24 Ontario (2011), 25 and Scotland (2013) 12 because they allow comparisons against an ideal scenario while identifying trends for moderate improvements. 26,27 The third step included meetings with the RWG that considered different perspectives on how the country is doing in the promotion of PA. The research group of Epiandes held 5 meetings (one per institution) with members of the RWG.…”

Benchmarking Australia's mental health services: is it possible and useful?

“…9 Representatives from each of the participating sites came together for a half-day workshop to identify a set of preliminary indicators. A total of 25 indicators covering four key domains were identified.…”