Clinical performance comparators in audit and feedback: a review of theory and evidence

Abstract

Background Audit and feedback (A&F) is a common quality improvement strategy with highly variable effects on patient care. It is unclear how A&F effectiveness can be maximised. Since the core mechanism of action of A&F depends on drawing attention to a discrepancy between actual and desired performance, we aimed to understand current and best practices in the choice of performance comparator. Methods We described current choices for performance comparators by conducting a secondary review of randomised trials of A&F interventions and identifying the associated mechanisms that might have implications for effective A&F by reviewing theories and empirical studies from a recent qualitative evidence synthesis. Results We found across 146 trials that feedback recipients’ performance was most frequently compared against the performance of others (benchmarks; 60.3%). Other comparators included recipients’ own performance over time (trends; 9.6%) and target standards (explicit targets; 11.0%), and 13% of trials used a combination of these options. In studies featuring benchmarks, 42% compared against mean performance. Eight (5.5%) trials provided a rationale for using a specific comparator. We distilled mechanisms of each comparator from 12 behavioural theories, 5 randomised trials, and 42 qualitative A&F studies. Conclusion Clinical performance comparators in published literature were poorly informed by theory and did not explicitly account for mechanisms reported in qualitative studies. Based on our review, we argue that there is considerable opportunity to improve the design of performance comparators by (1) providing tailored comparisons rather than benchmarking everyone against the mean, (2) limiting the amount of comparators being displayed while providing more comparative information upon request to balance the feedback’s credibility and actionability, (3) providing performance trends but not trends alone, and (4) encouraging feedback recipients to set personal, explicit targets guided by relevant information. Electronic supplementary material The online version of this article (10.1186/s13012-019-0887-1) contains supplementary material, which is available to authorized users.

Using Audit and Feedback and the Benchmark to Enhance Infection Prevention and Control of Multidrug Resistant Organisms: a Quasi-experiment Study

“…We believe that the high compliance and the key factors of the persistence of multimodal intervention to control MDROs in LMICs were associated closely to the following issues: (1) practicable A&F interventions on improving quality needs an organizational structure and strong leadership in the institute [19,22]; (2) it is more practicable to transform main components into standardized implementation indicators that can be observed objectively in auditing 35 ; (3) quantitative analysis can help to count or measure what is happening now and guide clinical practices [26]; (4) specifying the working scope of doctors and nurses to construct separate implementation indicators, feedback and education can greatly enhance the intervention implementation; (5) benchmark comparison clearly from feedback report could inspire clinical staff to improve the implementation on those indicators below benchmark e ciently according to the control theory [22,24].…”

“…Besides, benchmarking techniques could help participants in A&F avoid setting unnecessarily low or unrealistically high target levels of performance [22]. Our quasi-experiment study aimed to assess the impact of multimodal intervention which treated A&F and benchmark as key components to reduce the occurrence of MDROs and the implementation effect in three Chinese public hospitals.…”

eRegCom – Quality Improvement Dashboard for healthcare providers and Targeted Client Communication to pregnant women using data from an electronic health registry to improve attendance and quality of antenatal care: study protocol for a multi-arm cluster randomized trial

“…We found that healthcare providers received irregular supervision visits with limited focus on performance improvement at the clinical level. Findings and recommendations regarding benchmarking [32], the use of SMART (Speci c, Measurable, Achievable, Relevant and Timely) criteria, concepts from social nudging and Enhanced active choice [33], and the Model of Actionable Feedback [13], informed the design of the QID. eRegistry users', nursing and medical directors', and PNIPH and MoH staff's reviews informed the revisions.…”

A short cultural history of the UK Renal Registry 1995–2020

“…The optimism that feedback of audit data would promote corrective action has been the subject of academic examination and is far from validated [23,24]. In practice, the early RA default assumption, that closure of audit cycles would be attempted at unit level, left some discomfort in the UKRR [25].…”