Meta-Analysis

Abstract

Background Audit and feedback (A&F) has long been used to improve quality of care, albeit with variable results. This meta-analytic study tested whether Feedback Intervention Theory, a framework from industrial/organizational psychology, explains the observed variability in health care A&F research. Method Data Source Studies cited by Jamtvedt's 2006 Cochrane systematic review of A&F, followed by database searches using the Cochrane review's search strategy to identify more recent studies. Inclusion Criteria Cochrane review criteria, plus: presence of a treatment group receiving only A&F; a control group receiving no intervention; a quantitatively measurable outcome; minimum n of 10 per arm; sufficient statistics for effect size calculations. Moderators Presence of discouragement and praise; correct solution, attainment level, velocity, frequency, and normative information; feedback format (verbal, textual, graphic, public, computerized, group vs. individual); goal setting activity. Procedure Meta-analytic procedures using the Hedges-Olkin method. Results Of 519 studies initially identified, 19 met all inclusion criteria. Studies were most often excluded due to the lack of a feedback-only arm. A&F has a modest, though significant positive effect on quality outcomes (d=.40, 95% CI ±.20); providing specific suggestions for improvement, written, and more frequent feedback strengthened this effect, whereas graphical and verbal feedback attenuated this effect. Conclusions A&F effectiveness is improved when feedback is delivered with specific suggestions for improvement, in writing, and frequently. Other feedback characteristics could also potentially improve effectiveness; however, research with stricter experimental controls is needed to identify the specific feedback characteristics that maximize its effectiveness.

Evaluation of a continuous monitoring and feedback initiative to improve quality of anaesthetic care: a mixed-methods quasi-experimental study

“…Prior literature demonstrates the importance of key characteristics such as timeliness, feedback format, level of feedback, frequency and data quality. 52,97,98,107,149 The provision of feedback does not seem to have worked through the stimulation of local climate for quality improvement across the department in the case of our intervention.…”

“…Moonsinghe et al 6 describe that anaesthetists should evaluate outcome measures against their potential to lead to improvements in standards of care and benchmarking. Hysong et al 159 state that features of effective feedback involve the degree of perceived timeliness, data formatting and frequency of feedback delivery. Environment, individual opinion, knowledge and attitudes were also thought to be key.…”

“…149,152,176,204,205 Recent work in implementation science has sought to strengthen the research base for mounting effective audit and feedback interventions through establishing an agenda for future research that highlights key areas such as characteristics of the setting and recipient, cointerventions, mode of delivery and nature of the content of feedback. 205 The model described in this report draws on improvement science methods and philosophy (process control and continuous evaluation) and may, therefore, be more accurately conceptualised as a continuous data feedback intervention.…”

“…52,55,135,148 High intensity and frequency of feedback improves outcomes along with sustained monitoring. 98,107,135,144,148,149 A number of strategies have been identified to support the effectiveness of performance feedback, including providing recipients with information on specific areas for improvement, action planning and educational components. 52,55,98,107,135,144,148,149 In addition to individual factors, the situation (contextual factors) in which feedback occurs influences its acceptance and effectiveness in improving performance.…”

How do I audit and receive feedback on my clinical practice?

“…A number of factors increase the success of A&F. 10,11 Basing the feedback on chart data is important; equally important is considering how the feedback is provided and the consequences of that feedback. The emotional involvement is higher and the ability to recall details is greater when the A&F is performed soon after the event(s).…”