Effects of a computerized feedback intervention on safety performance by junior doctors: results from a randomized mixed method study

Abstract

Background: The behaviour of doctors and their responses to warnings can inform the effective design of Clinical Decision Support Systems. We used data from a University hospital electronic prescribing and laboratory reporting system with hierarchical warnings and alerts to explore junior doctors' behaviour. The objective of this trial was to establish whether a Junior Doctor Dashboard providing feedback on prescription warning information and laboratory alerting acceptance rates was effective in changing junior doctors' behaviour. Methods: A mixed methods approach was employed which included a parallel group randomised controlled trial, and individual and focus group interviews. Junior doctors below the specialty trainee level 3 grade were recruited and randomised to two groups. Every doctor (N = 42) in the intervention group was e-mailed a link to a personal dashboard every week for 4 months. Nineteen participated in interviews. The 44 control doctors did not receive any automated feedback. The outcome measures were the difference in responses to prescribing warnings (of two severities) and laboratory alerting (of two severities) between the months before and the months during the intervention, analysed as the difference in performance between the intervention and the control groups.

A Review of Dashboards for Data Analytics in Nursing

“…In the reviewed literature, the most mentioned benefits of dashboards were the ability to evaluate large amounts of data and presenting the results in an easy-to-interpret visual format, 4,[15][16][17][18]23 to provide notifications of metrics that deviate from predefined acceptable levels to minimize adverse events, 4,5,10,13,14,17,24,25 to provide decision support to improve efficiency and quality, 5,[12][13][14][23][24][25][26] and to provide decision support that promotes data-driven decision making for executive-level management. 11,15,21 Other benefits mentioned in the literature include the high degree of customizability of dashboards to specific purposes 18,21 and workflows 17 ; the ability of dashboards to coordinate patient care among different providers 12,16 ; and the use of adequately defined key performance indicators to provide reliable and valid measurements used to evaluate performance over time.…”

“…The identified social factors include decreased acceptance of dashboards secondary to (1) clinician anxiety about electronic surveillance of their performance, 26 (2) information overload caused by excessive monitoring of key performance indicators, 10 and (3) use of key performance indicators that clinicians believe cannot be directly changed (eg, an RN in a surgical ICU might believe that a surgical-site infection was acquired intraoperatively, and nothing could have been done postoperatively to prevent the infection). The identified social factors include decreased acceptance of dashboards secondary to (1) clinician anxiety about electronic surveillance of their performance, 26 (2) information overload caused by excessive monitoring of key performance indicators, 10 and (3) use of key performance indicators that clinicians believe cannot be directly changed (eg, an RN in a surgical ICU might believe that a surgical-site infection was acquired intraoperatively, and nothing could have been done postoperatively to prevent the infection).…”

“…(1) dashboard metrics can have different work environment contexts (ie, work environments can be different and require slightly different metrics for evaluation) 26 ; (2) the organizational culture has to value objective, data-driven decision making to gain the full benefits of dashboards (ie, the organization has to be willing to actually use the information gained by dashboards to make decisions) 1,12 ; and (3) dashboard utilization requires financial and human resources. 21 The technical limitations that can impair the functionality of dashboards are related to database management issues and properties of the data.…”

“…26 Research into the optimal alert design to prevent alert fatigue is needed to ensure warnings issued by dashboards result in prompt corrective actions instead of being ignored. Research is needed to identify the optimal design of the graphical user interface for dashboards (eg, placement on screen, format, and color of in-formation to improve usability).…”

Best practice strategies to safeguard drug prescribing and drug administration: an anthology of expert views and opinions

“…As safety climate was also identified as an independent predictor for patient safety, it is even more important to work on the overall safety climate of an institution in addition to introduction of specific quality improvement strategies [16]. Internal [17] and external benchmarking [18] might help to support a longitudinal improvement. Especially for external benchmarking, self-assessment tools can be used, such as Failure Mode and Effects Analysis (FMEA) used at the University Hospital in Geneva [19].…”