Timely Data for Targeted Quality Improvement Interventions: Use of a Visual Analytics Dashboard for Bronchiolitis

Hester, Gabrielle; Lang, Tom; Madsen, Laura B.; Tambyraja, Rabindra; Zenker, Paul

doi:10.1055/s-0039-1679868

Cited by 20 publications

(40 citation statements)

References 44 publications

(61 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Of these studies, 56% (5/9) of the studies were deployed and evaluated in real-world hospital environments, and 44% (4/9) of the studies were conducted in controlled laboratory settings. The evaluation study duration ranged between 2 months [ 23 ] and 42 months [ 27 ].…”

Section: Resultsmentioning

confidence: 99%

State-of-the-art Dashboards on Clinical Indicator Data to Support Reflection on Practice: Scoping Review

Bucalon¹,

Brown²,

Shaw³

et al. 2022

JMIR Med Inform

View full text Add to dashboard Cite

Background There is an increasing interest in using routinely collected eHealth data to support reflective practice and long-term professional learning. Studies have evaluated the impact of dashboards on clinician decision-making, task completion time, user satisfaction, and adherence to clinical guidelines. Objective This scoping review aims to summarize the literature on dashboards based on patient administrative, medical, and surgical data for clinicians to support reflective practice. Methods A scoping review was conducted using the Arksey and O’Malley framework. A search was conducted in 5 electronic databases (MEDLINE, Embase, Scopus, ACM Digital Library, and Web of Science) to identify studies that met the inclusion criteria. Study selection and characterization were performed by 2 independent reviewers (BB and CP). One reviewer extracted the data that were analyzed descriptively to map the available evidence. Results A total of 18 dashboards from 8 countries were assessed. Purposes for the dashboards were designed for performance improvement (10/18, 56%), to support quality and safety initiatives (6/18, 33%), and management and operations (4/18, 22%). Data visualizations were primarily designed for team use (12/18, 67%) rather than individual clinicians (4/18, 22%). Evaluation methods varied among asking the clinicians directly (11/18, 61%), observing user behavior through clinical indicators and use log data (14/18, 78%), and usability testing (4/18, 22%). The studies reported high scores on standard usability questionnaires, favorable surveys, and interview feedback. Improvements to underlying clinical indicators were observed in 78% (7/9) of the studies, whereas 22% (2/9) of the studies reported no significant changes in performance. Conclusions This scoping review maps the current literature landscape on dashboards based on routinely collected clinical indicator data. Although there were common data visualization techniques and clinical indicators used across studies, there was diversity in the design of the dashboards and their evaluation. There was a lack of detail regarding the design processes documented for reproducibility. We identified a lack of interface features to support clinicians in making sense of and reflecting on their personal performance data.

show abstract

Section: Resultsmentioning

confidence: 99%

State-of-the-art Dashboards on Clinical Indicator Data to Support Reflection on Practice: Scoping Review

Bucalon¹,

Brown²,

Shaw³

et al. 2022

JMIR Med Inform

View full text Add to dashboard Cite

show abstract

“…Several recent studies have developed visual analytics solutions that show timely and relevant clinical data, as well as the patterns found within them, which can be used for targeting patient and/or quality improvement interventions. [16][17][18] These visual analytics solutions often require a significant amount of effort to be put toward data integration and information management, 19 and their effectiveness on improving patient outcomes is unknown in the early stages of deployment and use. 17 More importantly, these visual analytics solutions are subject to potential humancomputer interaction (HCI) and usability issues resulting from the complicated interplay between human cognition and system functionality.…”

Section: Introductionmentioning

confidence: 99%

Usability Testing of an Interactive Dashboard for Surgical Quality Improvement in a Large Congenital Heart Center

Vennemeyer

Brown

et al. 2019

Appl Clin Inform

View full text Add to dashboard Cite

Background Interactive data visualization and dashboards can be an effective way to explore meaningful patterns in large clinical data sets and to inform quality improvement initiatives. However, these interactive dashboards may have usability issues that undermine their effectiveness. These usability issues can be attributed to mismatched mental models between the designers and the users. Unfortunately, very few evaluation studies in visual analytics have specifically examined such mismatches between these two groups. Objectives We aimed to evaluate the usability of an interactive surgical dashboard and to seek opportunities for improvement. We also aimed to provide empirical evidence to demonstrate the mismatched mental models between the designers and the users of the dashboard. Methods An interactive dashboard was developed in a large congenital heart center. This dashboard provides real-time, interactive access to clinical outcomes data for the surgical program. A mixed-method, two-phase study was conducted to collect user feedback. A group of designers (N = 3) and a purposeful sample of users (N = 12) were recruited. The qualitative data were analyzed thematically. The dashboards were compared using the System Usability Scale (SUS) and qualitative data. Results The participating users gave an average SUS score of 82.9 on the new dashboard and 63.5 on the existing dashboard (p = 0.006). The participants achieved high task accuracy when using the new dashboard. The qualitative analysis revealed three opportunities for improvement. The data analysis and triangulation provided empirical evidence to the mismatched mental models. Conclusion We conducted a mixed-method usability study on an interactive surgical dashboard and identified areas of improvements. Our study design can be an effective and efficient way to evaluate visual analytics systems in health care. We encourage researchers and practitioners to conduct user-centered evaluation and implement education plans to mitigate potential usability challenges and increase user satisfaction and adoption.

show abstract

“…For instance, Hester et al have demonstrated that providing integrated visual information for care of patients with bronchiolitis in the ED led to improvement in certain workflow metrics such as reduced length of stay. 33 A design issue highlighted by the analysis was that the timeline view did not appropriately signal changes in vital signs when values were outside of the plot boundaries (e.g., were too high or too low to fit on the screen). Off screen values were shown at the boundary (top or bottom) and were animated (flashed), but were connected to previous values with dashed lines and therefore did not indicate the degree of change using the salient feature of slope.…”

Section: Discussionmentioning

confidence: 99%

Design and Evaluation of an Integrated, Patient-Focused Electronic Health Record Display for Emergency Medicine

et al. 2019

View full text Add to dashboard Cite

Background Hospital emergency departments (EDs) are dynamic environments, involving coordination and shared decision making by staff who care for multiple patients simultaneously. While computerized information systems have been widely adopted in such clinical environments, serious issues have been raised related to their usability and effectiveness. In particular, there is a need to support clinicians to communicate and maintain awareness of a patient's health status, and progress through the ED plan of care. Objective This study used work-centered usability methods to evaluate an integrated patient-focused status display designed to support ED clinicians' communication and situation awareness regarding a patient's health status and progress through their ED plan of care. The display design was informed by previous studies we conducted examining the information and cognitive support requirements of ED providers and nurses. Methods ED nurse and provider participants were presented various scenarios requiring patient-prioritization and care-planning tasks to be performed using the prototype display. Participants rated the display in terms of its cognitive support, usability, and usefulness. Participants' performance on the various tasks, and their feedback on the display design and utility, was analyzed. Results Participants provided ratings for usability and usefulness for the display sections using a work-centered usability questionnaire—mean scores for nurses and providers were 7.56 and 6.6 (1 being lowest and 9 being highest), respectively. General usability scores, based on the System Usability Scale tool, were rated as acceptable or marginally acceptable. Similarly, participants also rated the display highly in terms of support for specific cognitive objectives. Conclusion A novel patient-focused status display for emergency medicine was evaluated via a simulation-based study in terms of work-centered usability and usefulness. Participants' subjective ratings of usability, usefulness, and support for cognitive objectives were encouraging. These findings, including participants' qualitative feedback, provided insights for improving the design of the display.

show abstract

Timely Data for Targeted Quality Improvement Interventions: Use of a Visual Analytics Dashboard for Bronchiolitis

Cited by 20 publications

References 44 publications

State-of-the-art Dashboards on Clinical Indicator Data to Support Reflection on Practice: Scoping Review

State-of-the-art Dashboards on Clinical Indicator Data to Support Reflection on Practice: Scoping Review

Usability Testing of an Interactive Dashboard for Surgical Quality Improvement in a Large Congenital Heart Center

Design and Evaluation of an Integrated, Patient-Focused Electronic Health Record Display for Emergency Medicine

Contact Info

Product

Resources

About