Cranky comments: detecting clinical decision support malfunctions through free-text override reasons

Aaron, Skye; McEvoy, Dustin; Ray, Soumi; Hickman, Thu-Trang T.; Wright, Adam

doi:10.1093/jamia/ocy139

Cited by 30 publications

(31 citation statements)

References 20 publications

(15 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Aaron et al described using natural language processing (NLP) to identify "cranky comments" provided in the free- text alert override comments. 41 Although manual review of these override comments was part of our process, this required significant resources without NLP. We found that using the embedded feedback survey provided a more focused venue for users to vent their frustration or to notify us when something had gone wrong.…”

Section: Discussionmentioning

confidence: 99%

Reducing Interruptive Alert Burden Using Quality Improvement Methodology

et al. 2020

View full text Add to dashboard Cite

Background Increased adoption of electronic health records (EHR) with integrated clinical decision support (CDS) systems has reduced some sources of error but has led to unintended consequences including alert fatigue. The “pop-up” or interruptive alert is often employed as it requires providers to acknowledge receipt of an alert by taking an action despite the potential negative effects of workflow interruption. We noted a persistent upward trend of interruptive alerts at our institution and increasing requests for new interruptive alerts. Objectives Using Institute for Healthcare Improvement (IHI) quality improvement (QI) methodology, the primary objective was to reduce the total volume of interruptive alerts received by providers. Methods We created an interactive dashboard for baseline alert data and to monitor frequency and outcomes of alerts as well as to prioritize interventions. A key driver diagram was developed with a specific aim to decrease the number of interruptive alerts from a baseline of 7,250 to 4,700 per week (35%) over 6 months. Interventions focused on the following key drivers: appropriate alert display within workflow, clear alert content, alert governance and standardization, user feedback regarding overrides, and respect for user knowledge. Results A total of 25 unique alerts accounted for 90% of the total interruptive alert volume. By focusing on these 25 alerts, we reduced interruptive alerts from 7,250 to 4,400 per week. Conclusion Systematic and structured improvements to interruptive alerts can lead to overall reduced interruptive alert burden. Using QI methods to prioritize our interventions allowed us to maximize our impact. Further evaluation should be done on the effects of reduced interruptive alerts on patient care outcomes, usability heuristics on cognitive burden, and direct feedback mechanisms on alert utility.

show abstract

Section: Discussionmentioning

confidence: 99%

Reducing Interruptive Alert Burden Using Quality Improvement Methodology

et al. 2020

View full text Add to dashboard Cite

show abstract

“…[55] These may also lead to yet another cost-driver and possibly cause unpredictable economic damage. [56,57] Transferability for other countries All included studies where based on cost data and trials from the United States or Canada. Consequently, current research progress on the economic potentials of EHR based CDS systems on rising healthcare expenditure in Europe and worldwide cannot be derived.…”

Section: Discussionmentioning

confidence: 99%

Economic impact of clinical decision support interventions based on electronic health records

Lewkowicz

Wohlbrandt

Boettinger

2020

Preprint

View full text Add to dashboard Cite

Background Unnecessary healthcare utilization, non-adherence to current clinical guidelines, or insufficient personalized care are perpetual challenges and remain potential major cost-drivers for healthcare systems around the world. Implementing decision support systems into clinical care is promised to improve quality of care and thereby yield substantial effects on reducing healthcare expenditure. In this article, we evaluate the economic impact of clinical decision support (CDS) interventions based on electronic health records (EHR).Methods We searched for studies published after 2014 using MEDLINE, CENTRAL, WEB OF SCIENCE, EBSCO, and TUFTS CEA registry databases that encompass an economic evaluation or consider cost outcome measures of EHR based CDS interventions. Thereupon, we identified best practice application areas and categorized the investigated interventions according to an existing taxonomy of front-end CDS tools. Results and discussion Twenty-seven studies are investigated in this review. Of those, twenty-two studies indicate a reduction of healthcare expenditure after implementing an EHR based CDS system, especially towards prevalent application areas, such as unnecessary laboratory testing, duplicate order entry, efficient transfusion practice, or reduction of antibiotic prescriptions. On the contrary, order facilitators and undiscovered malfunctions revealed to be threats and could lead to new cost drivers in healthcare. While high upfront and maintenance costs of CDS systems are a worldwide implementation barrier, most studies do not consider implementation cost. Finally, four included economic evaluation studies report mixed monetary outcome results and thus highlight the importance of further high-quality economic evaluations for these CDS systems. Conclusion Current research studies lack consideration of comparative cost-outcome metrics as well as detailed cost components in their analyses. Nonetheless, the positive economic impact of EHR based CDS interventions is highly promising, especially with regard to reducing waste in healthcare.

show abstract

“…Organizations should appreciate the value of these messages and seek to learn from them. 44 While we do not know of specific override rate goals, each organization should determine what alert override rates should prompt further alert revision, changes in alert actions, or further clinical education.…”

Section: Governance Structure and Foundational Processesmentioning

confidence: 99%

Reducing Alert Burden in Electronic Health Records: State of the Art Recommendations from Four Health Systems

et al. 2020

View full text Add to dashboard Cite

Background Electronic health record (EHR) alert fatigue, while widely recognized as a concern nationally, lacks a corresponding comprehensive mitigation plan. Objectives The goal of this manuscript is to provide practical guidance to clinical informaticists and other health care leaders who are considering creating a program to manage EHR alerts. Methods This manuscript synthesizes several approaches and recommendations for better alert management derived from four U.S. health care institutions that presented their experiences and recommendations at the American Medical Informatics Association 2019 Clinical Informatics Conference in Atlanta, Georgia, United States. The assembled health care institution leaders represent academic, pediatric, community, and specialized care domains. We describe governance and management, structural concepts and components, and human–computer interactions with alerts, and make recommendations regarding these domains based on our experience supplemented with literature review. This paper focuses on alerts that impact bedside clinicians. Results The manuscript addresses the range of considerations relevant to alert management including a summary of the background literature about alerts, alert governance, alert metrics, starting an alert management program, approaches to evaluating alerts prior to deployment, and optimization of existing alerts. The manuscript includes examples of alert optimization successes at two of the represented institutions. In addition, we review limitations on the ability to evaluate alerts in the current state and identify opportunities for further scholarship. Conclusion Ultimately, alert management programs must strive to meet common goals of improving patient care, while at the same time decreasing the alert burden on clinicians. In so doing, organizations have an opportunity to promote the wellness of patients, clinicians, and EHRs themselves.

show abstract

Cranky comments: detecting clinical decision support malfunctions through free-text override reasons

Cited by 30 publications

References 20 publications

Reducing Interruptive Alert Burden Using Quality Improvement Methodology

Reducing Interruptive Alert Burden Using Quality Improvement Methodology

Economic impact of clinical decision support interventions based on electronic health records

Reducing Alert Burden in Electronic Health Records: State of the Art Recommendations from Four Health Systems

Contact Info

Product

Resources

About