Avoiding alert fatigue in pulmonary embolism decision support: a new method to examine ‘trigger rates’

Abstract

A clinical decision support system (CDSS) is integrated into the electronic health record (EHR) and allows physicians to easily use a clinical decision support (CDS) tool. However, often CDSSs are integrated into the EHR with poor adoption rates. One reason for this is secondary to 'trigger fatigue'. Therefore, we developed a new and innovative usability process named 'sensitivity and specificity trigger analysis' (SSTA) as part of our larger project around a pulmonary embolism decision support tool. SSTA will enable programmers to examine optimal trigger rates prior to the integration of a CDS tool into the EHR, by using a formal method of analysis. We performed a retrospective chart review. The outcome of interest was physician ordering of a CT angiography (CTA). Phrases that signify common symptoms associated with pulmonary embolism were assessed as possible triggers for the CDSS tool. We then analysed each trigger's ability to predict physician ordering of a CTA. We found that the most sensitive way to trigger the Pulmonary Embolism CDS tool while still maintaining a high specificity was by combining 1 or more pertinent symptoms with 1 or more elements of the Wells criteria. This study explored a unique methodology, SSTA, used to limit inaccurate triggering of a CDS tool prior to integration into the EHR. This methodology can be applied to other studies aiming to decrease triggering rates and increase adoption rates of previously validated CDSS tools.

Improving Provider Adoption With Adaptive Clinical Decision Support Surveillance: An Observational Study (Preprint)

“…It is thus crucial to devise computerized tools that are optimally integrated into clinician workflow without causing alert fatigue [3] and that usability may be as important as accuracy and effectiveness in implementing CDS [11]. We speculate that the high adoption rates of 29-42% in our study, as opposed to an average of 10-20% adoption from previous CDS research [9], were a result of the comprehensive user centered development process including workflow assessment of the emergency department, focus groups, usability testing, and collaboration with a multidisciplinary team [23,27,31].…”

“…Thus we implemented a two-phase usability testing approach with emergency physicians prior to the integration of the Wells' CPR into the EHR [31]. Furthermore, the "sensitivity and specificity trigger analysis" (SSTA) was developed to identify optimal trigger locations to avoid underutilization and overtriggering [23]. From our past studies, we have also learned that adoption of CDS tools can be dependent on individual user characteristics such as age, training level, and experiences with health technology [32].…”

“…After thorough usability testing of the tool in the EHR playground environment within our Usability Lab, [23,26,27] the tool was launched within the emergency rooms at two large academic tertiary centers in a staggered roll out.…”

“…A key element in the post-implementation period is the continuous monitoring and sustainability of the tool amongst clinical providers [3,23]. It has been noted that evaluations of postimplementation alerts' appropriateness can be labor-intensive and costly [24].…”

Agile Acceptance Test–Driven Development of Clinical Decision Support Advisories: Feasibility of Using Open Source Software

“…any restrictions on where the CDS alert logic should be evaluated (ie, restricted to only certain practice locations, encounter types, provider types) to help target the most appropriate situations and limit “alert fatigue” [ 18 , 19 ]…”