Development of an Emergency Department Trigger Tool Using a Systematic Search and Modified Delphi Process

Griffey, Richard T.; Schneider, Ryan; Adler, Lee; Capp, Roberta; Carpenter, Christopher R.; Farmer, Brenna M.; Groner, Kathyrn Y.; Hodkins, Sheridan; McCammon, Craig; Powell, Jonathan T.; Sather, Jonathan E.; Schuur, Jeremiah D.; Shapiro, Marc J.; Sharp, Brian; Venkatesh, Arjun K.; Vrablik, Marie C.; Wiler, Jennifer L.

doi:10.1097/pts.0000000000000243

Cited by 20 publications

(23 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In 2013, we led a multicenter, multidisciplinary team to develop the ED Trigger Tool (EDTT) using a robust modified Delphi process, beginning with 104 candidate triggers and ultimately retaining 46 consensus-derived triggers. 28 We conducted a 1-year multicenter pilot study of this tool at four sites (monthly random sampling of 50 records) with an AE yield of 6.7%. 29 To further improve the efficiency of the EDTT and establish a basis for identifying individual triggers that did not rely on expert consensus alone, we returned to the broad set of candidate triggers and performed large scale testing of the 92 triggers that were mappable to our electronic medical record (EMR).…”

mentioning

confidence: 99%

“…We thus set out to create a dedicated trigger tool for the ED. In 2013, we led a multicenter, multidisciplinary team to develop the ED Trigger Tool (EDTT) using a robust modified Delphi process, beginning with 104 candidate triggers and ultimately retaining 46 consensus‐derived triggers 28 . We conducted a 1‐year multicenter pilot study of this tool at four sites (monthly random sampling of 50 records) with an AE yield of 6.7% 29 .…”

mentioning

confidence: 99%

See 1 more Smart Citation

The Emergency Department Trigger Tool: Validation and Testing to Optimize Yield

Griffey

Schneider

Todorov

2020

Academic Emergency Medicine

Self Cite

View full text Add to dashboard Cite

Objective: Recognized as a premier approach for adverse event (AE) detection, trigger tools have been developed for multiple clinical settings outside the emergency department (ED). We recently derived and tested an ED trigger tool (EDTT) with enhanced features for high-yield detection of harm, consisting of 30 triggers associated with AEs. In this study, we validate the EDTT in an independent sample and compare record selection approaches to optimize yield for quality improvement. Methods: This is a retrospective observational study using data from 13 months of visits to an urban, academic ED by patients aged ≥ 18 years (92,859 records). We conducted standard two-tiered trigger tool reviews on an independent validation sample of 3,724 records with at least one of the 30 triggers found associated with AEs in our previous derivation sample (N = 1,786). We also tested three new candidate triggers and reviewed 72 records with no triggers for comparison purposes. We compare derivation and validation samples on: 1) triggers showing persistent associations with AEs, 2) AE yield (AEs detected/records reviewed), and 3) representativeness of AE types detected. We use bivariate associations of triggers with AEs as the basis for trigger selection. We then use multivariable modeling in the combined derivation and validation samples to determine AE risk scores using trigger weights. This allows us to predict occurrence of AEs and derive population prevalence estimates. Finally, we compare yield for detection of AEs under three record selection strategies (random selection, trigger counts, weighted trigger counts). Results: Twenty-four of the 30 triggers were confirmed to be associated with AEs on bivariate testing. Three previously marginal triggers and two of three new candidate triggers were also found to be associated with AEs. The presence of any of these 29 triggers was associated with an AE rate of 10% in our selected sample (compared to 1.1% for none, p < 0.001). The risk of an AE increased with number of triggers. Combining data from both phases, we identified 461 AEs in 429 unique visits in 5,582 records reviewed. Our multivariable model (which emphasized parsimony) retained 12 triggers with a ROC AUC of 82% in both samples. Selecting records for review based on number of triggers improves yield to 14% for 4+ triggers (top 10% of visits) and to 28% for 8+ (top 1%). A weighted trigger count has corresponding yields of 18 and 38%. The method for selecting records for review did not appear to affect event-type representativeness, with similar distributions of event types and severities detected. Conclusions: In this single-site study of the EDTT we observed high levels of validity in trigger selection, yield, and representativeness of AEs, with yields that are superior to estimates for traditional approaches to AE detection. Record selection using weighted triggers outperforms a trigger count threshold approach and far outperforms random sampling from records with at least one trigger. The EDTT is a promising efficient ...

show abstract

mentioning

confidence: 99%

mentioning

confidence: 99%

The Emergency Department Trigger Tool: Validation and Testing to Optimize Yield

Griffey

Schneider

Todorov

2020

Academic Emergency Medicine

Self Cite

View full text Add to dashboard Cite

show abstract

“…It has been applied in modified ways, 38 39 giving rise to debates about the applicability of certain principles, for example, selection and definition of experts, anonymity and the number of ‘rounds’. Like other studies using the Delphi process, 40–42 we found it suitable and valuable in the development of the TT. Although participant anonymity is often considered a key factor in a Delphi process, 43 the literature is divided on this.…”

Section: Discussionmentioning

confidence: 58%

Development of a trigger tool to identify adverse events and no-harm incidents that affect patients admitted to home healthcare

et al. 2017

View full text Add to dashboard Cite

BackgroundAdverse events (AEs) and no-harm incidents are common and of great concern in healthcare. A common method for identification of AEs is retrospective record review (RRR) using predefined triggers. This method has been used frequently in inpatient care, but AEs in home healthcare have not been explored to the same extent. The aim of this study was to develop a trigger tool (TT) for the identification of both AEs and no-harm incidents affecting adult patients admitted to home healthcare in Sweden, and to describe the methodology used for this development.MethodsThe TT was developed and validated in a stepwise manner, in collaboration with experts with different skills, using (1) literature review and interviews, (2) a five-round modified Delphi process, and (3) two-stage RRRs. Ten trained teams from different sites in Sweden reviewed 600 randomly selected records.ResultsIn all, triggers were found 4031 times in 518 (86.3%) records, with a mean of 6.7 (median 4, range 1–54) triggers per record with triggers. The positive predictive values (PPVs) for AEs and no-harm incidents were 25.4% and 16.3%, respectively, resulting in a PPV of 41.7% (range 0.0%–96.1% per trigger) for the total TT when using 38 triggers. The most common triggers were unplanned contact with physician and/or registered nurse, moderate/severe pain, moderate/severe worry, anxiety, suffering, existential pain and/or psychological pain. AEs were identified in 37.7% of the patients and no-harm incidents in 29.5%.ConclusionThis study shows that adapted triggers with definitions and decision support, developed to identify AEs and no-harm incidents that affect patients admitted to home healthcare, may be a valid method for safety and quality improvement work in home healthcare.

show abstract

“…The Delphi method is widely accepted as a systematic approach to reach consensus in emergency medicine. [46][47][48][49] It is often used to select indicators, particularly when parameter selection is complex and several experts' opinions are needed to…”

Section: Comparability With Other Findingsmentioning

confidence: 99%

Consensus on predictors of clinical deterioration in emergency departments: A Delphi process study

Tygesen

Kirkegaard

Raaber

et al. 2020

Acta Anaesthesiol Scand

View full text Add to dashboard Cite

Clinical deterioration in hospitalised patients is often preceded by deterioration in physiological parameters. 1 In 1997, Morgan et al 2 proposed the first early warning score (EWS) system to alert clinicians to deteriorating patients using an aggregated, weighted score of vital signs. Early warning scores generally have a track (abnormal vital signs) and a trigger (predetermined calling or response criteria). 3 The predetermined calling or responses of the EWS often consist of increased observation, alerts to senior

show abstract

Development of an Emergency Department Trigger Tool Using a Systematic Search and Modified Delphi Process

Abstract: Our modified Delphi process resulted in the identification of 46 final triggers for the detection of adverse events among ED patients. These triggers should be pilot field tested to quantify their individual and collective performance in detecting all-cause harm to ED patients.

Cited by 20 publications

References 29 publications

The Emergency Department Trigger Tool: Validation and Testing to Optimize Yield

The Emergency Department Trigger Tool: Validation and Testing to Optimize Yield

Development of a trigger tool to identify adverse events and no-harm incidents that affect patients admitted to home healthcare

Consensus on predictors of clinical deterioration in emergency departments: A Delphi process study

Contact Info

Product

Resources

About