Diagnostic accuracy and effectiveness of automated electronic sepsis alert systems: A systematic review

Abstract

Background Although timely treatment of sepsis improves outcomes, delays in administering evidence-based therapies are common. Purpose To determine whether automated real-time electronic sepsis alerts can: 1) accurately identify sepsis, and 2) improve process measures and outcomes. Data Sources We systematically searched MEDLINE, Embase, The Cochrane Library, and CINAHL from database inception through June 27, 2014. Study Selection Included studies that empirically evaluated one or both of the prespecified objectives. Data Extraction Two independent reviewers extracted data and assessed the risk of bias. Diagnostic accuracy of sepsis identification was measured by sensitivity, specificity, positive (PPV) and negative predictive values (NPV) and likelihood ratios (LR). Effectiveness was assessed by changes in sepsis care process measures and outcomes. Data Synthesis Of 1,293 citations, 8 studies met inclusion criteria, 5 for the identification of sepsis (n=35,423) and 5 for the effectiveness of sepsis alerts (n=6,894). Though definition of sepsis alert thresholds varied, most included systemic inflammatory response syndrome criteria ± evidence of shock. Diagnostic accuracy varied greatly, with PPV ranging from 20.5-53.8%, NPV 76.5-99.7%; LR+ 1.2-145.8; and LR- 0.06-0.86. There was modest evidence for improvement in process measures (i.e., antibiotic escalation), but only among patients in non-critical care settings; there were no corresponding improvements in mortality or length of stay. Minimal data were reported on potential harms due to false positive alerts. Conclusions Automated sepsis alerts derived from electronic health data may improve care processes but tend to have poor positive predictive value and do not improve mortality or length of stay.

AME evidence series 001—The Society for Translational Medicine: clinical practice guidelines for diagnosis and early identification of sepsis in the hospital

“…Use of automated electronic sepsis alert system (AeSAS) to improve sepsis management represents an area of active research (63). Advances in electronic medical system technology and sophisticated machine learning techniques will "upgrade" prediction models making them more accurate and individualized (64)(65)(66).…”

Early management of sepsis with emphasis on early goal directed therapy: AME evidence series 002

“…Tools that reliably detect early community-acquired sepsis are warranted and are the topic of intensive research. Meanwhile, there are studies focusing on early recognition of sepsis in hospitalized patients (110)(111)(112). As mentioned before, this patient population may be identified more easily and thus become more amenable for straightforward EGDT.…”

“…Studies investigating the impact of automated electronic sepsis alert systems on clinical outcome produced inconsistent results (Table 3) (112). Two RCTs failed to show a beneficial effect of such alert systems (114,118).…”

Advancing In-Hospital Clinical Deterioration Prediction Models

“…1,2 Even if early warning systems and rapid response teams improve patients’ outcomes, the incidence of in-hospital clinical deterioration remains high and is associated with low survival rates. 3–5 Given that the prevention of adverse outcomes will depend on early recognition followed by appropriate management, tools to aid these processes are needed.…”

“…Clinical prediction models, especially those incorporated into decision support tools that automatically retrieve data from electronic health records, are becoming increasingly popular and might be able to assist in the early identification of clinical deterioration. 2,6–15 …”