Diagnostic Performance of Electronic Syndromic Surveillance Systems in Acute Care

Abstract

SummaryContext: Healthcare Electronic Syndromic Surveillance (ESS) is the systematic collection, analysis and interpretation of ongoing clinical data with subsequent dissemination of results, which aid clinical decision-making. Objective:To evaluate, classify and analyze the diagnostic performance, strengths and limitations of existing acute care ESS systems. Data Sources: All available to us studies in Ovid MEDLINE, Ovid EMBASE, CINAHL and Scopus databases, from as early as January 1972 through the first week of September 2012. Study Selection: Prospective and retrospective trials, examining the diagnostic performance of inpatient ESS and providing objective diagnostic data including sensitivity, specificity, positive and negative predictive values. Data Extraction: Two independent reviewers extracted diagnostic performance data on ESS systems, including clinical area, number of decision points, sensitivity and specificity. Positive and negative likelihood ratios were calculated for each healthcare ESS system. A likelihood matrix summarizing the various ESS systems performance was created. Results: The described search strategy yielded 1639 articles. Of these, 1497 were excluded on abstract information. After full text review, abstraction and arbitration with a third reviewer, 33 studies met inclusion criteria, reporting 102,611 ESS decision points. The yielded I2 was high (98.8%), precluding meta-analysis. Performance was variable, with sensitivities ranging from 21% -100% and specificities ranging from 5%-100%. Conclusions: There is significant heterogeneity in the diagnostic performance of the available ESS implements in acute care, stemming from the wide spectrum of different clinical entities and ESS systems. Based on the results, we introduce a conceptual framework using a likelihood ratio matrix for evaluation and meaningful application of future, frontline clinical decision support systems.

Differentiating infectious and noninfectious ventilator-associated complications: A new challenge

“…This is not surprising given the clinical heterogeneity of VAEs because electronic syndromic surveillance systems presently work by detecting aberrancies in particular values, rather than assessing pre- and post-test probability for a patient with a given a priori suspicion of disease. 16 …”

Testing modes of computerized sepsis alert notification delivery systems

“…This is one of strengths of this type of delivery method. As the accuracy of electronically generated alerts is imperfect, the feedback loop for any clinical notifications can help not only to ameliorate quality of care, but also to optimize and diminish alert fatigue [33, 34]. However, working stations with EHR-embedded notifications were inappropriate to display time critical notification messages in the ICU settings [20] due to limited duration of login authentication intervals, which delayed acknowledgements.…”

Data elements and validation methods used for electronic surveillance of health care-associated infections: A systematic review

“…The increased number of data sources used in ESS reflects the fact that more electronic clinical data continues to become available 6 . However, the fact that 17% of the studies in this review did not use the recommended data suggests that availability of relevant data is still one gap that must be filled before fully automated surveillance of HAI is possible.…”

“…To address these issues, the infection prevention community has pursued the development of automated electronic surveillance systems (ESS). While ESS using electronically available patient data have been found to be accurate and potentially time saving 3–5 , their performance is not consistent across settings 6 . The performance of ESS often depends on implementation issues related to data sources and data capture 7 .…”