A methodological framework for the evaluation of syndromic surveillance systems: a case study of England

Abstract

BackgroundSyndromic surveillance complements traditional public health surveillance by collecting and analysing health indicators in near real time. The rationale of syndromic surveillance is that it may detect health threats faster than traditional surveillance systems permitting more timely, and hence potentially more effective public health action. The effectiveness of syndromic surveillance largely relies on the methods used to detect aberrations. Very few studies have evaluated the performance of syndromic surveillance systems and consequently little is known about the types of events that such systems can and cannot detect.MethodsWe introduce a framework for the evaluation of syndromic surveillance systems that can be used in any setting based upon the use of simulated scenarios. For a range of scenarios this allows the time and probability of detection to be determined and uncertainty is fully incorporated. In addition, we demonstrate how such a framework can model the benefits of increases in the number of centres reporting syndromic data and also determine the minimum size of outbreaks that can or cannot be detected. Here, we demonstrate its utility using simulations of national influenza outbreaks and localised outbreaks of cryptosporidiosis.ResultsInfluenza outbreaks are consistently detected with larger outbreaks being detected in a more timely manner. Small cryptosporidiosis outbreaks (<1000 symptomatic individuals) are unlikely to be detected. We also demonstrate the advantages of having multiple syndromic data streams (e.g. emergency attendance data, telephone helpline data, general practice consultation data) as different streams are able to detect different outbreak types with different efficacy (e.g. emergency attendance data are useful for the detection of pandemic influenza but not for outbreaks of cryptosporidiosis). We also highlight that for any one disease, the utility of data streams may vary geographically, and that the detection ability of syndromic surveillance varies seasonally (e.g. an influenza outbreak starting in July is detected sooner than one starting later in the year). We argue that our framework constitutes a useful tool for public health emergency preparedness in multiple settings.ConclusionsThe proposed framework allows the exhaustive evaluation of any syndromic surveillance system and constitutes a useful tool for emergency preparedness and response.Electronic supplementary materialThe online version of this article (10.1186/s12889-018-5422-9) contains supplementary material, which is available to authorized users.

Timely detection of Pertussis outbreaks in Iran: The comparison performance of Wavelet-based outbreak detector, Exponential weighted moving average, and Poisson regression-based methods

“…As a result of an increase in the amount of K, the probability of generation alarm on the first day of outbreaks decreased due to an increase in the level of alarm thresholds. The median, minimum and maximum timeliness (according to day) in Haar (k; 0.5) and db10 (k: 0.5) wavelets based method was 2 (1 to 14) and 2 (1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14) respectively. This amount was less than the median timeliness of other algorithms.…”

“…In this study, three types of outbreaks were simulated and injected into real data. The three types of outbreaks are exponential (2,4,8,16,8,4,2), Linear (2,4,6,8,6,4,2) and uniform (6,6,6,6,6,6) increase cases over time.…”

“…The syndrome surveillance includes collection, analysis, interpretation, and dissemination of health-related data defined in public health sectors (5)(6)(7). The objective of the syndrome surveillance system is to provide an early warning (system) for public health threats in near-real-time (8). The Important feature of the syndrome surveillance system is early warning or detection of health-related aberration or outbreak which leads to the reduction of morbidity and mortality of affected people (9).…”

The comparative performance of wavelet‐based outbreak detector, exponential weighted moving average, and Poisson regression‐based methods in detection of pertussis outbreaks in Iranian infants: A simulation‐based study

“…The syndrome surveillance includes collection, analysis, interpretation, and dissemination of health‐related data defined in public health sectors 8‐10 . The syndrome surveillance system's objective is to provide an early warning (system) for public health threats in near‐real‐time 11 . The Important feature of this surveillance system is early warning or detection of health‐related aberration or outbreak, which leads to the reduction of morbidity and mortality of affected people 12 …”

Communicating the detection capabilities of syndromic surveillance systems

“…Increasingly public health decision-makers are using syndromic surveillance for real-time reassurance and situational awareness in addition to early warning [1]. Decision-makers using intelligence, including syndromic data, need to understand what the systems are capable of detecting, what they cannot detect and specifically how much reassurance should be inferred when syndromic systems report ‘nothing detected’.…”