Evaluating and implementing temporal, spatial, and spatio-temporal methods for outbreak detection in a local syndromic surveillance system

Abstract

The New York City Department of Health and Mental Hygiene has operated an emergency department syndromic surveillance system since 2001, using temporal and spatial scan statistics run on a daily basis for cluster detection. Since the system was originally implemented, a number of new methods have been proposed for use in cluster detection. We evaluated six temporal and four spatial/spatio-temporal detection methods using syndromic surveillance data spiked with simulated injections. The algorithms were compared on several metrics, including sensitivity, specificity, positive predictive value, coherence, and timeliness. We also evaluated each method’s implementation, programming time, run time, and the ease of use. Among the temporal methods, at a set specificity of 95%, a Holt-Winters exponential smoother performed the best, detecting 19% of the simulated injects across all shapes and sizes, followed by an autoregressive moving average model (16%), a generalized linear model (15%), a modified version of the Early Aberration Reporting System’s C2 algorithm (13%), a temporal scan statistic (11%), and a cumulative sum control chart (<2%). Of the spatial/spatio-temporal methods we tested, a spatial scan statistic detected 3% of all injects, a Bayes regression found 2%, and a generalized linear mixed model and a space-time permutation scan statistic detected none at a specificity of 95%. Positive predictive value was low (<7%) for all methods. Overall, the detection methods we tested did not perform well in identifying the temporal and spatial clusters of cases in the inject dataset. The spatial scan statistic, our current method for spatial cluster detection, performed slightly better than the other tested methods across different inject magnitudes and types. Furthermore, we found the scan statistics, as applied in the SaTScan software package, to be the easiest to program and implement for daily data analysis.

Timely detection of Pertussis outbreaks in Iran: The comparison performance of Wavelet-based outbreak detector, Exponential weighted moving average, and Poisson regression-based methods

“…According to our results, the performance of understudy methods in different approaches to outbreaks simulation was absolutely different. It's worth mentioning that these results were consistent with other studies conducted in this field (42,43). The performance of outbreak detection methods was affected by many factors including the type of outbreaks, duration, and magnitude.…”

Species‐specific spatiotemporal patterns of leopard, lion and tiger attacks on humans

“…This method only requires the location and date of each attack and makes no assumptions about the fine-scale distribution of atrisk humans across the survey area (Kulldorff et al, 2005a), whereas methods such as log-Gaussian Cox processes (Diggle, Moraga, Rowlingson, & Taylor, 2013) assume the at-risk population distribution is either known or is uniform across the landscape (Kulldorff et al, 2005a) which is rarely the case. Not only do space-time scan methods require fewer assumptions, but they also generally outperform spatiotemporal methods and are easier to perform (Mathes et al, 2017), and the SaTScan software is freely available with a graphic user interface requiring minimal epidemiological training (https://www.SaTScan.org/). Spatiotemporal clusters were identified from a significant excess of cases occurring within a geographical area over a continuous period of time.…”

What can urban mobility data reveal about the spatial distribution of infection in a single city?

“…This has implications for using surveillance data to characterise and predict disease dynamics [57]. Using spatially-indexed surveillance data to characterise urban disease activity and to detect spatial disease clusters and other patterns is challenging [58]. For this reason, spatially-explicit models of infection at urban scales are of real value, despite the limitations of the available mobility data sets informing the models.…”