in silico Surveillance: evaluating outbreak detection with simulation models

Abstract

BackgroundDetecting outbreaks is a crucial task for public health officials, yet gaps remain in the systematic evaluation of outbreak detection protocols. The authors’ objectives were to design, implement, and test a flexible methodology for generating detailed synthetic surveillance data that provides realistic geographical and temporal clustering of cases and use to evaluate outbreak detection protocols.MethodsA detailed representation of the Boston area was constructed, based on data about individuals, locations, and activity patterns. Influenza-like illness (ILI) transmission was simulated, producing 100 years of in silico ILI data. Six different surveillance systems were designed and developed using gathered cases from the simulated disease data. Performance was measured by inserting test outbreaks into the surveillance streams and analyzing the likelihood and timeliness of detection.ResultsDetection of outbreaks varied from 21% to 95%. Increased coverage did not linearly improve detection probability for all surveillance systems. Relaxing the decision threshold for signaling outbreaks greatly increased false-positives, improved outbreak detection slightly, and led to earlier outbreak detection.ConclusionsGeographical distribution can be more important than coverage level. Detailed simulations of infectious disease transmission can be configured to represent nearly any conceivable scenario. They are a powerful tool for evaluating the performance of surveillance systems and methods used for outbreak detection.

Public Health Surveillance Systems: Recent Advances in Their Use and Evaluation

“…For example, the last decade has seen a considerable amount of research on the statistical detection of aberrations in surveillance data (15,50), but most of these studies have focused on the statistical properties of detection algorithms. With few exceptions, researchers have tended not to examine the effect of detection algorithms on outcomes, such as the time to initiating a response (17) or the number of infections averted (46).…”

Simulation training improves clinical knowledge of major haemorrhage management in foundation year doctors

“…A literature search for studies on the use of simulation as a means for testing protocols in medicine yielded some results (Wright et al ., ). The majority were based on development of computer programmes to enact virtual simulation (Lewis et al ., ). However the use of simulation for protocol development has been present in the technology industry for many years.…”

Fast and near-optimal monitoring for healthcare acquired infection outbreaks

“…For example, an infected agent can infect a susceptible agent if they are in the same location based on the transition probabilities of the disease/health state. EpiSimdemics and its extensions have been used in wide range of studies from informing pandemic influenza policy [29], to modeling the immune system in the gut [30], to assisting in the design of surveillance systems [4]. This platform has been shown to scale to what was a record-breaking level [31] for an epidemic model.…”

“…For example, including the culture site or antibiotic resistance was shown to boost detectability. Past simulation based approaches [4] tackle optimal surveillance system design, by choosing clinics as sensors, to increase sensitivity and time to detection for outbreaks in a population. In contrast, our approach selects most vulnerable people and locations to infections as sensors to detect outbreaks in a hospital setting.…”