Assessing surveillance using sensitivity, specificity and timeliness

Kleinman, Ken; Abrams, Alan

doi:10.1177/0962280206071641

Cited by 48 publications

(53 citation statements)

References 27 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kleinman and Abrams (2006) extended the use of the ROC curve to incorporate measures of timeliness such as the median time to signal. They recognized the fact that some measures of timeliness, such as the average time-to-signal, are not directly applicable for cases in which an outbreak may not be detected.…”

Section: Timeliness Under the Historical Scenariomentioning

confidence: 99%

Performance Metrics for Surveillance Schemes

Fraker

Woodall

Mousavi

2008

Quality Engineering

View full text Add to dashboard Cite

We review various statistical performance metrics that have been used with prospective surveillance schemes. We consider scenarios under which the metrics are most useful and discuss some of their advantages and disadvantages. A contrast is made between the approaches and metrics used in industrial process monitoring and those used in public health surveillance. The in-control average time between signal events (ATBSE) and the in-control average signal event length (ASEL) are introduced as performance metrics that are useful when a monitoring procedure is not reset to its initial state after a signal.We give particular attention to the recurrence interval, defined as the fixed length of time (measured in number of time periods) for which the expected number of false alarms is one. The recurrence interval is used in public health surveillance whereas in-control time-to-signal measures are used in industrial statistical process control. We compare the recurrence interval and measures based on the time-to-signal properties for the temporal monitoring case using exponentially weighted moving average (EWMA) charts, cumulative sum (CUSUM) charts, and Markov dependent signaling processes. We recommend that measures based on the time-to-signal properties be used when possible to evaluate the performance of surveillance schemes for ongoing monitoring.

show abstract

Section: Timeliness Under the Historical Scenariomentioning

confidence: 99%

Performance Metrics for Surveillance Schemes

Fraker

Woodall

Mousavi

2008

Quality Engineering

View full text Add to dashboard Cite

show abstract

“…Analyses such as those determining epidemic thresholds from public health data (e.g. [45]–[47]) will be useful starting points for integrating thresholds into detecting epidemics of endemic zoonotic diseases.…”

Section: Discussionmentioning

confidence: 99%

Optimal Sampling Strategies for Detecting Zoonotic Disease Epidemics

et al. 2014

View full text Add to dashboard Cite

The early detection of disease epidemics reduces the chance of successful introductions into new locales, minimizes the number of infections, and reduces the financial impact. We develop a framework to determine the optimal sampling strategy for disease detection in zoonotic host-vector epidemiological systems when a disease goes from below detectable levels to an epidemic. We find that if the time of disease introduction is known then the optimal sampling strategy can switch abruptly between sampling only from the vector population to sampling only from the host population. We also construct time-independent optimal sampling strategies when conducting periodic sampling that can involve sampling both the host and the vector populations simultaneously. Both time-dependent and -independent solutions can be useful for sampling design, depending on whether the time of introduction of the disease is known or not. We illustrate the approach with West Nile virus, a globally-spreading zoonotic arbovirus. Though our analytical results are based on a linearization of the dynamical systems, the sampling rules appear robust over a wide range of parameter space when compared to nonlinear simulation models. Our results suggest some simple rules that can be used by practitioners when developing surveillance programs. These rules require knowledge of transition rates between epidemiological compartments, which population was initially infected, and of the cost per sample for serological tests.

show abstract

“…We combined these metrics and estimated the area under the weighted receiver operating characteristic curve (AUWROC) as an overall measure of performance (10). The Table shows the highest AU-WROC, for each method, from a predefi ned selection of parameter combinations and the sensitivity and timeliness at a fi xed specifi city of 95%.…”

Section: The Studymentioning

confidence: 99%

Optimizing Use of Multistream Influenza Sentinel Surveillance Data

Lau

Cowling

et al. 2008

Emerg. Infect. Dis.

View full text Add to dashboard Cite

We applied time-series methods to multivariate sentinel surveillance data recorded in Hong Kong during 1998-2007. Our study demonstrates that simultaneous monitoring of multiple streams of infl uenza surveillance data can improve the accuracy and timeliness of alerts compared with monitoring of aggregate data or of any single stream alone.T he use of separate data streams based on sentinel surveillance has long been an accepted approach to monitor community incidence and to enable timely detection of infectious disease outbreaks (1,2). Recently, more attention has been given to the combined analysis of multivariate sentinel data (3)(4)(5).In this study we explored the possibility of improving the ability to more quickly detect peak periods of infl uenza activity in Hong Kong through simultaneous monitoring of multiple streams of sentinel surveillance data. Our fi ndings have general implications in the choice of surveillance algorithms where multistream data are available. The StudyThe local Department of Health publishes weekly reports (6) from a network of 50 private-sector sentinel general practitioners (GP) and 62 public-sector sentinel general outpatient clinics (GOPC) on the proportion of patients seeking treatment for infl uenza-like illness (ILI), defi ned as fever plus cough or sore throat (7). In this study, we used the GP and GOPC sentinel surveillance data in 9 annual infl uenza seasons from 1998-1999 to 2006-2007, stratifi ed by 4 geographic regions in Hong Kong-Hong Kong Island, Kowloon, New Territories East, and New Territories West-resulting in 8 separate data streams (Figure).Each month a median of 1,555 specimens (interquartile range 1,140-2,740), primarily from hospitals, were sent to the Government Virus Unit of the Department of Health (7). We calculated the highest proportion of positive infl uenza isolations each season, and used these laboratory data to defi ne the onset of each peak activity period when the proportion of positive infl uenza A or B isolates exceeded 30% of the maximum seasonal level (7).Dynamic linear models (8) were used to generate alerts (online Technical Appendix, available from www.cdc.gov/ EID/content/13/7/1154-Techapp.pdf). We determined that an aberration had occurred when the current observation fell outside a forecast interval generated by the model. For methods based on monitoring of single data streams only, an aberration triggers an alert. For simultaneous monitoring of all 8 data streams, we monitored separate aberrations as above and generated alerts based on the fi rst occurrence of any aberration (M1), 2 simultaneous aberrations (M2), the fi rst occurrence of 3 simultaneous aberrations (M3), any 2 aberrations within a 2-week period (M4), and any 3 aberrations within a 2-week period (M5). In the multistream analyses, we compared alerts produced by univariate models, which effectively assumed independence between the data streams, and multivariate models, which allowed for correlation between the data streams (online Technical Appendix).Alerts were compared...

show abstract

Assessing surveillance using sensitivity, specificity and timeliness

Cited by 48 publications

References 27 publications

Performance Metrics for Surveillance Schemes

Performance Metrics for Surveillance Schemes

Optimal Sampling Strategies for Detecting Zoonotic Disease Epidemics

Optimizing Use of Multistream Influenza Sentinel Surveillance Data

Contact Info

Product

Resources

About