A simulation study to evaluate the performance of five statistical monitoring methods when applied to different time-series components in the context of control programs for endemic diseases

Abstract

Disease monitoring and surveillance play a crucial role in control and eradication programs, as it is important to track implemented strategies in order to reduce and/or eliminate a specific disease. The objectives of this study were to assess the performance of different statistical monitoring methods for endemic disease control program scenarios, and to explore what impact of variation (noise) in the data had on the performance of these monitoring methods. We simulated 16 different scenarios of changes in weekly sero-prevalence. The changes included different combinations of increases, decreases and constant sero-prevalence levels (referred as events). Two space-state models were used to model the time series, and different statistical monitoring methods (such as univariate process control algorithms–Shewart Control Chart, Tabular Cumulative Sums, and the V-mask- and monitoring of the trend component–based on 99% confidence intervals and the trend sign) were tested. Performance was evaluated based on the number of iterations in which an alarm was raised for a given week after the changes were introduced. Results revealed that the Shewhart Control Chart was better at detecting increases over decreases in sero-prevalence, whereas the opposite was observed for the Tabular Cumulative Sums. The trend-based methods detected the first event well, but performance was poorer when adapting to several consecutive events. The V-Mask method seemed to perform most consistently, and the impact of noise in the baseline was greater for the Shewhart Control Chart and Tabular Cumulative Sums than for the V-Mask and trend-based methods. The performance of the different statistical monitoring methods varied when monitoring increases and decreases in disease sero-prevalence. Combining two of more methods might improve the potential scope of surveillance systems, allowing them to fulfill different objectives due to their complementary advantages.

Comparison of time-series models for monitoring temporal trends in endemic diseases sero-prevalence: lessons from porcine reproductive and respiratory syndrome in Danish swine herds

“…Two Bayesian forecast models, a DLM and a DGLM, both with a linear trend, as described by [6] were used to model the different time-series. The aim of these models is to estimate the underlying true value combining the observed data (i.e.…”

“…However, these methods might result in false alarms when applied to laboratory diagnostic data characterized by random noise and, as a consequence, with the costs of investigation of these alarms as well as a lower trust on the monitoring system. One alternative approach could be to monitor the trend of the underlying level of the time series, which can be positive or negative depending on whether the time series exhibits an increasing or decreasing pattern [5, 6]. This is particularly useful for monitoring temporal changes in trends laboratory diagnostic results collected as part of voluntary disease monitoring programs.…”

“…In a previous simulation study [6], monitoring the trend component showed great potential as a basis for monitoring temporal changes in diseases prevalence. However, those methods were not applied to real laboratory diagnostic data collected as part of voluntary disease monitoring programs, where challenges such as shifts in the collection frequency often occur.…”

Close to a Decade of Decrease in Antimicrobial Usage in Danish Pig Production–Evaluating the Effect of the Yellow Card Scheme

“…A univariate dynamic linear model (DLM) with a local linear trend component, as described in detail by West and Harrison (11) and applied in previous studies (12)(13)(14)(15), was used to model data at the herd level. A previous study showed that Bayesian forecasting methods adapt faster to changes in the data, compared to the deterministic Holt's linear trend methods for monitoring trends of time-series (14).…”

“…The growth was extracted from the θ vector of the model for each time step t for every single herd. The variance of the trend parameter, calculated from the variance-covariance matrix for the posterior distribution, as previously described (12), was used to calculate 95% credible intervals (CI). The percentage of herds with a negative growth or a significant decline (based on 95%) below zero, CI in the antimicrobial consumption for a given month i were both calculated as:…”