Estimating the power to detect a change caused by a vaccine from time series data

Abstract

When evaluating the effects of vaccination programs, it is common to estimate changes in rates of disease before and after vaccine introduction. There are a number of related approaches that attempt to adjust for trends unrelated to the vaccine and to detect changes that coincide with introduction. However, characteristics of the data can influence the ability to estimate such a change. These include, but are not limited to, the number of years of available data prior to vaccine introduction, the expected strength of the effect of the intervention, the strength of underlying secular trends, and the amount of unexplained variability in the data. Sources of unexplained variability include model misspecification, epidemics due to unidentified pathogens, and changes in ascertainment or coding practice among others. In this study, we present a simple simulation framework for estimating the power to detect a decline and the precision of these estimates. We use real-world data from a pre-vaccine period to generate simulated time series where the vaccine effect is specified . We a priori present an interactive web-based tool to implement this approach. We also demonstrate the use of this approach using observed data on pneumonia hospitalization from the states in Brazil from a period prior to introduction of pneumococcal vaccines to generate the simulated time series. We relate the power of the hypothesis tests to the number of cases per year and the amount of unexplained variability in the data and demonstrate how fewer years of data influence the results.

Evaluation strategies for measuring pneumococcal conjugate vaccine impact in low-resource settings

“…This information provides important supportive evidence to recommend the ongoing use of PCV in Laos. However, we were not able to do a hospital administrative review for pneumonia cases post-PCV with a PCV impact evaluation as we had insufficient power to undertake this; a simulation analysis based on pre-PCV13 data suggested that with the case load in the population selected, we had only 63% power to detect a 20% decline in pneumonia admissions in the three years post-PCV13, using an interrupted time series analysis [25]. It is important to undertake power calculations prior to embarking on pneumonia impact evaluations as there is a danger of drawing erroneous conclusions if the study is undertaken but underpowered [26].…”