Estimating the population-level impact of vaccines using synthetic controls

Abstract

When a new vaccine is introduced, it is critical to monitor trends in disease rates to ensure that the vaccine is effective and to quantify its impact. However, estimates from observational studies can be confounded by unrelated changes in healthcare utilization, changes in the underlying health of the population, or changes in reporting. Other diseases are often used to detect and adjust for these changes, but choosing an appropriate control disease a priori is a major challenge. The “synthetic controls” (causal impact) method, which was originally developed for website analytics and social sciences, provides an appealing solution. With this approach, potential comparison time series are combined into a composite and are used to generate a counterfactual estimate, which can be compared with the time series of interest after the intervention. We sought to estimate changes in hospitalizations for all-cause pneumonia associated with the introduction of pneumococcal conjugate vaccines (PCVs) in five countries in the Americas. Using synthetic controls, we found a substantial decline in hospitalizations for all-cause pneumonia in infants in all five countries (average of 20%), whereas estimates for young and middle-aged adults varied by country and were potentially influenced by the 2009 influenza pandemic. In contrast to previous reports, we did not detect a decline in all-cause pneumonia in older adults in any country. Synthetic controls promise to increase the accuracy of studies of vaccine impact and to increase comparability of results between populations compared with alternative approaches.

The changing epidemiology of community‐acquired pneumonia: nationwide register‐based study in Sweden

“…Reductions in AC‐CAP hospitalizations of 36%, 20% and 16% in children <2, 2–4 and 5–17 years of age, respectively, and even more pronounced decreases in PL‐CAP, are consistent with reports from most other countries . These findings are concordant with previous publications from Sweden that reported up to 37% reduction of inpatient all‐cause pneumonia in children < 2 years in 2012 .…”

“…The first reports on reduced incidence of AC-CAP hospitalizations came from the United States, and thereafter from Canada and Finland [11,12,15]. However, in a recent study that adjusted for secular trends using the synthetic control method, there was no observed reduction of hospitalization for all-cause pneumonia after PCV-CIP introduction in any of five countries in the Americas (the United States, Mexico, Ecuador, Brazil and Chile) [9]. Neither was there a reduction in pneumonia among the elderly in Australia, and studies from the UK and Denmark have observed similar increases of AC-CAP in the elderly as in our study [13,14,16,27,28].…”

“…Yet, the conjugate vaccines only cover 7-13 pneumococcal serotypes and due to expansion of nonvaccine serotypes, the overall impact on IPD in adults has been small in many settings, especially in older persons [5,8]. The impact of PCV childhood immunization programmes (PCV-CIP) on CAP among children show a reduced burden of disease for both all-cause pneumonia and pneumonia with more pneumococcal-specific diagnoses, but there are discrepant results concerning the impact among the elderly [9][10][11][12][13][14][15][16][17].…”

Direct and indirect impact of 10-valent pneumococcal conjugate vaccine introduction on pneumonia hospitalizations and economic burden in all age-groups in Brazil: A time-series analysis

“…The comparison groups were a combination of multiple sets of diseases, having as a basis the ICD-10 codes groups proposed by Bruhn et al . [ 26 ], but performing a manual selection of which ones should be excluded from each age-groups. Excluded disease groups were those potentially influenced by the introduction of PCV-10 vaccine, those whose trend in the pre-vaccination period differed much from the observed trend for pneumonia, those that could have suffered the concurrent effect of another national public health intervention and those that were related to very short-stay or very long-stay hospitalizations.…”

Estimating the power to detect a change caused by a vaccine from time series data

“…As a demonstration of the point-and-click interface, we use hospitalization data from Chile among children <24 months (raw data available from http://www.deis.cl/) 5 Figure 5B). This yields 54% power to detect a rate ratio of 0.8 ( Figure 5B).…”