A collaborative multiyear, multimodel assessment of seasonal influenza forecasting in the United States

Abstract

Influenza infects an estimated 9–35 million individuals each year in the United States and is a contributing cause for between 12,000 and 56,000 deaths annually. Seasonal outbreaks of influenza are common in temperate regions of the world, with highest incidence typically occurring in colder and drier months of the year. Real-time forecasts of influenza transmission can inform public health response to outbreaks. We present the results of a multiinstitution collaborative effort to standardize the collection and evaluation of forecasting models for influenza in the United States for the 2010/2011 through 2016/2017 influenza seasons. For these seven seasons, we assembled weekly real-time forecasts of seven targets of public health interest from 22 different models. We compared forecast accuracy of each model relative to a historical baseline seasonal average. Across all regions of the United States, over half of the models showed consistently better performance than the historical baseline when forecasting incidence of influenza-like illness 1 wk, 2 wk, and 3 wk ahead of available data and when forecasting the timing and magnitude of the seasonal peak. In some regions, delays in data reporting were strongly and negatively associated with forecast accuracy. More timely reporting and an improved overall accessibility to novel and traditional data sources are needed to improve forecasting accuracy and its integration with real-time public health decision making.

A novel sub-epidemic modeling framework for short-term forecasting epidemic waves

“…The factors influencing infectious disease transmission include the mode of transmission (e.g., close contact, airborne, via vector, sexual route), the individual-level network that captures the dynamics of disease-relevant interactions (which are often influenced by cultural factors) [2], the natural history of the disease, variations in the risk behavior of individuals, reactive public health interventions, the behavior changes in response to an epidemic, and the background immunity of the population shaped by genetic factors and prior exposure to the disease or vaccination campaigns [3][4][5][6]. Our ability to generate accurate epidemic forecasts is challenged by the sparse data on the individual-and grouplevel heterogeneity that affect the dynamics of infectious disease transmission [7][8][9].…”

Coordinating the real‐time use of global influenza activity data for better public health planning

“…A number of different techniques are used in the competition including crowd‐sourced expert judgement forecasts, mechanistic models, machine learning and statistical models . Also, different approaches can be combined into a single ensemble …”

“…A, Forecast made for week 49, 2018 for national percentage of outpatient visits in the United States that would be for influenza‐like illness for the following 4 wks. Based on results from multiple groups . B, Forecasts of the number of laboratory‐confirmed influenza cases in Melbourne made on 2nd September 2018 (blue), with pre‐season “prior” forecast based also on data from previous seasons (brown, made 8th July 2018) (C) Real‐time forecast accuracy for 36 European countries during the 2017‐18 influenza season .…”

“…24 Also, different approaches can be combined into a single ensemble. 25 However, despite this success in translating influenza surveillance data into actionable forecasts and a number of forecasting studies from other locations (eg Refs 26,27 ), there are currently no real-time efforts to apply advanced analytics to influenza data at a global scale. Also, most public health agencies do not have resident expertise in advanced analytics such as infectious disease forecasting, nor the expertise to interpret those results.…”

“…Based on results from multiple groups. 25 B, Forecasts of the number of laboratory-confirmed influenza cases in Melbourne made on 2nd September 2018 (blue), with pre-season "prior" forecast based also on data from previous seasons (brown, made 8th July 2018) 35 (C) Real-time forecast accuracy for 36 European countries during the 2017-18 influenza season. 28 Plots show the proportion of forecasts that accurately predicted peak timing within 1 week of the observed value (red) and that accurately predicted peak intensity within 25% of the observed value (blue).…”