Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S.

Abstract

Seasonal influenza results in substantial annual morbidity and mortality in the United States and worldwide. Accurate forecasts of key features of influenza epidemics, such as the timing and severity of the peak incidence in a given season, can inform public health response to outbreaks. As part of ongoing efforts to incorporate data and advanced analytical methods into public health decision-making, the United States Centers for Disease Control and Prevention (CDC) has organized seasonal influenza forecasting challenges since the 2013/ 2014 season. In the 2017/2018 season, 22 teams participated. A subset of four teams created a research consortium called the FluSight Network in early 2017. During the 2017/2018 season they worked together to produce a collaborative multi-model ensemble that combined 21 separate component models into a single model using a machine learning technique called stacking. This approach creates a weighted average of predictive densities where the weight for each component is determined by maximizing overall ensemble accuracy over past seasons. In the 2017/2018 influenza season, one of the largest seasonal outbreaks in the last 15 years, this multi-model ensemble performed better on average than all individual component models and placed second overall in the CDC challenge. It also outperformed the baseline multi-model ensemble created by the CDC that took a simple average of all models submitted to the forecasting challenge. This project shows that collaborative efforts between research teams to develop ensemble forecasting approaches can bring measurable improvements in forecast accuracy and important reductions in the variability of performance from year to year. Efforts such as this, that emphasize real-time testing and evaluation of forecasting models and facilitate the close collaboration between

Characteristic CT findings distinguishing 2019 novel coronavirus disease (COVID-19) from influenza pneumonia

“…Additionally, influenza has been a very common cause of respiratory tract infection in this period in China. Influenza leads to significant morbidity and mortality worldwide every year [19]. Influenza pneumonia and COVID-19 usually show similar clinical manifestations, such as cough, fever, and myalgia.…”

Improving Probabilistic Infectious Disease Forecasting Through Coherence

“…We saw that the forecast skill benefit for both methods was largest in the 2017/2018 season. In addition, the relative difference between the unordered OLS method and the ordered OLS method was also largest during this season, suggesting that exploiting the correlation structure matters most in difficult seasons [12]. This is intuitively true, since during an epidemic season both the regions and the national wILI is elevated beyond normal levels, so the forecasting models tend to under-predict at all levels of the hierarchy.…”

“…Here we see that both methods improve over the independent forecasts when stratified by seasons. Additionally, the ordered OLS method showed the greatest improvement in the season that was hardest to predict out of the three: the 2017/2018 season [12]. In fact, the top skills obtained by all participants in the FluSight challenge during the test seasons were .45, .33, and .44 respectively, representing a noticeable drop in forecast skill for the 2017/2018 season.…”

“…Participants in the FluSight challenge have harnessed a variety of models and methods to forecast the targets under consideration, which include both short-term forecasts and seasonal targets. These efforts have included time series models [6], mechanistic disease transmission models [7] [8], and machine learning techniques [9][10] [11] [12]. Teams have also incorporated external data, such as internet search queries or point of care data, to improve forecasts [13] [14][15] [16].…”

Mathematical Models for COVID-19 Pandemic: A Comparative Analysis

“…Like projection approaches, models for epidemic forecasting can be broadly classified into two broad groups: (i) statistical and machine learning-based data-driven models, (ii) causal or mechanistic models—see 29 , 30 , 2 , 31 , 32 , 6 , 33 and the references therein for the current state of the art in this rapidly evolving field.…”