Ensemble Forecasts of Coronavirus Disease 2019 (COVID-19) in the U.S.

Abstract

Background The COVID-19 pandemic has driven demand for forecasts to guide policy and planning. Previous research has suggested that combining forecasts from multiple models into a single "ensemble" forecast can increase the robustness of forecasts. Here we evaluate the real-time application of an open, collaborative ensemble to forecast deaths attributable to COVID-19 in the U.S. Methods Beginning on April 13, 2020, we collected and combined one- to four-week ahead forecasts of cumulative deaths for U.S. jurisdictions in standardized, probabilistic formats to generate real-time, publicly available ensemble forecasts. We evaluated the point prediction accuracy and calibration of these forecasts compared to reported deaths. Results Analysis of 2,512 ensemble forecasts made April 27 to July 20 with outcomes observed in the weeks ending May 23 through July 25, 2020 revealed precise short-term forecasts, with accuracy deteriorating at longer prediction horizons of up to four weeks. At all prediction horizons, the prediction intervals were well calibrated with 92-96% of observations falling within the rounded 95% prediction intervals. Conclusions This analysis demonstrates that real-time, publicly available ensemble forecasts issued in April-July 2020 provided robust short-term predictions of reported COVID-19 deaths in the United States. With the ongoing need for forecasts of impacts and resource needs for the COVID-19 response, the results underscore the importance of combining multiple probabilistic models and assessing forecast skill at different prediction horizons. Careful development, assessment, and communication of ensemble forecasts can provide reliable insight to public health decision makers.

Short-term forecasts to inform the response to the Covid-19 epidemic in the UK

“…As SARS-CoV-2 continues to spread in populations around the world, real time modelling and short-term forecasting can be key tools for short-term resource planning and pandemic management. The short-term forecasts described here and related initiatives in the US 34 , Germany and Poland 35 , and elsewhere are reflecting efforts to provide decision makers with the information they need to make informed decisions. In the UK, similar methodologies to the ones presented here are now used to generate medium-term projections, that is extrapolations over time periods longer than three weeks of what would be expected to happen if nothing changed from the current situation.…”

Impact of Relaxing Covid-19 Social Distancing Measures on Rural North Wales: A Simulation Analysis

“…The Covid-19 mortality forecasts made by the US Centers for Disease Control and Prevention are based on an "ensemble" forecast which combines independently developed forecasts into one aggregate forecast to improve prediction (11). This is equivalent to our model averaging approach, although it may be preferable to weight models based on historical performance (12). Model averaging benefits from possible reduction of predictive error.…”

Reflecting on prediction strategies for epidemics

“… 3 , 47 , 49 , 80 Some of these models extensively use data science and machine learning techniques (J. Baek et al, unpublished data, 2020) or an ensemble approach. 81 …”