National and subnational short-term forecasting of COVID-19 in Germany and Poland during early 2021

Abstract

Background During the COVID-19 pandemic there has been a strong interest in forecasts of the short-term development of epidemiological indicators to inform decision makers. In this study we evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland for the period from January through April 2021. Methods We evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland. These were issued by 15 different forecasting models, run by independent research teams. Moreover, we study the performance of combined ensemble forecasts. Evaluation of probabilistic forecasts is based on proper scoring rules, along with interval coverage proportions to assess calibration. The presented work is part of a pre-registered evaluation study. Results We find that many, though not all, models outperform a simple baseline model up to four weeks ahead for the considered targets. Ensemble methods show very good relative performance. The addressed time period is characterized by rather stable non-pharmaceutical interventions in both countries, making short-term predictions more straightforward than in previous periods. However, major trend changes in reported cases, like the rebound in cases due to the rise of the B.1.1.7 (Alpha) variant in March 2021, prove challenging to predict. Conclusions Multi-model approaches can help to improve the performance of epidemiological forecasts. However, while death numbers can be predicted with some success based on current case and hospitalization data, predictability of case numbers remains low beyond quite short time horizons. Additional data sources including sequencing and mobility data, which were not extensively used in the present study, may help to improve performance.

A Deep Dive into the Delta Wave: Forecasting SARS-CoV-2 Epidemic Dynamic in Poland with the pDyn Agent-Based Model

“…• Due to a meticulous generative description of the epidemic spread, the pDyn model achieves remarkable performance in predicting cases, hospitalizations, ICU hospitalizations, and deaths, as assessed by German and Polish COVID-19 Forecast Hub [24,25] and European COVID-19 Forecast Hub [54]. • Our generative ABM beneőts from complex internal states, enabling the implementation of mechanisms that phenomenological models cannot capture and allowing the incorporation of vast amounts of data.…”

“…A systematic review of 126 SARS-CoV-2 ABMs revealed that only 17% were validated against real-world data, 3% were compared with other models, and 2% were systematically tested [22]. Furthermore, pDyn has continuously undergone external validation with real-world data [23] as part of the German and Polish COVID-19 Forecast Hub since November 2020 [24,25]. Individual ABMs, MOCOS [19] and pDyn, have achieved signiőcant performance improvements in long-term case forecasting in Poland due to their meticulous tailoring to speciőc country situations [25].…”

“…Furthermore, pDyn has continuously undergone external validation with real-world data [23] as part of the German and Polish COVID-19 Forecast Hub since November 2020 [24,25]. Individual ABMs, MOCOS [19] and pDyn, have achieved signiőcant performance improvements in long-term case forecasting in Poland due to their meticulous tailoring to speciőc country situations [25].…”

Combining models to generate a consensus effective reproduction number R for the COVID-19 epidemic status in England

“…Generating a combined estimate from a set of models is not a new concept; they are widely used across many disciplines, such as forecasting the weather [14], hydrology [15], flood losses [16], cancer prediction [17] and climate modelling [18]. Within infectious diseases, combined model estimates have been applied to modelling human immunodeficiency virus (HIV) [19], influenza [20], and Ebola [21,22] transmission and recently for outbreak analysis related to COVID-19 in the United States [23] and Europe [24].…”

Combining models to generate a consensus effective reproduction numberRfor the COVID-19 epidemic status in England

“…Generating a combined estimate from a set of models is not a new concept; they are widely used across many disciplines; in forecasting the weather [41], hydrology [31], flood losses [22], in cancer prediction [59] and climate modelling [43]. Within infectious diseases, combined model estimates have been applied to modelling HIV [21], influenza [48] and Ebola [51, 13] transmission, and recently for outbreak analysis related to COVID-19 in the USA [53] and Europe [9].…”