Background Short-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here we report on the performance of ensembles in predicting COVID-19 cases and deaths across Europe between 08 March 2021 and 07 March 2022. Methods We used open-source tools to develop a public European COVID-19 Forecast Hub. We invited groups globally to contribute weekly forecasts for COVID-19 cases and deaths reported from a standardised source over the next one to four weeks. Teams submitted forecasts from March 2021 using standardised quantiles of the predictive distribution. Each week we created an ensemble forecast, where each predictive quantile was calculated as the equally-weighted average (initially the mean and then from 26th July the median) of all individual models’ predictive quantiles. We measured the performance of each model using the relative Weighted Interval Score (WIS), comparing models’ forecast accuracy relative to all other models. We retrospectively explored alternative methods for ensemble forecasts, including weighted averages based on models’ past predictive performance. Results Over 52 weeks we collected and combined up to 28 forecast models for 32 countries. We found a weekly ensemble had a consistently strong performance across countries over time. Across all horizons and locations, the ensemble performed better on relative WIS than 84% of participating models’ forecasts of incident cases (with a total N=862), and 92% of participating models’ forecasts of deaths (N=746). Across a one to four week time horizon, ensemble performance declined with longer forecast periods when forecasting cases, but remained stable over four weeks for incident death forecasts. In every forecast across 32 countries, the ensemble outperformed most contributing models when forecasting either cases or deaths, frequently outperforming all of its individual component models. Among several choices of ensemble methods we found that the most influential and best choice was to use a median average of models instead of using the mean, regardless of methods of weighting component forecast models. Conclusions Our results support the use of combining forecasts from individual models into an ensemble in order to improve predictive performance across epidemiological targets and populations during infectious disease epidemics. Our findings further suggest that median ensemble methods yield better predictive performance more than ones based on means. Our findings also highlight that forecast consumers should place more weight on incident death forecasts than incident case forecasts at forecast horizons greater than two weeks. Code and data availability All data and code are publicly available on Github: covid19-forecast-hub-europe/euro-hub-ensemble.
Studies demonstrating the waning of post-vaccination and post-infection immunity against covid-19 generally analyzed a limited range of vaccines or subsets of populations. Using Czech national health data from the beginning of the covid-19 pandemic till November 20, 2021 we estimated the risks of reinfection, breakthrough infection, hospitalization and death by a Cox regression adjusted for sex, age, vaccine type and vaccination status. Vaccine effectiveness against infection declined from 87% at 0-2 months after the second dose to 53% at 7-8 months for BNT162b2 vaccine, from 90% at 0-2 months to 65% at 7-8 months for mRNA-1273, and from 83% at 0-2 months to 55% at 5-6 months for the ChAdOx1-S. Effectiveness against hospitalization and deaths declined by about 15% and 10%, respectively, during the first 6-8 months. Boosters (third dose) returned the protection to the levels observed shortly after dose 2. In unvaccinated, previously infected individuals the protection against infection declined from 97% after 2 months to 72% at 18 months. Our results confirm the waning of vaccination-induced immunity against infection and a smaller decline in the protection against hospitalization and death. Boosting restores the original vaccine effectiveness. Post-infection immunity also decreases over time.
SummaryBackgroundEvidence is accumulating that the effectiveness of covid-19 vaccines against infection wanes, reaching relatively low values after 6 months. Published studies demonstrating this effect based their findings on a limited range of vaccines or subset of populations, and did not include booster vaccine doses or immunity obtained due to covid-19 infection. Here we evaluate effectiveness of covid-19 vaccines, booster doses or previous infection against covid-19 infection, hospital admission or death for the whole population in the Czech Republic.MethodsData used in this study cover the whole population of the Czech Republic reported as infected and/or vaccinated between the first detected case on March 1, 2020 and November 20, 2021 (for reinfections), or December 26, 2020 and November 20, 2021 (for vaccinations), including hospital admissions and deaths. Vaccinations by all vaccines approved in the EU were included in this study. Anonymous, individual-level data including dates of vaccination, infection, hospital admission and death were provided by the the Institute of Health Information and Statistics of the Czech Republic. The risks of reinfection, breakthrough infection after vaccination, hospital admission and death were calculated using hazard ratios from a Cox regression adjusted for sex, age, vaccine type and vaccination status.FindingsThe vaccine effectiveness against any PCR-confirmed infection declined from 87% (95% CI 86-87) at 0-2 months after the second dose to 53% (95% CI 52-54) at 7-8 months for Comirnaty, from 90% (95% CI 89-91) at 0-2 monthsto 65% (95% CI 63-67) at 7-8 months for Spikevax, and from 83% (95% CI 80-85) at 0-2 months to 55% at (95% CI 54-56) 5-6 months for the Vaxzevria. For Janssen Covid-19 Vaccine we found no significant decline but the estimates are less certain. The vaccine effectiveness against hospital admissions and deaths decayed at a significantly lower rate with about 15%, resp. 10% decline during the first 6-8 months. The administration of a booster dose returns the protection to or above the estimates in the first two months after dose 2. In unvaccinated but previously SARS-CoV-2-positive individuals the protection against PCR-confirmed SARS-CoV-2 infection declined from close to 97% (95% CI 97-97) after 2 months through 90% at 6 months down to 72% (95% CI 65-78) at 18 months.InterpretationOur results confirm the waning of vaccination-induced immunity against infection and a smaller decline in the protection against hospital admission and death. A booster dose is shown to restore the vaccine effectiveness back to the levels seen soon after the completion of the basic vaccination schedule. The post-infection immunity decreases over time, too.FundingNo external funding was used to conduct this study.Research in contextEvidence before this studyAccumulating evidence from several countries indicates that the effectiveness of covid-19 vaccines against infection declines in time, from about 80-90% shortly after completing the vaccination to about 50-60% and even less after 6 months. Published studies also suggest a significant boosting in vaccine effectiveness against infection about one week after the third vaccine dose. However, these observations come from different and often limited data sets. Moreover, the existing studies do not compare the decline in vaccine effectiveness with a decline in infection-based immunity in unvaccinated individuals.Added value of this studyIn our study, we bring together data on infections, vaccinations (including booster doses), hospital admissions and deaths to estimate how the protection due to vaccination or previous SARS-CoV-2 infection declines with time, for the whole population of the Czech Republic. Our findings show an overall decrease in vaccine effectiveness over time and a large increase after the administration of a booster dose. At the same time we show a fairly stable and high post-infection immunity over the study period. We hope this evidence will contribute to a better understanding of the changing impact of vaccines and previous infection in complex, real-world environments, which is crucial for the development of more effective and more easily communicated public health policies.Implications of all the available evidenceOur results strongly support a timely and widespread application of booster vaccine doses since their application appears to restore the vaccine-induced protection to the levels attained soon after completing the original vaccination scheme, including the high protection against mild disease or asymptomatic infection.
Background: Short-term forecasts of infectious disease contribute to situational awareness and capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise forecasts’ predictive performance by combining independent models into an ensemble. Here we report the performance of ensemble predictions of COVID-19 cases and deaths across Europe from March 2021 to March 2022.Methods: We created the European COVID-19 Forecast Hub, an online open-access platform where modellers upload weekly forecasts for 32 countries with results publicly visualised and evaluated. We created a weekly ensemble forecast from the equally-weighted average across individual models' predictive quantiles. We measured forecast accuracy using a baseline and relative Weighted Interval Score (rWIS). We retrospectively explored ensemble methods, including weighting by past performance.Results: We collected weekly forecasts from 48 models, of which we evaluated 29 models alongside the ensemble model. The ensemble had a consistently strong performance across countries over time, performing better on rWIS than 91% of forecasts for deaths (N=763 predictions from 20 models), and 83% forecasts for cases (N=886 predictions from 23 models). Performance remained stable over a 4-week horizon for death forecasts but declined with longer horizons for cases. Among ensemble methods, the most influential choice came from using a median average instead of the mean, regardless of weighting component models.Conclusions: Our results support combining independent models into an ensemble forecast to improve epidemiological predictions, and suggest that median averages yield better performance than methods based on means. We highlight that forecast consumers should place more weight on incident death forecasts than case forecasts at horizons greater than two weeks.Funding: European Commission, Ministerio de Ciencia, Innovación y Universidades, FEDER; Agència de Qualitat i Avaluació Sanitàries de Catalunya; Netzwerk Universitätsmedizin; Health Protection Research Unit; Wellcome Trust; European Centre for Disease Prevention and Control; Ministry of Science and Higher Education of Poland; Federal Ministry of Education and Research; Los Alamos National Laboratory; German Free State of Saxony; NCBiR; FISR 2020 Covid-19 I Fase; Spanish Ministry of Health / REACT-UE (FEDER); National Institutes of General Medical Sciences; Ministerio de Sanidad/ISCIII; PERISCOPE European H2020; PERISCOPE European H2021; InPresa; National Institutes of Health, NSF, US Centers for Disease Control and Prevention, Google, University of Virginia, Defense Threat Reduction Agency.
Running across the globe for nearly 2 years, the Covid-19 pandemic keeps demonstrating its strength. Despite a lot of understanding, uncertainty regarding the efficiency of interventions still persists. We developed an age-structured epidemic model parameterized with epidemiological and sociological data for the first Covid-19 wave in the Czech Republic and found that (1) starting the spring 2020 lockdown 4 days earlier might prevent half of the confirmed cases by the end of lockdown period, (2) personal protective measures such as face masks appear more effective than just a realized reduction in social contacts, (3) the strategy of sheltering just the elderly is not at all effective, and (4) leaving schools open is a risky strategy. Despite vaccination programs, evidence-based choice and timing of non-pharmaceutical interventions remains an effective weapon against the Covid-19 pandemic.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.