Significance This paper compares the probabilistic accuracy of short-term forecasts of reported deaths due to COVID-19 during the first year and a half of the pandemic in the United States. Results show high variation in accuracy between and within stand-alone models and more consistent accuracy from an ensemble model that combined forecasts from all eligible models. This demonstrates that an ensemble model provided a reliable and comparatively accurate means of forecasting deaths during the COVID-19 pandemic that exceeded the performance of all of the models that contributed to it. This work strengthens the evidence base for synthesizing multiple models to support public-health action.

Wastewater-based epidemiology (WBE) has been shown to coincide with, or anticipate, confirmed COVID-19 case numbers. During periods with high test positivity rates, however, case numbers may be underreported, whereas wastewater does not suffer from this limitation. Here we investigated how the dynamics of new COVID-19 infections estimated based on wastewater monitoring or confirmed cases compare to true COVID-19 incidence dynamics. We focused on the first pandemic wave in Switzerland (February to April, 2020), when test positivity ranged up to 26%. SARS-CoV-2 RNA loads were determined 2-4 times per week in three Swiss wastewater treatment plants (Lugano, Lausanne and Zurich). Wastewater and case data were combined with a shedding load distribution and an infection-to-case confirmation delay distribution, respectively, to estimate incidence dynamics. Finally, the estimates were compared to reference incidence dynamics determined by a validated compartmental model. Incidence dynamics estimated based on wastewater data were found to better track the timing and shape of the reference infection peak compared to estimates based on confirmed cases. In contrast, case confirmations provided a better estimate of the subsequent decline in infections. Under a regime of high-test positivity rates, WBE thus provides critical information that is complementary to clinical data to monitor the pandemic trajectory.

No abstract

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multi-model ensemble forecast that combined predictions from dozens of different research groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-week horizon 3-5 times larger than when predicting at a 1-week horizon. This project underscores the role that collaboration and active coordination between governmental public health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks. Significance Statement This paper compares the probabilistic accuracy of short-term forecasts of reported deaths due to COVID-19 during the first year and a half of the pandemic in the US. Results show high variation in accuracy between and within stand-alone models, and more consistent accuracy from an ensemble model that combined forecasts from all eligible models. This demonstrates that an ensemble model provided a reliable and comparatively accurate means of forecasting deaths during the COVID-19 pandemic that exceeded the performance of all of the models that contributed to it. This work strengthens the evidence base for synthesizing multiple models to support public health action.

Following the rapid dissemination of COVID-19 cases in Switzerland, large-scale non-pharmaceutical interventions (NPIs) were implemented by the cantons and the federal government between 28 February and 20 March 2020. Estimates of the impact of these interventions on SARS-CoV-2 transmission are critical for decision making in this and future outbreaks. We here aim to assess the impact of these NPIs on disease transmission by estimating changes in the basic reproduction number (R 0 ) at national and cantonal levels in relation to the timing of these NPIs. We estimated the time-varying R 0 nationally and in eleven cantons by fitting a stochastic transmission model explicitly simulating within-hospital dynamics. We used individual-level data from more than 1000 hospitalised patients in Switzerland and public daily reports of hospitalisations and deaths. We estimated the national R 0 to be 2.8 (95% confidence interval 2.1-3.8) at the beginning of the epidemic. Starting from around 7 March, we found a strong reduction in time-varying R 0 with a 86% median decrease (95% quantile range [QR] 79-90%) to a value of 0.40 (95% QR 0.3-0.58) in the period of 29 March to 5 April. At the cantonal level, R 0 decreased over the course of the epidemic between 53% and 92%. Reductions in time-varying R 0 were synchronous with changes in mobility patterns as estimated through smartphone activity, which started before the official implementation of NPIs. We inferred that most of the reduction of transmission is attributable to behavioural changes as opposed to natural immunity, the latter accounting for only about 4% of the total reduction in effective transmission. As Switzerland considers relaxing some of the restrictions of social mixing, current estimates of time-varying R 0 well below one are promising. However, as of 24 April 2020, at least 96% (95% QR 95.7-96.4%) of the Swiss population remains susceptible to SARS-CoV-2. These results warrant a cautious relaxation of social distance practices and close monitoring of changes in both the basic and effective reproduction numbers.

Following the rapid dissemination of COVID-19 cases in Switzerland, large-scale non- pharmaceutical interventions (NPIs) were implemented by the cantons and the federal government between February 28 and March 20. Estimates of the impact of these interventions on SARS-CoV-2 transmission are critical for decision making in this and future outbreaks. We here aim to assess the impact of these NPIs on disease transmission by estimating changes in the basic reproduction number (R0) at national and cantonal levels in relation to the timing of these NPIs. We estimate the time- varying R0 nationally and in twelve cantons by fitting a stochastic transmission model explicitly simulating within hospital dynamics. We use individual-level data of >1,000 hospitalized patients in Switzerland and public daily reports of hospitalizations and deaths. We estimate the national R0 was 3.15 (95% CI: 2.13-3.76) at the start of the epidemic. Starting from around March 6, we find a strong reduction in R0 with an 85% median decrease (95% quantile range, QR: 83%-90%) to a value of 0.44 (95% QR: 0.27-0.65) in the period of March 29-April 5. At the cantonal-level R0 decreased over the course of the epidemic between 71% and 94%. We found that reductions in R0 were synchronous with changes in mobility patterns as estimated through smartphone activity, which started before the official implementation of NPIs. We found that most of the reduction of transmission is due to behavioural changes as opposed to natural immunity, the latter accounting for only about 3% of the total reduction in effective transmission. As Switzerland considers relaxing some of the restrictions of social mixing, current estimates of R0 well below one are promising. However most of inferred transmission reduction was due to behaviour change (<3% due to natural immunity buildup), with an estimated 97% (95% QR: 96.6%-97.2%) of the Swiss population still susceptible to SARS-CoV-2 as of April 24. These results warrant a cautious relaxation of social distance practices and close monitoring of changes in both the basic and effective reproduction numbers.

Wastewater-based epidemiology (WBE) has been shown to coincide with, or anticipate, confirmed COVID-19 case numbers. During periods with high test positivity rates, however, case numbers may be underreported, whereas wastewater does not suffer from this limitation. Here we investigated how the dynamics of new COVID-19 infections estimated based on wastewater monitoring or confirmed cases compare to true COVID-19 incidence dynamics. We focused on the first pandemic wave in Switzerland (February to April, 2020), when test positivity ranged up to 26%. SARS-CoV-2 RNA loads were determined 2-4 times per week in three Swiss wastewater treatment plants (Lugano, Lausanne and Zurich). Wastewater and case data were combined with a shedding load distribution and an infection-to-case confirmation delay distribution, respectively, to estimate incidence dynamics. Finally, the estimates were compared to reference incidence dynamics determined by a validated compartmental model. Incidence dynamics estimated based on wastewater data were found to better track the timing and shape of the reference infection peak compared to estimates based on confirmed cases. In contrast, case confirmations provided a better estimate of the subsequent decline in infections. Under a regime of high-test positivity rates, WBE thus provides critical information that is complementary to clinical data to monitor the pandemic trajectory.

Non-pharmaceutical interventions (NPIs) remain the only widely available tool for controlling the ongoing SARS-CoV-2 pandemic. We estimated weekly values of the effective basic reproductive number (Reff) using a mechanistic metapopulation model and associated these with county-level characteristics and NPIs in the United States (US). Interventions that included school and leisure activities closure and nursing home visiting bans were all associated with a median Reff below 1 when combined with either stay at home orders (median Reff 0.97, 95% confidence interval (CI) 0.58–1.39) or face masks (median Reff 0.97, 95% CI 0.58–1.39). While direct causal effects of interventions remain unclear, our results suggest that relaxation of some NPIs will need to be counterbalanced by continuation and/or implementation of others.