Significance This paper compares the probabilistic accuracy of short-term forecasts of reported deaths due to COVID-19 during the first year and a half of the pandemic in the United States. Results show high variation in accuracy between and within stand-alone models and more consistent accuracy from an ensemble model that combined forecasts from all eligible models. This demonstrates that an ensemble model provided a reliable and comparatively accurate means of forecasting deaths during the COVID-19 pandemic that exceeded the performance of all of the models that contributed to it. This work strengthens the evidence base for synthesizing multiple models to support public-health action.

The COVID-19 pandemic has forced societies across the world to resort to social distancing to slow the spread of the SARS-CoV-2 virus. Due to the economic impacts of social distancing, there is growing desire to relax these measures. To characterize a range of possible strategies for control and to understand their consequences, we performed an optimal control analysis of a mathematical model of SARS-CoV-2 transmission. Given that the pandemic is already underway and controls have already been initiated, we calibrated our model to data from the USA and focused our analysis on optimal controls from May 2020 through December 2021. We found that a major factor that differentiates strategies that prioritize lives saved versus reduced time under control is how quickly control is relaxed once social distancing restrictions expire in May 2020. Strategies that maintain control at a high level until at least summer 2020 allow for tapering of control thereafter and minimal deaths, whereas strategies that relax control in the short term lead to fewer options for control later and a higher likelihood of exceeding hospital capacity. Our results also highlight that the potential scope for controlling COVID-19 until a vaccine is available depends on epidemiological parameters about which there is still considerable uncertainty, including the basic reproduction number and the effectiveness of social distancing. In light of those uncertainties, our results do not constitute a quantitative forecast and instead provide a qualitative portrayal of possible outcomes from alternative approaches to control.

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multi-model ensemble forecast that combined predictions from dozens of different research groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-week horizon 3-5 times larger than when predicting at a 1-week horizon. This project underscores the role that collaboration and active coordination between governmental public health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks. Significance Statement This paper compares the probabilistic accuracy of short-term forecasts of reported deaths due to COVID-19 during the first year and a half of the pandemic in the US. Results show high variation in accuracy between and within stand-alone models, and more consistent accuracy from an ensemble model that combined forecasts from all eligible models. This demonstrates that an ensemble model provided a reliable and comparatively accurate means of forecasting deaths during the COVID-19 pandemic that exceeded the performance of all of the models that contributed to it. This work strengthens the evidence base for synthesizing multiple models to support public health action.

The COVID-19 pandemic has forced societies across the world to resort to social distancing to slow the spread of the SARS-CoV-2 virus. Due to the economic impacts of social distancing, there is growing desire to relax these measures. To characterize a range of possible strategies for control and to understand their consequences, we performed an optimal control analysis of a mathematical model of SARS-CoV-2 transmission. Given that the pandemic is already underway and controls have already been initiated, we calibrated our model to data from the US and focused our analysis on optimal controls from May 2020 through December 2021. We found that a major factor that differentiates strategies that prioritize lives saved versus reduced time under control is how quickly control is relaxed once social distancing restrictions expire in May 2020. Strategies that maintain control at a high level until summer 2020 allow for tapering of control thereafter and minimal deaths, whereas strategies that relax control in the short term lead to fewer options for control later and a higher likelihood of exceeding hospital capacity. Our results also highlight that the potential scope for controlling COVID-19 until a vaccine is available depends on epidemiological parameters about which there is still considerable uncertainty, including the basic reproduction number and the effectiveness of social distancing. In light of those uncertainties, our results do not constitute a quantitative forecast and instead provide a qualitative portrayal of possible outcomes from alternative approaches to control.

Importance: In the United States, schools closed in March 2020 to reduce the burden of COVID-19. They are now reopening amid high incidence in many places, necessitating analyses of the associated risks and benefits. Objective: To determine the impact of school reopening with varying levels of operating capacity and face-mask adherence on COVID-19 burden. Design: Modeling study using an agent-based model that simulates daily activities of the population. Transmission can occur in places such as schools, workplaces, community, and households. Model parameters were calibrated to and validated against multiple types of COVID-19 data. Setting: Indiana, United States of America. Participants: Synthetic population of Indiana. K-12 students, teachers, their families, and others in the state were studied separately. Interventions: Reopening of schools under three levels of school operating capacity (50%, 75%, and 100%), as well as three assumptions about face-mask adherence in schools (50%, 75%, and 100%). We compared the impact of these scenarios to reopening at full capacity without face masks and a scenario with schools operating remotely, for a total of 11 scenarios. Main outcomes: SARS-CoV-2 infections, symptomatic cases, and deaths due to COVID-19 from August 24 to December 31. Results: We projected 19,527 (95% CrI: 4,641-56,502) infections and 360 (95% CrI: 67-967) deaths in the scenario where schools operated remotely from August 24 to December 31. Reopening at full capacity with low face-mask adherence in schools resulted in a proportional increase of 81.7 (95% CrI: 78.2-85.3) times the number of infections and 13.4 (95% CrI: 12.8-14.0) times the number of deaths. High face-mask adherence resulted in a proportional increase of 3.0 (95% CrI: 2.8-3.1) times the number of infections. Operating at reduced capacity with high face-mask adherence resulted in only an 11.6% (95% CrI: 5.50%-17.9%) increase in the number of infections. Conclusions and Relevance: Reduced capacity and high face-mask adherence in schools would substantially reduce the burden of COVID-19 in schools and across the state. We did not explore the impact of other reopening scenarios, such as alternating days of attendance. Heterogeneous decisions could be made across different districts throughout the state, which our model does not capture. Hence, caution should be taken in interpreting our results as specific quantitative targets for operating capacity or face-mask adherence. Rather, our results suggest that schools should give serious consideration to reducing capacity as much as is feasible and enforcing adherence to wearing face masks.