Significance This paper compares the probabilistic accuracy of short-term forecasts of reported deaths due to COVID-19 during the first year and a half of the pandemic in the United States. Results show high variation in accuracy between and within stand-alone models and more consistent accuracy from an ensemble model that combined forecasts from all eligible models. This demonstrates that an ensemble model provided a reliable and comparatively accurate means of forecasting deaths during the COVID-19 pandemic that exceeded the performance of all of the models that contributed to it. This work strengthens the evidence base for synthesizing multiple models to support public-health action.

Background: Training in innovation and entrepreneurship (I&E) in medical education has become increasingly prevalent among medical schools to train students in complex problem solving and solution design. Objective: We aim to characterize I&E education in US allopathic medical schools to provide insight into the features and objectives of this growing field. Design: I&E programs were identified in 2016 via structured searches of 158 US allopathic medical school websites. Program characteristics were identified from public program resources and structured phone interviews with program directors. Curricular themes were identified via thematic analysis of program resources, and themes referenced by >50% of programs were analyzed. Results: Thirteen programs were identified. Programs had a median age of four years, and contained a median of 13 students. Programs were led by faculty from diverse professional backgrounds, and all awarded formal recognition to graduates. Nine programs spanned all four years of medical school and ten programs required a capstone project. Thematic analysis revealed seven educational themes (innovation, entrepreneurship, technology, leadership, healthcare systems, business of medicine, and enhanced adaptability) and two teaching method themes (active learning, interdisciplinary teaching) referenced by >50% of programs. Conclusions: The landscape of medical school I&E programs is rapidly expanding to address newfound skills needed by physicians due to ongoing changes in healthcare, but programs remain relatively few and small compared to class size. This landscape analysis is the first review of I&E in medical education and may contribute to development of a formal educational framework or competency model for current or future programs. Abbreviations: AAMC: American Association of Medical Colleges; AMA: American Medical Association; I&E: Innovation and entrepreneurship

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multi-model ensemble forecast that combined predictions from dozens of different research groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-week horizon 3-5 times larger than when predicting at a 1-week horizon. This project underscores the role that collaboration and active coordination between governmental public health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks. Significance Statement This paper compares the probabilistic accuracy of short-term forecasts of reported deaths due to COVID-19 during the first year and a half of the pandemic in the US. Results show high variation in accuracy between and within stand-alone models, and more consistent accuracy from an ensemble model that combined forecasts from all eligible models. This demonstrates that an ensemble model provided a reliable and comparatively accurate means of forecasting deaths during the COVID-19 pandemic that exceeded the performance of all of the models that contributed to it. This work strengthens the evidence base for synthesizing multiple models to support public health action.

Managing the outbreak of COVID-19 in India constitutes an unprecedented health emergency in one of the largest and most diverse nations in the world. On May 4, 2020, India started the process of releasing its population from a national lockdown during which extreme social distancing was implemented. We describe and simulate an adaptive control approach to exit this situation, while maintaining the epidemic under control. Adaptive control is a flexible countercyclical policy approach, whereby different areas release from lockdown in potentially different gradual ways, dependent on the local progression of the disease. Because of these features, adaptive control requires the ability to decrease or increase social distancing in response to observed and projected dynamics of the disease outbreak. We show via simulation of a stochastic Susceptible-Infected-Recovered (SIR) model and of a synthetic intervention (SI) model that adaptive control performs at least as well as immediate and full release from lockdown starting May 4 and as full release from lockdown after a month (i.e., after May 31). The key insight is that adaptive response provides the option to increase or decrease socioeconomic activity depending on how it affects disease progression and this freedom allows it to do at least as well as most other policy alternatives. We also discuss the central challenge to any nuanced release policy, including adaptive control, specifically learning how specific policies translate into changes in contact rates and thus COVID-19's reproductive rate in real time.

Academic researchers, government agencies, industry groups, and individuals have produced forecasts at an unprecedented scale during the COVID-19 pandemic. To leverage these forecasts, the United States Centers for Disease Control and Prevention (CDC) partnered with an academic research lab at the University of Massachusetts Amherst to create the US COVID-19 Forecast Hub. Launched in April 2020, the Forecast Hub is a dataset with point and probabilistic forecasts of incident cases, incident hospitalizations, incident deaths, and cumulative deaths due to COVID-19 at county, state, and national, levels in the United States. Included forecasts represent a variety of modeling approaches, data sources, and assumptions regarding the spread of COVID-19. The goal of this dataset is to establish a standardized and comparable set of short-term forecasts from modeling teams. These data can be used to develop ensemble models, communicate forecasts to the public, create visualizations, compare models, and inform policies regarding COVID-19 mitigation. These open-source data are available via download from GitHub, through an online API, and through R packages.

As we reach the apex of the COVID-19 pandemic across the globe, the most pressing question facing us all is: can we, even partially, reopen the economy without risking the second wave? Towards answering this question, we first need to understand if shutting down the economy helped. And second if it did, is it possible to achieve similar gains in the war against the pandemic while partially opening up the economy? To do so, it is critical to understand the effects of the various interventions that can be put into place and their corresponding health and economic implications. Since many possible interventions exist, the key challenge facing policy makers is understanding the potential trade-offs between them, and choosing the particular set of interventions that works best for their circumstance -for example, what would be the effect on COVID-19 related deaths if the U.S. had decreased mobility by 20% instead of 60% versus if Italy had done the same?In this memo, we provide an overview of Synthetic Interventions [3] (a natural generalization of the widely applied Synthetic Control method [2]), which is a data-driven and statistically principled method to perform what-if scenario planning, i.e., for policy makers to understand the trade-offs between different interventions before having to actually enact them. In essence, the Synthetic Interventions method leverages information from different interventions that have already been enacted across the world and fits it to a policy maker's setting of interest -for instance, to estimate the effect of mobility-restricting interventions on the U.S., we use daily death data from countries that enforced severe mobility restrictions to create a "synthetic low mobility U.S." and predict the counterfactual trajectory of the U.S. if it had indeed applied a similar intervention.We use the Synthetic Interventions method towards answering the two questions laid out above via a case study of the effectiveness of different interventions in mitigating the COVID-19 pandemic at the international level. Encouragingly, we find that lifting severe mobility restrictions and only retaining moderate mobility restrictions (at retail and transit locations), seems to effectively flatten the curve. We hope that this will provide useful guidance for policy makers as they weigh the trade-offs between the safety of the population, strain on the healthcare system, and impact on the economy.

No abstract

Recent methods in the machine learning literature have proposed a Gaussian noisebased exogenous signal to learn the parameters of a dynamic system. In this paper, we propose the use of a spectral lines-based deterministic exogenous signal to solve the same problem. Our theoretical analysis consists of a new toolkit which employs the theory of spectral lines, retains the stochastic setting, and leads to non-asymptotic bounds on the parameter estimation error. The results are shown to lead to a tunable parameter identification error. In particular, it is shown that the identification error can be minimized through an an optimal choice of the spectrum of the exogenous signal.Preprint. Under review.