Retrospective analysis of Covid-19 hospitalization modelling scenarios which guided policy response in France

Abstract

Epidemiological modelling has played a key role in proposing, analyzing and justifying non-pharmaceuticals interventions in response to the COVID-19 pandemic. Despite its importance, evaluations of models’ ability to accurately anticipate the evolution of the disease remain scarce. Thus, robust, systematic, and pre-specified evaluation criteria are needed to assess the relevance of modelling scenarios that guided policy response during the pandemic. We conduct a retrospective assessment of modelling reports which guided policy response in France from April 2020 to April 2022. After systematically verifying the scenarios hypotheses (e.g., exclusion of no-lockdown scenarios when a lockdown was effectively in place), we find that epidemiological models were (a) uncertain, (b) unaccurate, and (c) biased towards an overestimation of predicted COVID-19 related hospitalizations. In more than half of the reports, reality is below or equal to even the best-case scenario. To our knowledge, this is the only national systematic retrospective assessment of COVID-19 pandemic scenarios; such an approach should be reproduced in other countries whenever possible.Abstract FigureGraphical Abstract: Comparison of Pasteur Institute’s scenarios to reality. Forecasting errors of scenarios (colors) compared to reality (black line) are expressed as a percentage of the maximum Intensive Care Units occupancy reached during the covid-19 pandemic in France (horizontal dashed line).