The impact of uncertainty on predictions of the CovidSim epidemiological code

Edeling, Wouter; Arabnejad, Hamid; Sinclair, Robbie; Suleimenova, Diana; Gopalakrishnan, Krishnakumar; Bosak, Bartosz; Groen, Derek; Mahmood, Imran; Crommelin, Daan; Coveney, Peter V.

doi:10.1038/s43588-021-00028-9

Cited by 65 publications

(73 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yet our analysis comprehends a larger set of uncertainties and is mathematically more rigorous as it is based on the theory and concepts of uncertainty quantification and sensitivity analysis. UQ techniques are not often applied to epidemiological models; an exception is the recent study by Edeling et al [35]. However the analysis presented there has a different scope from our study: the analyzed model and the UQ methods differ, moreover our study encompasses a broader set of strategies and addresses extensively the intrinsic model uncertainty.…”

Section: Discussionmentioning

confidence: 93%

Uncertainty quantification and sensitivity analysis of COVID-19 exit strategies in an individual-based transmission model

Gugole

Coffeng

Edeling

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Many countries are currently dealing with the COVID-19 epidemic and are searching for an exit strategy such that life in society can return to normal. To support this search, computational models are used to predict the spread of the virus and to assess the efficacy of policy measures before actual implementation. The model output has to be interpreted carefully though, as computational models are subject to uncertainties. These can stem from, e.g., limited knowledge about input parameters values or from the intrinsic stochastic nature of some computational models. They lead to uncertainties in the model predictions, raising the question what distribution of values the model produces for key indicators of the severity of the epidemic. Here we show how to tackle this question using techniques for uncertainty quantification and sensitivity analysis. We assess the uncertainties and sensitivities of four exit strategies implemented in an agent-based transmission model with geographical stratification. The exit strategies are termed Flattening the Curve, Contact Tracing, Intermittent Lockdown and Phased Opening. We consider two key indicators of the ability of exit strategies to avoid catastrophic health care overload: the maximum number of prevalent cases in intensive care (IC), and the total number of IC patient-days in excess of IC bed capacity. Our results show that uncertainties not directly related to the exit strategies are secondary, although they should still be considered in comprehensive analysis intended to inform policy makers. The sensitivity analysis discloses the crucial role of the intervention uptake by the population and of the capability to trace infected individuals. Finally, we explore the existence of a safe operating space. For Intermittent Lockdown we find only a small region in the model parameter space where the key indicators of the model stay within safe bounds, whereas this region is larger for the other exit strategies.

show abstract

Section: Discussionmentioning

confidence: 93%

Uncertainty quantification and sensitivity analysis of COVID-19 exit strategies in an individual-based transmission model

Gugole

Coffeng

Edeling

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The same model may perform differently if applied to datasets from those two sources. Therefore, because of the multitude of uncertainties, quantifying the parametric input uncertainty is not sufficient 5 . Consequently, the model selection becomes the major issue, in our opinion, not only on how to best fit the data but also on how robust the model is given these uncertainties.…”

Section: Discussionmentioning

confidence: 99%

“…In addition to seasonal variation in transmission, e.g., due to weather or mobility, for a given model the data uncertainty is usually propagated into model parameters, rendering them as random variables/processes with an underlying probability distribution. The uncertainties in the input parameters affect the model predictability adversely rendering many of these models inadequate for any decision making, as they lack robustness, which is a measure of the extent to which the forward solvers amplify uncertainties from the input to the output 5 . In general, quantification of parametric input uncertainty is only based on a single given model, hence ignoring the bigger source of uncertainty associated with the model structure.…”

mentioning

confidence: 99%

Identifiability and predictability of integer- and fractional-order epidemiological models using physics-informed neural networks

Kharazmi

Cai

Zheng

et al. 2021

Preprint

View full text Add to dashboard Cite

We analyze a plurality of epidemiological models through the lens of physics-informed neural networks (PINNs) that enable us to identify multiple time-dependent parameters and to discover new data-driven fractional differential operators. In particular, we consider several variations of the classical susceptible-infectious-removed (SIR) model by introducing more compartments and delay in the dynamics described by integer-order, fractional-order, and time-delay models. We report the results for the spread of COVID-19 in New York City, Rhode Island and Michigan states, and Italy, by simultaneously inferring the unknown parameters and the unobserved dynamics. For integer-order and time-delay models, we fit the available data by identifying time-dependent parameters, which are represented by neural networks (NNs). In contrast, for fractional differential models, we fit the data by determining different time-dependent derivative orders for each compartment, which we represent by NNs. We investigate the identifiability of these unknown functions for different datasets, and quantify the uncertainty associated with NNs and with control measures in forecasting the pandemic.

show abstract

“…Since then, several groups of prominent scientists and software engineers have sought to evaluate the reliability of CovidSim 4,5 . In this issue of Nature Computational Science, Edeling et al 6 present a comprehensive examination of CovidSim simulations to quantify the uncertainties in the model and better assess the impact of these uncertainties on the model output (Fig. 1).…”

mentioning

confidence: 99%

“…1 | The sources of uncertainty of COVID-19 pandemic models. Edeling et al6 present a comprehensive examination of CovidSim simulations to quantify the uncertainties from three sources, including parameter uncertainty, model structure uncertainty and scenario uncertainty. The study also compares the CovidSim output with actual data to evaluate the robustness of model predictions.…”

mentioning

confidence: 99%