On the sensitivity of non-pharmaceutical intervention models for SARS-CoV-2 spread estimation

Soltesz, Kristian; Gustafsson, Fredrik; Timpka, Toomas; Jaldén, Joakim; Jidling, Carl; Heimerson, Albin; Schön, Thomas B.; Spreco, Armin; Ekberg, Joakim; Dahlström, Örjan; Carlson, Fredrik Bagge; Jöud, Anna; Bernhardsson, Bo

doi:10.1101/2020.06.10.20127324

Cited by 10 publications

(13 citation statements)

References 10 publications

(28 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The nominal NPI effectuation dates considered in [2], do not suffer from the pathological case of any NPI being effectuated on the same day across all 11 countries, see Figure 1, but the observed sensitivity issues can still be qualitatively understood from the analysis of the regression matrix for the linear regression problem at the heart of estimating the NPI parameters, as performed in [5,10]. The identifiability issues are to some extent acknowledged in [4], by recognizing that "The close spacing of interventions in time [...] mean the individual effects of the other interventions are not identifiable".…”

Section: Discussionmentioning

confidence: 99%

“…As revealed in [5], the model version used in [2] had difficulties fitting the model to data from Sweden, which is the only modelled country in which lockdown had not been effectuated. This was fixed in the code version used in [4] by allowing country-specific variations of not only the lockdown intervention effect but also of "the last intervention to be effectuated in a country".…”

Section: The Role Of the Last Interventionmentioning

confidence: 99%

See 1 more Smart Citation

Sensitivity analysis of the effects of non-pharmaceutical interventions on COVID-19 in Europe

Soltesz

Gustafsson

Timpka

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

The role of non-pharmaceutical interventions (NPIs) on the spread of SARS-CoV-2 has drawn significant attention, both scientific and political. Particularly, an article by the Imperial College COVID-19 Response Team (ICCRT), published online in Nature on June 8, 2020, evaluates the efficiency of 5 NPIs. Based on mortality data up to early May, it concludes that only one of the interventions, lockdown, has been efficient in 10 out of 11 studied European countries. We show, via simulations using the ICCRT model code, that conclusions regarding the effectiveness of individual NPIs are not justified. Our analysis focuses on the 11th country, Sweden, an outlier in that no lockdown was effectuated. The new simulations show that estimated NPI efficiencies across all 11 countries change drastically unless the model is adapted to give the Swedish data special treatment. While stated otherwise in the Nature article, such adaptation has been done in the model code reproducing its results: An ungrounded country-specific parameter said to have been introduced in all 11 countries, is in the code only activated for Sweden. This parameter de facto provides a new NPI category, only present in Sweden, and with an impact comparable to that of a lockdown. While the considered NPIs have unarguably contributed to reduce virus spread, our analysis reveals that their individual efficiency cannot be reliably quantified by the ICCRT model, provided mortality data up to early May.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: The Role Of the Last Interventionmentioning

confidence: 99%

Sensitivity analysis of the effects of non-pharmaceutical interventions on COVID-19 in Europe

Soltesz

Gustafsson

Timpka

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Additionally, under excessive collinearity, and insufficient data to overcome it, individual effectiveness estimates would be highly sensitive to variations in the data and model parameters (15). Indeed, high sensitivity prevented Flaxman et al (1), who had a smaller dataset, from disentangling NPI effects (9). In contrast, our effectiveness estimates are substantially less sensitive (see below).…”

Section: Effectiveness Of Individual Npismentioning

confidence: 92%

“…Second, the data are retrospective and observational, meaning that unobserved factors could confound the results. Third, NPI effectiveness estimates can be highly sensitive to arbitrary modeling decisions, as shown by two recent replication studies (9,10). Fourth, large-scale public NPI datasets suffer from frequent inconsistencies (11) and missing data (12).…”

mentioning

confidence: 99%

Inferring the effectiveness of government interventions against COVID-19

Brauner

Mindermann

Sharma

et al. 2021

Science

893

919

View full text Add to dashboard Cite

Governments are attempting to control the COVID-19 pandemic with nonpharmaceutical interventions (NPIs). However, the effectiveness of different NPIs at reducing transmission is poorly understood. We gathered chronological data on the implementation of NPIs for several European, and other, countries between January and the end of May 2020. We estimate the effectiveness of NPIs, ranging from limiting gathering sizes, business closures, and closure of educational institutions to stay-at-home orders. To do so, we used a Bayesian hierarchical model that links NPI implementation dates to national case and death counts and supported the results with extensive empirical validation. Closing all educational institutions, limiting gatherings to 10 people or less, and closing face-to-face businesses each reduced transmission considerably. The additional effect of stay-at-home orders was comparatively small.

show abstract

“…Second, the data are retrospective and observational, meaning that unobserved factors could confound the results. Third, NPI effectiveness estimates can be highly sensitive to arbitrary modelling decisions, as demonstrated by two recent replication studies 9,10 . Fourth, large-scale public NPI datasets suffer from frequent inconsistencies 11 and missing data 12 .…”

Section: Introductionmentioning

confidence: 99%

The effectiveness of eight nonpharmaceutical interventions against COVID-19 in 41 countries

Brauner

Mindermann

Sharma

et al. 2020

Preprint

176

View full text Add to dashboard Cite

Background: Existing analyses of nonpharmaceutical interventions (NPIs) against COVID19 transmission have concentrated on the joint effectiveness of large-scale NPIs. With increasing data, we can move beyond estimating joint effects towards disentangling individual effects. In addition to effectiveness, policy decisions ought to account for the burden placed by different NPIs on the population. Methods: To our knowledge, this is the largest data-driven study of NPI effectiveness to date. We collected chronological data on 9 NPIs in 41 countries between January and April 2020, using extensive fact-checking to ensure high data quality. We infer NPI effectiveness with a novel semi-mechanistic Bayesian hierarchical model, modelling both confirmed cases and deaths to increase the signal from which NPI effects can be inferred. Finally, we study how much perceived burden different NPIs impose on the population with an online survey of preferences using the MaxDiff method. Results: Eight NPIs have a >95% posterior probability of being effective: closing schools (mean reduction in R: 50%; 95% credible interval: 39%-59%), closing nonessential businesses (34%; 16%-49%), closing high-risk businesses (26%; 8%-42%), and limiting gatherings to 10 people or less (28%; 8%-45%), to 100 people or less (17%; -3%-35%), to 1000 people or less (16%; -2%-31%), issuing stay-at-home orders (14%; -2%-29%), and testing patients with respiratory symptoms (13%; -1%-26%). As validation is crucial for NPI models, we performed 15 sensitivity analyses and evaluated predictions on unseen data, finding strong support for our results. We combine the effectiveness and preference results to estimate effectiveness-to-burden ratios. Conclusions: Our results suggest a surprisingly large role for schools in COVID-19 transmission, a contribution to the ongoing debate about the relevance of asymptomatic carriers in disease spreading. We identify additional interventions with good effectiveness-burden tradeoffs, namely symptomatic testing, closing high-risk businesses, and limiting gathering size. Closing most nonessential businesses and issuing stay-at-home orders impose a high burden while having a limited additional effect.

show abstract

On the sensitivity of non-pharmaceutical intervention models for SARS-CoV-2 spread estimation

Cited by 10 publications

References 10 publications

Sensitivity analysis of the effects of non-pharmaceutical interventions on COVID-19 in Europe

Sensitivity analysis of the effects of non-pharmaceutical interventions on COVID-19 in Europe

Inferring the effectiveness of government interventions against COVID-19

The effectiveness of eight nonpharmaceutical interventions against COVID-19 in 41 countries

Contact Info

Product

Resources

About