The methodologies to assess the effectiveness of non-pharmaceutical interventions during COVID-19: a systematic review

Abstract

Non-pharmaceutical interventions, such as school closures and stay-at-home orders, have been implemented around the world to control the spread of SARS-CoV-2. Their effectiveness in improving health-related outcomes has been the subject of numerous empirical studies. However, these studies show fairly large variation among methodologies in use, reflecting the absence of an established methodological framework. On the one hand, variation in methodologies may be desirable to assess the robustness of results; on the other hand, a lack of common standards can impede comparability among studies. To establish a comprehensive overview over the methodologies in use, we conducted a systematic review of studies assessing the effectiveness of non-pharmaceutical interventions between January 1, 2020 and January 12, 2021 (n = 248). We identified substantial variation in methodologies with respect to study setting, outcome, intervention, methodological approach, and effectiveness assessment. On this basis, we point to shortcomings of existing studies and make recommendations for the design of future studies.

Effectiveness of Canadian travel restrictions in reducing burden of SARS-CoV-2 variants of concern

“…Globally, evidence relating to travel restrictions’ effectiveness in reducing COVID-19 burden is mixed. Several reviews on the topic highlight a lack of consensus in methodologies for empirically evaluating COVID-19 interventions’ effectiveness, including different definitions or criteria for effectiveness ( 11 , 12 , 23 ). In United Arab Emirates, a decline of transmissions from international sources followed implementation of international travel restrictions in 2020 ( 14 ).…”

“…Travel restrictions are a class of NPI applied to mitigate pandemic burden; they include restricted entry of foreign nationals, flight bans, border entry requirements such as testing or vaccination, and quarantine requirements. Globally, evidence for their effectiveness has been mixed (11)(12)(13)(14)(15). Some evidence suggests countries where VOCs were first detected had diminishing contributions towards international dispersal over time, particularly those with high global connectivity, calling into question targeted travel restriction effectiveness on domestic case burden (16).…”

Policy impact evaluation: A potential use case for longitudinal monitoring of viruses in wastewater at small geographic scales

“…11 The most common raw outcomes analyzed were clinical surveillance reports (e.g., confirmed cases or deaths) and human mobility (e.g., tracking of mobile phones). 11 The most common computed outcomes analyzed were COVID-19 growth rate and effective reproduction number. 11…”

“…11 The most common computed outcomes analyzed were COVID-19 growth rate and effective reproduction number. 11…”

“…11 The most common raw outcomes analyzed were clinical surveillance reports (e.g., confirmed cases or deaths) and human mobility (e.g., tracking of mobile phones). 11 The most common computed outcomes analyzed were COVID-19 growth rate and effective reproduction number. 11 Although clinical surveillance and mobile phone tracking are the most common sources of data used to evaluate NPIs, these data are not without biases and limitations.…”