Deep reinforcement learning approaches for global public health strategies for COVID-19 pandemic

Abstract

Background Unprecedented public health measures have been used during this coronavirus 2019 (COVID-19) pandemic to control the spread of SARS-CoV-2 virus. It is a challenge to implement timely and appropriate public health interventions. Methods and findings Population and COVID-19 epidemiological data between 21st January 2020 to 15th November 2020 from 216 countries and territories were included with the implemented public health interventions. We used deep reinforcement learning, and the algorithm was trained to enable agents to try to find optimal public health strategies that maximized total reward on controlling the spread of COVID-19. The results suggested by the algorithm were analyzed against the actual timing and intensity of lockdown and travel restrictions. Early implementations of the actual lockdown and travel restriction policies, usually at the time of local index case were associated with less burden of COVID-19. In contrast, our agent suggested to initiate at least minimal intensity of lockdown or travel restriction even before or on the day of the index case in each country and territory. In addition, the agent mostly recommended a combination of lockdown and travel restrictions and higher intensity policies than the policies implemented by governments, but did not always encourage rapid full lockdown and full border closures. The limitation of this study was that it was done with incomplete data due to the emerging COVID-19 epidemic, inconsistent testing and reporting. In addition, our research focuses only on population health benefits by controlling the spread of COVID-19 without balancing the negative impacts of economic and social consequences. Interpretation Compared to actual government implementation, our algorithm mostly recommended earlier intensity of lockdown and travel restrictions. Reinforcement learning may be used as a decision support tool for implementation of public health interventions during COVID-19 and future pandemics.

Modeling approaches for early warning and monitoring of pandemic situations as well as decision support

“…• those that retrospectively evaluate the effects of NPIs (26, 27, 31-34, 40, 48, 49, 56, 60, 97, 98), • those that make forecasts on the effects of a specified NPI in the sense of scenario planning (26, 31, 35, 38, 50-55, 58, 96), • and those that develop methods for optimal control policy identification (59,(100)(101)(102)(103)(104).…”

“…To find optimal control policies, offline RL strategies have been proposed by several authors. While Kwak et al ( 100 ) solely relied on deep learning and only focused on health aspects, other studies ( 101 – 104 ) focused on a hybrid modeling strategy incorporating an extended SEIR compartmental model for predicting potential NPI effects. Moreover, the latter studies incorporated the economic costs of NPIs as well.…”

A deep reinforcement learning based decision‐making approach for avoiding crowd situation within the case of Covid'19 pandemic

“…Kwak et al 22 used deep reinforcement learning to allow agents to try to find public health strategies for controlling the spread of COVID‐19. The research focused only on population health benefits without considering the negative impacts of economic and social effects.…”

The effect of the Ontario stay-at-home order on Covid-19 third wave infections including vaccination considerations: An interrupted time series analysis

“…Lockdown is a package of policies such as non-essential business closure [ 8 ], mandatory face mask use in businesses that provide essential services, social and physical distancing, public event and mass gathering bans, restrictions on the number of people in gatherings, school closures replaced by online and remote education, restrictions on international and internal travel, including border closures, temperature screening, and testing of travelers who are permitted to travel only for essential reasons, according to Ontario grey lockdown zone rules [ 9 ]. Kwak et al used a deep reinforcement learning algorithm- an extension of the machine learning method- to find the optimal level of lockdown, which helps to decelerate Covid-19 incidence and death [ 10 ]. After running the algorithm, the agent (intelligent decision-making) suggested earlier implementation of lockdown even before the first wave began.…”