A Methodology Based on Deep Q-Learning/Genetic Algorithms for Optimizing COVID-19 Pandemic Government Actions

Abstract

Whenever countries are threatened by a pandemic, as is the case with the COVID-19 virus, governments need help to take the right actions to safeguard public health as well as to mitigate the negative effects on the economy. A restrictive approach can seriously damage the economy. Conversely, a relaxed one may put at risk a high percentage of the population. Other investigations in this area are focused on modelling the spread of the virus or estimating the impact of the different measures on its propagation. However, in this paper, we propose a new methodology for helping governments in planning the phases to combat the pandemic based on their priorities. To this end, we implement the SEIR epidemiological model to represent the evolution of the COVID-19 virus on the population. To optimize the best sequences of actions governments can take, we propose a methodology with two approaches, one based on Deep Q-Learning and another one based on Genetic Algorithms. The sequences of actions (confinement, self-isolation, two-meter distance or not taking restrictions) are evaluated according to a reward system focused on meeting two objectives: firstly, getting few people infected so that hospitals are not overwhelmed, and secondly, avoiding taking drastic measures which could cause serious damage to the economy. The conducted experiments evaluate our methodology based on the accumulated rewards during the established period. The experiments also prove that it is a valid tool for governments to reduce the negative effects of a pandemic by optimizing the planning of the phases. According to our results, the approach based on Deep Q-Learning outperforms the one based on Genetic Algorithms.

Prediction and control of COVID-19 spreading based on a hybrid intelligent model

“…To solve this problem, the GA and ANN [27][28][29][30][31][32][33][34][35][36] hybrid method is proposed to optimize epidemic dynamics model and predict the COVID-19 spreading. Genetic algorithm is an adaptive global optimization search algorithm formed by simulating the genetic and evolutionary process of biological species in natural environment [37]. Combining the viewpoint of biogenetics and realizing the improvement of individual adaptability through the mechanism of natural selection, heredity, and variation.…”

Dynamic versus Continuous Interventions: Optimizing Lockdown Policies for COVID-19

“…In these cases, the optimizer’s goal is to minimize this cost, and factors such as healthcare costs or impact of lockdowns on the country’s workforce are explicitly quantified. For example, Miralles-Pechuán et al (2020) included health and economical costs, and Olivier et al (2020) included a cumulative economic impact of lockdown in the loss function of its optimizer.…”

“…Furthermore, Khadilkar et al (2020) presented a quantitative way to compute lockdown decisions using RL for individual cities or regions that balances health and economic considerations. Lastly, a report by Miralles-Pechuán et al (2020) proposed Deep Q-Learning and genetic algorithms to optimize the best sequences of actions governments can take to reduce the harmful effects of a pandemic and proved that their methodology is a valid tool to find actions that governments can take to reduce the adverse effects of a pandemic. Despite being a powerful tool when it comes to optimization of episodic optimization, RL presents its own challenges such as reward distribution (dense or sparse), number of episodes, and difficulty of parallel computing (Salimans et al, 2017).…”

“…Their results indicated that the ability to reduce social distancing depends on a 5–10% increase in the current percentage of people strictly following protective guidelines, highlighting the importance of protective behavior in controlling the pandemic. Furthermore, Miralles-Pechuán et al (2020) implemented both RL and GA to determine the sequences of actions (confinement, self-isolation, two-meter distance or not taking restrictions) on a SEIR model according to a reward system focused on meeting two objectives: firstly, getting few people infected so that hospitals are not overwhelmed, and secondly, avoiding taking drastic measures that could cause serious damage to the economy. Interestingly, their approach based on RL outperformed the one based on GA. Another work by Zhang G (2021) combined GA and a long short-term memory (LSTM) neural network to optimize the infection rates on a susceptible-infected-quarantined-recovered (SIQR) epidemic spreading model, achieving good predictive ability on infection and death cases.…”

Prediction and control of COVID-19 infection based on a hybrid intelligent model

“…To solve this problem, the GA and ANN hybrid method is proposed to optimize epidemic dynamics model and predict the COVID-19 spreading [24–27]. Genetic algorithm is an adaptive global optimization search algorithm formed by simulating the genetic and evolutionary process of biological species in the natural environment [28]. It uses the viewpoint of biogenetics and realizes the improvement of individual adaptability through the mechanism of natural selection, heredity and variation.…”