MoleGuLAR: Molecule Generation Using Reinforcement Learning with Alternating Rewards

Abstract

The design of new inhibitors for novel targets is a very important problem especially in the current scenario with the world being plagued by COVID-19. Conventional approaches such as highthroughput virtual screening require extensive combing through existing data sets in the hope of finding possible matches. In this study, we propose a computational strategy for de novo generation of molecules with high binding affinities to the specified target and other desirable properties for druglike molecules using reinforcement learning. A deep generative model built using a stack-augmented recurrent neural network initially trained to generate druglike molecules is optimized using reinforcement learning to start generating molecules with desirable properties like LogP, quantitative estimate of drug likeliness, topological polar surface area, and hydration free energy along with the binding affinity. For multiobjective optimization, we have devised a novel strategy in which the property being used to calculate the reward is changed periodically. In comparison to the conventional approach of taking a weighted sum of all rewards, this strategy shows an enhanced ability to generate a significantly higher number of molecules with desirable properties.

Modern machine learning for tackling inverse problems in chemistry: molecular design to realization

“…Several ML architectures have been proposed to generate valid molecules which have RNNs as the core of their generator. [22][23][24][25][26]…”

“…RL frameworks have proved themselves in the application of chemistry related tasks, especially in molecule optimization. 22,31,46,47 They also have shown promise in tasks like reaction and geometry optimization. [48][49][50] Fig.…”

“…A review by David et al 83 discusses and analyses various representations of molecules in great detail. Concisely, the majority of representations used in generative models fall into one of the following categories: discrete string based, 22,47,[73][74][75] continuous vector space, 81 and weighted connected graphs. 31,46,82,84…”

“…The generative model consisted of a stack-augmented recurrent neural network, and QSAR models were used for the predictions. Goel et al 22 proposed MoleGuLAR, another stack augmented RNN based deep generative model which generates molecules with optimized binding affinity to a target. As an example, the change in the distribution of the molecules generated after optimizing for SARS-Cov-2 M pro is shown in Fig.…”

On modeling and utilizing chemical compound information with deep learning technologies: A task-oriented approach

“…In general, because the reward should be calculated from arbitrary molecules which the model generates, simple molecular properties such as logP and QED is often used. A recently published MoleGuLAR [166] used multi-objective scheme for generating drug like molecules with high binding affinity to novel targets along with desired logP: −6.76 kcal/mol mean binding affinity, 2.9 mean logP, 0.42 mean QED for targeting 2.5 logP and 1 QED, respectively. The authors described that their switching reward functions rather than the sum of rewards improved optimization quality because an alternating reward takes the model into the better local chemical space where one property is optimal when optimization for the conflicting property is started.…”