De Novo Drug Design Using Reinforcement Learning with Graph-Based Deep Generative Models

Abstract

Machine learning provides effective computational tools for exploring the chemical space via deep generative models. Here, we propose a new reinforcement learning scheme to fine-tune graph-based deep generative models for de novo molecular design tasks. We show how our computational framework can successfully guide a pretrained generative model toward the generation of molecules with a specific property profile, even when such molecules are not present in the training set and unlikely to be generated by the pretrained model. We explored the following tasks: generating molecules of decreasing/increasing size, increasing drug-likeness, and increasing bioactivity. Using the proposed approach, we achieve a model which generates diverse compounds with predicted DRD2 activity for 95% of sampled molecules, outperforming previously reported methods on this metric.

Link-INVENT: generative linker design with reinforcement learning

“…In this work, we introduced Link-INVENT as an extension to the de novo design platform, REINVENT. 6 Link-INVENT is a recurrent neural network (RNN)-based generative model trained to [7,9], [10,12], and [13,15]. The baseline experiment does not enforce the linker length.…”

“…The curve in (c) shows the average score achieved by the batch of molecules sampled at a given epoch and the upper and lower bounds of the shaded region represent the maximum and minimum scores, respectively. (a) Experiment that fixes physico-chemical properties and tasks Link-INVENT with generating linkers with an effective length within the specified intervals: [4, 6],[7,9],[10,12], and[13,15]. The baseline experiment does not enforce the linker length.…”

A Reinforcement Learning Framework to Discover Natural Flavor Molecules

“…It speeds up the system’s learning. The agent’s remembering policy is presented in Equation (1) [ 29 ]. where J ( θ ) is the agent’s remember policy; α is a scaling factor based on the contribution of the best agent; N is the number of molecules sampled; M is the set of molecules, m , generated in the current agent’s epoch; is the policy for each molecule; is the current agent; is the previous model; is the set of actions chosen to build a molecule in the current agent’s epoch; the “~” means that it is the best so far.…”

“…The molecule’s remember policy is presented in Equation (2) [ 29 ]. where σ is a scaling factor treated as a hyperparameter, which tunes the contribution of the score; is the probability of choosing the sequence of actions in the model ; is the probability of the reference model for the same sequence of actions; is the score for the molecule generated by the actions .…”

The (Re)-Evolution of Quantitative Structure–Activity Relationship (QSAR) Studies Propelled by the Surge of Machine Learning Methods

“…Throughout this transformation, QSAR became a vital component of drug discovery, allowing for the highly efficient, low-cost prediction of activities and properties as well as structure-based virtual screening of potentially active hits from chemical libraries composed of millions of drug candidates. Machine learning is also applied in various other fields, , including retrosynthetic route prediction, , protein and compound design, conformer generation, force-field optimization, , and protein structure prediction . The classical QSAR approach relies on mathematical models to establish a relation between molecular structure embedded through various descriptors (i.e., two-dimensional (2D), fingerprints, graphs, or other mathematical representations) and biological activities, including absorption, distribution, metabolism, excretion, and toxicity (ADMET) profiling, binding free energies, , and kinetic rates for protein–ligand complexes, , derived from a set of molecules of similar topology and functionality.…”