Improving de novo molecular design with curriculum learning

Guo, Jeff; Fialková, Vendy; Arango, Juan Diego; Margreitter, Christian; Janet, Jon Paul; Papadopoulos, Kostas; Engkvist, Ola; Patronov, Atanas

doi:10.1038/s42256-022-00494-4

Cited by 33 publications

(45 citation statements)

References 43 publications

(67 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We acknowledge that alternative methods can be used to improve the sample-efficiency of RL [ 108 ]. For example, experience replay can be used to remind the agent of ‘good’ molecules [ 58 , 108 ], a margin guard [ 109 ] can be employed to dynamically change α during RL updates or curriculum learning can be used to accelerate learning by breaking the objective into a sequence of simpler tasks [ 110 ]. We are of the opinion that AHC is a more direct and principled approach to improve sample-efficiency and could even be used in combination with these methods to potentially further improve reinforcement learning for de novo molecule optimization.…”

Section: Resultsmentioning

confidence: 99%

Augmented Hill-Climb increases reinforcement learning efficiency for language-based de novo molecule generation

et al. 2022

View full text Add to dashboard Cite

A plethora of AI-based techniques now exists to conduct de novo molecule generation that can devise molecules conditioned towards a particular endpoint in the context of drug design. One popular approach is using reinforcement learning to update a recurrent neural network or language-based de novo molecule generator. However, reinforcement learning can be inefficient, sometimes requiring up to 105 molecules to be sampled to optimize more complex objectives, which poses a limitation when using computationally expensive scoring functions like docking or computer-aided synthesis planning models. In this work, we propose a reinforcement learning strategy called Augmented Hill-Climb based on a simple, hypothesis-driven hybrid between REINVENT and Hill-Climb that improves sample-efficiency by addressing the limitations of both currently used strategies. We compare its ability to optimize several docking tasks with REINVENT and benchmark this strategy against other commonly used reinforcement learning strategies including REINFORCE, REINVENT (version 1 and 2), Hill-Climb and best agent reminder. We find that optimization ability is improved ~ 1.5-fold and sample-efficiency is improved ~ 45-fold compared to REINVENT while still delivering appealing chemistry as output. Diversity filters were used, and their parameters were tuned to overcome observed failure modes that take advantage of certain diversity filter configurations. We find that Augmented Hill-Climb outperforms the other reinforcement learning strategies used on six tasks, especially in the early stages of training or for more difficult objectives. Lastly, we show improved performance not only on recurrent neural networks but also on a reinforcement learning stabilized transformer architecture. Overall, we show that Augmented Hill-Climb improves sample-efficiency for language-based de novo molecule generation conditioning via reinforcement learning, compared to the current state-of-the-art. This makes more computationally expensive scoring functions, such as docking, more accessible on a relevant timescale.

show abstract

Section: Resultsmentioning

confidence: 99%

Augmented Hill-Climb increases reinforcement learning efficiency for language-based de novo molecule generation

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Baselines. We used three well-established models, consisting of RationaleRL, 85 REINVENT, 91 and GB-GA, which exhibit impressive performance on the multi-constraints molecular design task, as the baselines in this study.…”

Section: ■ Methodsmentioning

confidence: 99%

ChemistGA: A Chemical Synthesizable Accessible Molecular Generation Algorithm for Real-World Drug Discovery

Wang

Sun

et al. 2022

J. Med. Chem.

View full text Add to dashboard Cite

Many deep learning (DL)-based molecular generative models have been proposed to design novel molecules. These models may perform well on benchmarks, but they usually do not take real-world constraints into account, such as available training data set, synthetic accessibility, and scaffold diversity in drug discovery. In this study, a new algorithm, ChemistGA, was proposed by combining the traditional heuristic algorithm with DL, in which the crossover of the traditional genetic algorithm (GA) was redefined by DL in conjunction with GA, and an innovative backcrossing operation was implemented to generate desired molecules. Our results clearly show that ChemistGA not only retains the strength of the traditional GA but also greatly enhances the synthetic accessibility and success rate of the generated molecules with desired properties. Calculations on the two benchmarks illustrate that ChemistGA achieves impressive performance among the state-of-the-art baselines, and it opens a new avenue for the application of generative models to real-world drug discovery scenarios.

show abstract

“…Generally, the physical and chemical properties and biological activity are used as constraints of the molecular generation models 9,10 to guide molecular design through reinforcement learning, 11,12 transfer learning, 13 or curriculum learning. 14 To obtain molecules with biological activity, new generated molecules are further evaluated and selected generally through molecular docking software such as Schrodinger, 15 AutoDock Vina, 16 etc. 17 However, such approaches merely perform the docking simulation after the molecular generation model 18 but do not optimize the binding activity to a target in the process of molecular generation.…”

Section: ■ Introductionmentioning

confidence: 99%

Deep Generation Model Guided by the Docking Score for Active Molecular Design

Yang

Hsieh

Kang

et al. 2023

J. Chem. Inf. Model.

View full text Add to dashboard Cite

A deep generation model, as a novel drug design and discovery tool, shows obvious advantages in generating compounds with novel backbones and has been applied successfully in the field of drug discovery. However, it is still a challenge to generate molecules with expected properties, especially high activity. Here, to obtain compounds both with novelty and high activity to a target, we proposed a conditional molecular generation model COMG by considering the docking score and 3D pharmacophore matching during molecular generation. The proposed model was based on the conditional variational autoencoder architecture constrained by the pharmacophore matching score. During Bayesian optimization, the docking score was applied to enhance the target relevance of generated compounds. Furthermore, to overcome the problem of high structural similarity caused by Bayesian optimization, the idea of the scaffold memory unit was also introduced. The evaluation results of COMG show that our model not only can improve the structural diversity of generated molecules but also can effectively improve the proportion of target-related drug-active molecules. The obtained results indicate that our proposed model COMG is a useful drug design tool.

show abstract

Improving de novo molecular design with curriculum learning

Cited by 33 publications

References 43 publications

Augmented Hill-Climb increases reinforcement learning efficiency for language-based de novo molecule generation

Augmented Hill-Climb increases reinforcement learning efficiency for language-based de novo molecule generation

ChemistGA: A Chemical Synthesizable Accessible Molecular Generation Algorithm for Real-World Drug Discovery

Deep Generation Model Guided by the Docking Score for Active Molecular Design

Contact Info

Product

Resources

About