Molecular Design Method Using a Reversible Tree Representation of Chemical Compounds and Deep Reinforcement Learning

Abstract

Automatic design of molecules with specific chemical and biochemical properties is an important process in material informatics and computational drug discovery. In this study, we designed a novel coarse-grained tree representation of molecules (Reversible Junction Tree; “RJT”) for the aforementioned purposes, which is reversely convertible to the original molecule without external information. By leveraging this representation, we further formulated the molecular design and optimization problem as a tree-structure construction using deep reinforcement learning (“RJT-RL”). In this method, all of the intermediate and final states of reinforcement learning are convertible to valid molecules, which could efficiently guide the optimization process in simple benchmark tasks. We further examined the multiobjective optimization and fine-tuning of the reinforcement learning models using RJT-RL, demonstrating the applicability of our method to more realistic tasks in drug discovery.

Tree-Invent: A novel molecular generative model constrained with topological tree

“…De novo molecular design has been proposed as a way to accelerate the drug discovery process. Although successes have been achieved for various de novo molecular design methods 2 , efficiently exploring drug-like chemical space is still quite challenging owing to the discrete nature of its representation and the gigantic scale 3 .…”

“…2 Alternatively, models using fragment-based action space were also proposed, in which functional groups were used as building blocks for structure growing. For example, Jin et al proposed a novel coarse-grained molecule representation by leveraging junction tree (JT) decomposition to a molecule graph; 12 Ishitani et al proposed an RJT-RL methods based on a reversible junction tree representation; 3 Poelking et al proposed a hierarchical model LIBPQR with multi-level self-contrastive learning to improve bias control and data efficiency. 13 Among these methods, a set of primary chemical group is pre-defined to form the action space, this type of model can, at some extent, improve the structure validity, while, as a drawback, restraining the search space of generative model.…”

Tree-Invent: A novel molecular generative model constrained with topological tree

“…[2] Alternatively, models using fragment-based action space were also proposed, in which functional groups were used as building blocks for structure growing. For example, Jin et al proposed a novel coarse-grained molecule representation by leveraging junction tree (JT) decomposition to a molecule graph; [12] Ishitani et al proposed an RJT-RL methods based on a reversible junction tree representation; [3] Poelking et al proposed a hierarchical model LIBPQR with multi-level self-contrastive learning to improve bias control and data efficiency. [13] Among these methods, a set of primary chemical group is pre-defined to form the action space, this type of model can, at some extent, improve the structure validity, while, as a drawback, restraining the search space of generative model.…”

“…Reinforcement learning (RL) is another way to optimize structure generation with regard to certain molecular properties, in which the molecule generation problem is formulated as a Markov decision process wherein an agent (neural network) learns the optimal policy based on the rewards offered by its surrounding environments. The RL has been widely used in both SMILES-based and graph-based generative models, for example, the REINVENT [19], GCPN [20], MolDQN, [21] RL-Graph-INVENT [3] models etc.…”

Tree-Invent: A Novel Multipurpose Molecular Generative Model Constrained with a Topological Tree

“…2 Alternatively, models using fragment-based action space were also proposed in which functional groups were used as building blocks for structure growing. For example, Jin et al proposed a novel coarse-grained molecule representation by leveraging junction tree (JT) decomposition to a molecule graph; 16 Ishitani et al proposed an RJT-RL method based on a reversible junction tree representation; 3 Poelking et al proposed a hierarchical model LIBPQR with multilevel selfcontrastive learning to improve bias control and data efficiency. 17 Among these methods, a set of primary chemical groups is predefined to form the action space, and this type of model can, at some extent, improve the structure validity, while, as a drawback, restraining the search space of the generative model.…”