We present a framework, which we call Molecule Deep
Q
-Networks (MolDQN), for molecule optimization by combining domain knowledge of chemistry and state-of-the-art reinforcement learning techniques (double
Q
-learning and randomized value functions). We directly define modifications on molecules, thereby ensuring 100% chemical validity. Further, we operate without pre-training on any dataset to avoid possible bias from the choice of that set. MolDQN achieves comparable or better performance against several other recently published algorithms for benchmark molecular optimization tasks. However, we also argue that many of these tasks are not representative of real optimization problems in drug discovery. Inspired by problems faced during medicinal chemistry lead optimization, we extend our model with multi-objective reinforcement learning, which maximizes drug-likeness while maintaining similarity to the original molecule. We further show the path through chemical space to achieve optimization for a molecule to understand how the model works.
Deep reinforcement
learning was employed to optimize chemical reactions. Our model iteratively
records the results of a chemical reaction and chooses new experimental
conditions to improve the reaction outcome. This model outperformed
a state-of-the-art blackbox optimization algorithm by using 71% fewer
steps on both simulations and real reactions. Furthermore, we introduced
an efficient exploration strategy by drawing the reaction conditions
from certain probability distributions, which resulted in an improvement
on regret from 0.062 to 0.039 compared with a deterministic policy.
Combining the efficient exploration policy with accelerated microdroplet
reactions, optimal reaction conditions were determined in 30 min for
the four reactions considered, and a better understanding of the factors
that control microdroplet reactions was reached. Moreover, our model
showed a better performance after training on reactions with similar
or even dissimilar underlying mechanisms, which demonstrates its learning
ability.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.