Human-in-the-loop assisted de novo molecular design

Abstract

A de novo molecular design workflow can be used together with technologies such as reinforcement learning to navigate the chemical space. A bottleneck in the workflow that remains to be solved is how to integrate human feedback in the exploration of the chemical space to optimize molecules. A human drug designer still needs to design the goal, expressed as a scoring function for the molecules that captures the designer’s implicit knowledge about the optimization task. Little support for this task exists and, consequently, a chemist usually resorts to iteratively building the objective function of multi-parameter optimization (MPO) in de novo design. We propose a principled approach to use human-in-the-loop machine learning to help the chemist to adapt the MPO scoring function to better match their goal. An advantage is that the method can learn the scoring function directly from the user’s feedback while they browse the output of the molecule generator, instead of the current manual tuning of the scoring function with trial and error. The proposed method uses a probabilistic model that captures the user’s idea and uncertainty about the scoring function, and it uses active learning to interact with the user. We present two case studies for this: In the first use-case, the parameters of an MPO are learned, and in the second use-case a non-parametric component of the scoring function to capture human domain knowledge is developed. The results show the effectiveness of the methods in two simulated example cases with an oracle, achieving significant improvement in less than 200 feedback queries, for the goals of a high QED score and identifying potent molecules for the DRD2 receptor, respectively. We further demonstrate the performance gains with a medicinal chemist interacting with the system. Graphical Abstract

Modeling needs user modeling

“…Instead, if the users had an advanced AI assistant that is able to interactively suggest good directions to explore the chemical landscape, this would make it easier for them to arrive at precise specifications. It has been shown by Sundin et al ( 2022 ) that pure RL-based solutions give unsatisfactory results, and that even simple human-in-the-loop helps. In the following, we will show how advanced user models can improve the assistance in this case.…”

Toward the Uniform of Chemical Theory, Simulation, and Experiments in Metaverse Technology

“…Both experts have directly input essential human knowledge or experience to accelerate AI systems' evolution, making the Metaverse unique. Figure 4b illustrates another example of how Mateverse benefits human-in-the-loop 47,49 research paradigms. The user first initiates the target molecule and sets up the MAOS parameters to commence the experiments, as depicted by the black dashed lines in Figure 4b.…”

“…Much of this work is devoted to digitizing chemical experiments that require scientific interpretation, where black-box-type AIs are far weaker than human experts. However, integrating humans into the loop in a time-continuous system is still very challenging, − even for simulations, let alone experiments. Fortunately, Metaverse technology is relatively friendly and popular, supporting experimental and theoretical experts for scientific exploration inside and showing them real-time chemical reaction experimental data.…”

“…Figure b illustrates another example of how Mateverse benefits human-in-the-loop , research paradigms. The user first initiates the target molecule and sets up the MAOS parameters to commence the experiments, as depicted by the black dashed lines in Figure b.…”

The Harmonic Memory: a Knowledge Graph of harmonic patterns as a trustworthy framework for computational creativity

“…By harnessing recent advancements in machine learning, a variety of systems have already been implemented across several domains. Examples include computational systems for material discovery [9], molecular design [66], and more broadly, for virtual laboratories [36]; but also models for generating textual artefacts [52], images [58,61], and even recipes [62] from a variety of prompts.…”