Improved Chemical Prediction from Scarce Data Sets via Latent Space Enrichment

Abstract

Modern machine learning provides promising methods for accelerating the discovery and characterization of novel chemical species. However, in many areas experimental data remain costly and scarce, and computational models are unavailable for targeted figures of merit. Here we report a promising pathway to address this challenge by using chemical latent space enrichment, whereby disparate data sources are combined in joint prediction tasks to enable improved prediction in data-scarce applications. The approach is demonstrated for pK a prediction of moderately sized molecular species using a combination of experimentally available pK a data and density functional theory-based characterizations of the (de)protonation free energy. A novel autoencoder framework is used to create a continuous chemical latent space that is then used in single and joint training tasks for property prediction. By combining these two data sets in a jointly trained autoencoder framework, we observe mutual improvement in property prediction tasks in the scarce data limit. We also demonstrate an enrichment mechanism that is unique to latent space training, whereby training on excess computational data can mitigate the prediction losses associated with scarce experimental data and advantageously organize the latent space. These results demonstrate that disparate chemical data sources can be advantageously combined in an autoencoder framework with potential general application to data-scarce chemical learning tasks.

A quantitative uncertainty metric controls error in neural network-driven chemical discovery

“…Thus, for multi-layer ANNs, feature-based proximity can be very different from the intrinsic relationship between points in the model. Such ideas have been explored in generative modeling where distances in auto-encoded latent representations have informed chemical diversity55,56 and in anomaly detection with separate models57,58 ( e.g. , autoencoders59–61 or nearest-neighbor classifiers62,63) have enabled identification of ‘poisoned’ input data 64.…”

Targeted sequence design within the coarse-grained polymer genome

“…For example, synthetic protocols have been optimized via the training of ML models on experimental reaction databases (USPTO, Reaxsys, and SciFinder) ( 6 ), while generative design strategies have enabled targeted small-molecule design ( 7 ). However, materials science often presents problems where substantially less data are available, thereby necessitating the development of creative approaches for navigating data-scarce regimes ( 8 , 9 ).…”

Locally Optimizable Joint Embedding Framework to Design Nitrogen‐rich Molecules that are Similar but Improved

“…It has been shown capable of learning underlying relationships from a diverse set of molecular data by letting multiple data tasks and domains interact adaptively while generating the joint embeddings. Joint training is also a type of joint embedding that has been successfully applied to improve deep learning‐based molecule generation and enable transfer learning [25–29] . Joint training incorporates a property prediction task into a variational autoencoder (VAE) [30] and has been shown to organize points in the VAE latent space, making the latent space amenable to inverse molecular design and optimization [25,29] .…”

“…jointly trained a VAE using drug likeliness and synthetic accessibility, then performed Bayesian optimization in the resulting latent space to identify novel drug‐like molecules [25] . In addition to latent space organization, joint training provides a platform for knowledge transfer between abundant and scarce data tasks [27,29] . When more than one property is used to develop the model, this constitutes a multitask transfer learning approach.…”

“…When more than one property is used to develop the model, this constitutes a multitask transfer learning approach. Iovanac and Savoie employed multitask joint training in which a data abundant property prediction task for (de)protonation energy, was combined with the target task predicting molecule acidity, [27] for which data are scarce. It therefore has been reasoned that joint embedding yields more expressive models [31] in part because of the likely organization they induce in the latent representation [25,28] .…”