A quantitative uncertainty metric controls error in neural network-driven chemical discovery

Janet, Jon Paul; Duan, Chenru; Yang, Tzuhsiung; Nandy, Aditya; Kulik, Heather J.

doi:10.1039/c9sc02298h

Cited by 162 publications

(214 citation statements)

References 78 publications

Supporting

Mentioning

194

Contrasting

Order By: Relevance

“…In prior work, we 78 evaluated the ability of an ANN to predict the HS−LS adiabatic spin splitting, ΔE H-L , of this same complex. We had observed 78 that the ΔE H-L ANN strongly overstabilized (ΔE H-L = −34.7 kcal/mol) the HS state with respect to ΔE H-L from hybrid (i.e., B3LYP 86−88 ) DFT. We rationalized this poor ΔE H-L ANN performance by the significant dissimilarity of the CSD complex to available training data.…”

Section: Resultsmentioning

confidence: 99%

Seeing is Believing: Experimental Spin States from Machine Learning Model Structure Predictions

Taylor¹,

Yang²,

Lin³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

<p>Determination of ground-state spins of open-shell transition metal complexes is critical to understanding catalytic and materials properties but also challenging with approximate electronic structure methods. As an alternative approach, we demonstrate how structure alone can be used to guide assignment of ground-state spin from experimentally determined crystal structures of transition metal complexes. We first identify the limits of distance-based heuristics from distributions of metal–ligand bond lengths of over 2,000 unique mononuclear Fe(II)/Fe(III) transition metal complexes. To overcome these limits, we employ artificial neural networks (ANNs) to predict spin-state-dependent metal–ligand bond lengths and classify experimental ground state spins based on agreement of experimental structures with the ANN predictions. Although the ANN is trained on hybrid density functional theory data, we exploit the method-insensitivity of geometric properties to enable assignment of ground states for the majority (ca. 80-90%) of structures. We demonstrate the utility of the ANN by data-mining the literature for spin-crossover (SCO) complexes, which have experimentally-observed temperature-dependent geometric structure changes, by correctly assigning almost all (> 95%) spin states in the 46 Fe(II) SCO complex set. This approach represents a promising complement to more conventional energy-based spin-state assignment from electronic structure theory at the low cost of a machine learning model. </p>

show abstract

Section: Resultsmentioning

confidence: 99%

Seeing is Believing: Experimental Spin States from Machine Learning Model Structure Predictions

Taylor¹,

Yang²,

Lin³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…There has been much progress in the first use-case through the development of quantitative structure activity relationship (QSAR) models using deep learning [Ma et al, 2015]. These models have achieved state-of-the-art results in predicting properties Ryu et al [2018a,b], Turcani et al [2018], Dey et al [2018], , Gu et al [2019], Zeng et al [2018], Coley et al [2019a] as well as property uncertainties Cortés-Ciriano and Bender [2018], Zhang and Lee [2019], Janet et al [2019], Ryu et al [2019] of known molecules.…”

Section: Introductionmentioning

confidence: 99%

Constrained Bayesian optimization for automatic chemical design using variational autoencoders

2020

View full text Add to dashboard Cite

Automatic Chemical Design is a framework for generating novel molecules with optimized properties. The original scheme, featuring Bayesian optimization over the latent space of a variational autoencoder, suffers from the pathology that it tends to produce invalid molecular structures. First, we demonstrate empirically that this pathology arises when the Bayesian optimization scheme queries latent points far away from the data on which the variational autoencoder has been trained. Secondly, by reformulating the search procedure as a constrained Bayesian optimization problem, we show that the effects of this pathology can be mitigated, yielding marked improvements in the validity of the generated molecules. We posit that constrained Bayesian optimization is a good approach for solving this class of training set mismatch in many generative tasks involving Bayesian optimization over the latent space of a variational autoencoder.

show abstract

“…We have seen few 15,16 comparisons of different methods for UQ within the field of catalysis and materials informatics. Here we examine a protocol 17,18 for comparing the performance of different modeling and UQ methods ( Figure 1).…”

Section: Introductionmentioning

confidence: 99%

Methods for comparing uncertainty quantifications for material property predictions

Tran

Neiswanger

Yoon

et al. 2020

Mach. Learn.: Sci. Technol.

132

154

View full text Add to dashboard Cite

Data science and informatics tools have been proliferating recently within the computational materials science and catalysis fields. This proliferation has spurned the creation of various frameworks for automated materials screening, discovery, and design. Underpinning these frameworks are surrogate models with uncertainty estimates on their predictions. These uncertainty estimates are instrumental for determining which materials to screen next, but the computational catalysis field does not yet have a standard procedure for judging the quality of such uncertainty estimates. Here we present a suite of figures and performance metrics derived from the machine learning community that can be used to judge the quality of such uncertainty estimates.This suite probes the accuracy, calibration, and sharpness of a model quantitatively.We then show a case study where we judge various methods for predicting densityfunctional-theory-calculated adsorption energies. Of the methods studied here, we find that the best performer is a model where a convolutional neural network is used 1 arXiv:1912.10066v2 [cond-mat.mtrl-sci] to supply features to a Gaussian process regressor, which then makes predictions of adsorption energies along with corresponding uncertainty estimates.

show abstract

A quantitative uncertainty metric controls error in neural network-driven chemical discovery

Abstract: A predictive approach for driving down machine learning model errors is introduced and demonstrated across discovery for inorganic and organic chemistry.

Cited by 162 publications

References 78 publications

Seeing is Believing: Experimental Spin States from Machine Learning Model Structure Predictions

Seeing is Believing: Experimental Spin States from Machine Learning Model Structure Predictions

Constrained Bayesian optimization for automatic chemical design using variational autoencoders

Methods for comparing uncertainty quantifications for material property predictions

Contact Info

Product

Resources

About