Compositionally restricted attention-based network for materials property predictions

Abstract

In this paper, we demonstrate an application of the Transformer self-attention mechanism in the context of materials science. Our network, the Compositionally Restricted Attention-Based network (), explores the area of structure-agnostic materials property predictions when only a chemical formula is provided. Our results show that ’s performance matches or exceeds current best-practice methods on nearly all of 28 total benchmark datasets. We also demonstrate how ’s architecture lends itself towards model interpretability by showing different visualization approaches that are made possible by its design. We feel confident that and its attention-based framework will be of keen interest to future materials informatics researchers.

DiSCoVeR: a materials discovery screening tool for high performance, unique chemical compositions

“…CrabNet [40] Composition-based property regression Predict performance for proxy scores ElMD [54] Composition-based distance metric Supply distance matrix to DensMAP DensMAP [59] Density-aware dimensionality reduction Obtain densities for density proxy HDBSCAN* [57] Density…”

“…diamond vs. carbon). We use CrabNet [40] as the regression model for bulk modulus which depends only on composition to generate machine learning features; however, a different composition-based model mentioned in Section 1 could have been used instead.…”

DiSCoVeR: a Materials Discovery Screening Tool for High Performance, Unique Chemical Compositions

“…The highest bulk modulus is chosen when considering identical formulae. We use CrabNet [44] as the regression model for bulk modulus which depends only on composition to generate machine learning features; however, one of the other models mentioned in Section 1 could have been used instead.…”

DiSCoVeR: a Materials Discovery Screening Tool for High Performance, Unique Chemical Compositions

“…We use CrabNet [44] as the regression model for bulk modulus which depends only on composition to generate machine learning features; however, one of the other models mentioned in Section 1 could have been used instead.…”

“…A suite of regression models are available for use as the backbone for a mate-rials discovery project. A non-exhaustive list ordered from oldest to newest by journal publication year includes GBM-Locfit [36], CGCNN [37], MEGNet [38], wren [39], GATGNN [40], iCGCNN [27], Automatminer [41], Roost [42], DimeNet++ [43], Compositionally-Restricted Attention-Based Network (CrabNet) [44], and MODNet [45], each with varying advantages and disadvantages.…”