Examining graph neural networks for crystal structures: limitations and opportunities for capturing periodicity

Abstract

Historically, materials informatics has relied on human-designed descriptors of materials structures. In recent years, graph neural networks (GNNs) have been proposed for learning representations of crystal structures from data end-to-end producing vectorial embeddings that are optimized for downstream prediction tasks. However, a systematic scheme is lacking to analyze and understand the limits of GNNs for capturing crystal structures. In this work, we propose to use human-designed descriptors as a bank of human knowledge to test whether black-box GNNs can capture the knowledge of crystal structures. We find that current state-of-the-art GNNs cannot capture the periodicity of crystal structures well, and we analyze the limitations of the GNN models that result in this failure from three aspects: local expressive power, long-range information, and readout function. We propose an initial solution, hybridizing descriptors with GNNs, to improve the prediction of GNNs for materials properties, especially phonon internal energy and heat capacity with 90% lower errors, and we analyze the mechanisms for the improved prediction. All the analysis can be extended easily to other deep representation learning models, human-designed descriptors, and systems such as molecules and amorphous materials.

Quantum-Accurate Modeling of Ferroelectric Phase Transition in Perovskites from Message-Passing Neural Networks

“…Compared to BPNNs, MPNNs do not require the manual tuning of descriptors, as the representation of an atom's environment (or the receptive field) is directly learned from atom types and positions during the model training process. Moreover, MPNNs have the advantage of the message-passing scheme (see Section ) that distinguishes them from BPNNs.…”

Representations of Materials for Machine Learning

“…While these methods have become widespread for property prediction, graph convolution updates based only on the local neighborhood may limit the sharing of information related to long-range interactions or extensive properties. Gong et al demonstrated that these models can struggle to learn materials properties reliant on periodicity, including characteristics as simple as primitive cell lattice parameters (Figure 3c) [63]. As a result, while graph-based learning is a high-capacity approach, performance can vary substantially by the target use case.…”

“…Various strategies to account for this limitation have been proposed. Gong et al found that if the pooled representation after convolutions was concatenated with human-tuned descriptors, errors could be reduced by 90% for related predictions, including phonon internal energy and heat capacity [63]. Algorithms have attempted to more explicitly account for long-range interactions by modulating convolutions with a mask defined by a local basis of Gaussians and a periodic basis of plane waves [65], employing a unique global pooling scheme that could include additional context such as stoichiometry [66], or constructing additional features from the reciprocal representation of the crystal [67].…”

“…Such models, developed for molecular systems, have since been extended to solid-state materials and shown exceptional performance. Indeed, Chen et al trained an equivariant model to predict phonon density of states and was able to screen for high heat capacity targets [74], tasks identified to be particularly challenging for baseline CGCNN and MEGNet models [63]. Therefore, equivariant representations may offer a more general alternative to the specialized architectures described above.…”

“…R 2 scores presented here were reported by Gong et al for lattice parameter, a, predictions in short and long 1D single carbon chain toy structures. Figure adapted from[63]. (d) Ability of graphical representations to distinguish toy structures.…”