Scalable deeper graph neural networks for high-performance materials property prediction

Omee, Sadman Sadeed; Louis, Steph-Yves M.; Fu, Nihang; Wei, Lai; Dey, Sourin; Dong, Rongzhi; Li, Qinyang; Hu, Jianjun

doi:10.1016/j.patter.2022.100491

Cited by 46 publications

(51 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This architecture was then used for all models. Interestingly while past works 40 have found the CGCNN can scale up to 25 graph convolutional layers we found that models with more than eight hidden layers suffered from the vanishing gradient problem 41 when training on the relaxed data while training on the augmented data allowed for deeper models. we speculate the models' ability to scale with the number of graph convolutional layers and not the number of hidden layers is a product of the graph convolutional layers containing batch normalization.…”

Section: Trainingmentioning

confidence: 71%

Data-augmentation for graph neural network learning of the relaxed energies of unrelaxed structures

2022

View full text Add to dashboard Cite

Computational materials discovery has grown in utility over the past decade due to advances in computing power and crystal structure prediction algorithms (CSPA). However, the computational cost of the ab initio calculations required by CSPA limits its utility to small unit cells, reducing the compositional and structural space the algorithms can explore. Past studies have bypassed unneeded ab initio calculations by utilizing machine learning to predict the stability of a material. Specifically, graph neural networks trained on large datasets of relaxed structures display high fidelity in predicting formation energy. Unfortunately, the geometries of structures produced by CSPA deviate from the relaxed state, which leads to poor predictions, hindering the model’s ability to filter unstable material. To remedy this behavior, we propose a simple, physically motivated, computationally efficient perturbation technique that augments training data, improving predictions on unrelaxed structures by 66%. Finally, we show how this error reduction can accelerate CSPA.

show abstract

Section: Trainingmentioning

confidence: 71%

Data-augmentation for graph neural network learning of the relaxed energies of unrelaxed structures

2022

View full text Add to dashboard Cite

show abstract

“…To learn the sophisticated structure to property relationship between the crystals and their vibrational frequency, we use our recently developed scalable deeper graph neural networks with a global attention mechanism. 32 Our deeperGATGNN model ( Figure 2 ) is composed of a set of augmented graph attention layers with ResNet style skip connections and differentiable group normalization to achieve complex deep feature extractions. After several such feature transformation steps, a global attention layer is used to aggregate the features at all nodes and a global pooling operator is further used to process the information to generate a latent feature representation for the crystal.…”

Section: Methodsmentioning

confidence: 99%

“…It is composed of several graph convolution layers with differentiable normalization and skip connections plus a global attention layer and final fully connected layers. Reproduced with permission from ref ( 32 ). Copyright 2022 Elsevier (in Patterns ).…”

Section: Methodsmentioning

confidence: 99%

“…One of the major advantages of our deeperGATGNN model for materials property prediction lies in its high scalability and state-of-the-art prediction performance as benchmarked over six data sets. 32 The scalability allows us to train a very deep network with 10 or more graph attention layers to achieve complex feature extraction without the performance degradation that many other graph neural networks suffer due to the oversmoothing issue. Another advantage is that the deeperGATGNN model has demonstrated good performance without the need of computationally expensive hyperparameter tuning.…”

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

See 2 more Smart Citations

Predicting Lattice Vibrational Frequencies Using Deep Graph Neural Networks

et al. 2022

Self Cite

View full text Add to dashboard Cite

Lattice vibrational frequencies are related to many important materials properties such as thermal and electrical conductivity as well as superconductivity. However, computational calculation of vibrational frequencies using density functional theory methods is computationally too demanding for large number of samples in materials screening. Here we propose a deep graph neural network based algorithm for predicting crystal vibrational frequencies from crystal structures. Our algorithm addresses the variable dimension of vibrational frequency spectrum using the zero padding scheme. Benchmark studies on two data sets with 15,000 mixed-structure and 35,552 rhombohedra samples show that the aggregated R 2 scores of the prediction reach 0.554 and 0.724. We also evaluate the structural transferability by predicting the vibration frequencies for 239 individual cubic target structures. The R 2 scores for more than 40% of the targets are greater than 0.8 and can reach as high as 0.98 for the model trained with mixed samples, while the average mean absolute error is 43.69 Thz showing low transferability across structure types. Our work demonstrates the capability of deep graph neural networks to learn to predict lattice vibration frequency when sufficient number of training samples are available.

show abstract

Artificial Intelligence in Material Engineering: A Review on Applications of Artificial Intelligence in Material Engineering

2023

View full text Add to dashboard Cite

The role of artificial intelligence (AI) in material science and engineering (MSE) is becoming increasingly important as AI technology advances. The development of high‐performance computing has made it possible to test deep learning (DL) models with significant parameters, providing an opportunity to overcome the limitation of traditional computational methods, such as density functional theory (DFT), in property prediction. Machine learning (ML)‐based methods are faster and more accurate than DFT‐based methods. Furthermore, the generative adversarial networks (GANs) have facilitated the generation of chemical compositions of inorganic materials without using crystal structure information. These developments have significantly impacted material engineering (ME) and research. Some of the latest developments in AI in ME herein are reviewed. First, the development of AI in the critical areas of ME, such as in material processing, the study of structure and material property, and measuring the performance of materials in various aspects, is discussed. Then, the significant methods of AI and their uses in MSE, such as graph neural network, generative models, transfer of learning, etc. are discussed. The use of AI to analyze the results from existing analytical instruments is also discussed. Finally, AI's advantages, disadvantages, and future in ME are discussed.

show abstract

Scalable deeper graph neural networks for high-performance materials property prediction

Cited by 46 publications

References 48 publications

Data-augmentation for graph neural network learning of the relaxed energies of unrelaxed structures

Data-augmentation for graph neural network learning of the relaxed energies of unrelaxed structures

Predicting Lattice Vibrational Frequencies Using Deep Graph Neural Networks

Artificial Intelligence in Material Engineering: A Review on Applications of Artificial Intelligence in Material Engineering

Contact Info

Product

Resources

About