SpookyNet: Learning force fields with electronic degrees of freedom and nonlocal effects

Abstract

Machine-learned force fields combine the accuracy of ab initio methods with the efficiency of conventional force fields. However, current machine-learned force fields typically ignore electronic degrees of freedom, such as the total charge or spin state, and assume chemical locality, which is problematic when molecules have inconsistent electronic states, or when nonlocal effects play a significant role. This work introduces SpookyNet, a deep neural network for constructing machine-learned force fields with explicit treatment of electronic degrees of freedom and nonlocality, modeled via self-attention in a transformer architecture. Chemically meaningful inductive biases and analytical corrections built into the network architecture allow it to properly model physical limits. SpookyNet improves upon the current state-of-the-art (or achieves similar performance) on popular quantum chemistry data sets. Notably, it is able to generalize across chemical and conformational space and can leverage the learned chemical insights, e.g. by predicting unknown spin states, thus helping to close a further important remaining gap for today’s machine learning models in quantum chemistry.

Equivariant Graph Attention Networks for Molecular Property Prediction

“…Our proposed Equivariant Graph Attention Networks (EQ-GAT) operates in 3D space and implements the message passing for each target node i on its local neighbourhood N (i) as defined in Section 2 to avoid the quadratic complexity of the vanilla self-attention when one target node would interact with all other nodes in the point cloud. We emphasize that the integration of local neighbourhoods manifests as a powerful inductive bias and in a bio-chemistry context, coincides with the assumption that a large part of energy variations can be attributed to local interactions, although the influence and importance of non-local effects in machine-learned force-fields has been recently analyzed in (Unke et al, 2021).…”

Neural Network Potentials: A Concise Overview of Methods

“…An example is the inclusion of long-range electrostatic interactions in third and fourthgeneration NNPs. While most fourth-generation NNPs employ predefined descriptors, also novel types of message passing methods are just emerging aiming to describe non-local effects 97,98,126 . Apart from electrostatics, also dispersion interactions, which are weak but can be important in large systems, have been included beyond the local atomic environments in NNPs 25,106 .…”

Toward Explainable AI for Regression Models

“…While explanation methods are often evaluated based on their technical merit (e.g. accuracy, runtime) [63], an increasingly relevant question is whether these explanations enable the human to truly understand the model at hand (also known as causability [35]) and whether these explanations can be turned into meaningful insights and decisions [25], [58], [12], [82]. This contribution will focus on single-instance (local), attribution-based explanations that assign a share of the model output f (x) to the individual features of the respective input sample x.…”

“…This allows for increased confidence in the model results through sanity-checks by an expert as well as new insights into physical phenomena previously not understood (e.g. [69], [38], [82]). XAI methods, especially XAIR methods, can be a key for both which we will demonstrate in the following sections by applying our proposed retraining approach in the quantum chemistry domain.…”