Higher-Order Explanations of Graph Neural Networks via Relevant Walks

Abstract

Graph Neural Networks (GNNs) are a popular approach for predicting graph structured data. As GNNs tightly entangle the input graph into the neural network structure, common explainable AI approaches are not applicable. To a large extent, GNNs have remained black-boxes for the user so far. In this paper, we show that GNNs can in fact be naturally explained using higher-order expansions, i.e. by identifying groups of edges that jointly contribute to the prediction. Practically, we find that such explanations can be extracted using a nested attribution scheme, where existing techniques such as layer-wise relevance propagation (LRP) can be applied at each step. The output is a collection of walks into the input graph that are relevant for the prediction. Our novel explanation method, which we denote by GNN-LRP, is applicable to a broad range of graph neural networks and lets us extract practically relevant insights on sentiment analysis of text data, structure-property relationships in quantum chemistry, and image classification.Index Terms-graph neural networks, higher-order explanations, layer-wise relevance propagation, explainable machine learning. ! INTRODUCTIONMany interesting structures found in scientific and industrial applications can be expressed as graphs. Examples are lattices in fluid modeling, molecular geometry, biological interaction networks, or social / historical networks. Graph neural networks (GNNs) [1], [2] have been proposed as a method to learn from observations in general graph structures and have found use in an ever growing number of applications [3]-[8]. While GNNs make useful predictions, they typically act as black-boxes, and it has neither been directly possible (1) to extract novel insight from the learned model nor (2) to verify that the model has made the intended use of the graph structure, e.g. that it has avoided Clever Hans phenomena [9].Explainable AI (XAI) is an emerging research area that aims to extract interpretable insights from trained ML models [10], [11]. So far, research has focused, for example, on full black-box models [12], [13], self-explainable models [14], [15], or deep neural networks [16], where in all cases, the prediction can be attributed to the input features. For a GNN, however, the graph being received as input is deeply

Toward Explainable AI for Regression Models

“…additivity or attention mechanisms) in the model [16], [92], [14], [59], [87]. Finally, some methods, commonly referred as higher-order methods aim to identify not individual features, but groups of features, that contribute only when occurring jointly [32], [81], [26], [66], [21], [47]. Moreover, methods have been proposed or adjusted to fit specific model architectures (e.g.…”

“…A further distinction can be made between methods that explain based on individual input features [78], [11], [10], [79], combinations of input features (e.g. [16], [66], [26]), or higher-level concepts [40]. While explanation methods are often evaluated based on their technical merit (e.g.…”

“…The LRP method, although most studied on deep ReLU networks, can also be applied or extended to different types of models, e.g. long short-term memory networks (LSTMs) [8], graph neural networks (GNNs) [66], Bayesian Neural networks [15] or non-neural networks, e.g. for anomaly detection [37], [60] or clustering [36].…”

XAI for Transformers: Better Explanations through Conservative Propagation

“…Previous works on explainability in graph neural networks include gradient-based methods Pope et al [2019], graph decomposition Schnake et al [2021] graph perturbations Ying et al [2019], Luo et al [2020], and a local approximation using simpler models Huang et al [2020].…”

“…We now test the performance of different explanation methods using an input perturbation scheme in which the most or least relevant input nodes are considered Schnake et al [2021]. Two different settings are considered for the Graph Transformer experiments.…”