Discovering Invariant Rationales for Graph Neural Networks

Wu, Yingxin; Wang, Xiang; Zhang, An; He, Xiangnan; Chua, Tat-Seng

doi:10.48550/arxiv.2201.12872

Cited by 9 publications

(21 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This illustrates that subgraphs with a middle size are the very informative substructures to reveal the biological or chemical properties of small molecules. This finding persistently accords with the fact that motifs such as functional groups play a key part in determining molecular attributes [99,86,92]. While for Hamiltonian dynamic systems, J (m) D is majorly intense for low-order and middle-order interactions ( m n ≤ 0.6).…”

Section: For Molecular Property Prediction J (M)supporting

confidence: 65%

“…For example, Cranmer et al [15] perform symbolic regression to components of well-trained GNNs and extract compact closed-form analytical expressions. A more mainstream line is to recognize an informative yet compressed subgraph from the original graph [99,86,87,92]. Identification of those subgraphs promotes GNNs to audit their inner workings and justify their predictions [92], which can shed light on meaningful scientific tasks like protein structure prediction [75].…”

Section: Related Workmentioning

confidence: 99%

“…A more mainstream line is to recognize an informative yet compressed subgraph from the original graph [99,86,87,92]. Identification of those subgraphs promotes GNNs to audit their inner workings and justify their predictions [92], which can shed light on meaningful scientific tasks like protein structure prediction [75]. Our work devises a contextual-aware method to comprehensively understand what complexity of interactions are the most influential and best interpret the decisions of GNNs.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Discovering and Explaining the Representation Bottleneck of Graph Neural Networks from Multi-order Interactions

Wu¹,

Liu²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Most graph neural networks (GNNs) rely on the message passing paradigm to propagate node features and build interactions. Recent works point out that different graph learning tasks require different ranges of interactions between nodes. To investigate the underlying mechanism, we explore the capacity of GNNs to capture pairwise interactions between nodes under contexts with different complexities, especially for their graph-level and node-level applications in scientific domains like biochemistry and physics. When formulating pairwise interactions, we study two standard graph construction methods in scientific domains, i.e., K-nearest neighbor (KNN) graphs and fully-connected (FC) graphs. Furthermore, we demonstrate that the inductive bias introduced by KNN-graphs and FC-graphs inhibits GNNs from learning the most informative order of interactions. Such a phenomenon is broadly shared by several GNNs for different graph learning tasks and prevents GNNs from reaching the global minimum loss, so we name it a representation bottleneck. To overcome that, we propose a novel graph rewiring approach based on the pairwise interaction strengths to adjust the reception fields of each node dynamically. Extensive experiments in molecular property prediction and dynamic system forecast prove the superiority of our method over state-of-the-art GNN baselines. Besides, this paper provides a reasonable explanation of why subgraphs play a vital role in determining graph properties. The code is available at https: //github.com/smiles724/bottleneck.

show abstract

Section: For Molecular Property Prediction J (M)supporting

confidence: 65%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Discovering and Explaining the Representation Bottleneck of Graph Neural Networks from Multi-order Interactions

Wu¹,

Liu²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Specifically, inspired by [30], we design three types of base graphs, i.e., Tree, Ladder, Wheel, and three types of motifs, i.e., Cycle, House, and Grid. With a mix ratio γ, motifs are preferably attached to base graphs.…”

Section: B More Experiments B1 Ability In Avoiding Spurious Explanationsmentioning

confidence: 99%

On Consistency in Graph Neural Network Interpretation

Zhao¹,

Luo²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Uncovering rationales behind predictions of graph neural networks (GNNs) has received increasing attention over recent years. Instance-level GNN explanation aims to discover critical input elements, like nodes or edges, that the target GNN relies upon for making predictions. These identified sub-structures can provide interpretations of GNN's behavior. Though various algorithms are proposed, most of them formalize this task by searching the minimal subgraph which can preserve original predictions. An inductive bias is deep-rooted in this framework: the same output cannot guarantee that two inputs are processed under the same rationale. Consequently, they have the danger of providing spurious explanations and fail to provide consistent explanations. Applying them to explain weaklyperformed GNNs would further amplify these issues. To address the issues, we propose to obtain more faithful and consistent explanations of GNNs. After a close examination on predictions of GNNs from the causality perspective, we attribute spurious explanations to two typical reasons: confounding effect of latent variables like distribution shift, and causal factors distinct from the original input. Motivated by the observation that both confounding effects and diverse causal rationales are encoded in internal representations, we propose a simple yet effective countermeasure by aligning embeddings. This new objective can be incorporated into existing GNN explanation algorithms with no effort. We implement both a simplified version based on absolute distance and a distribution-aware version based on anchors. Experiments on 5 datasets validate its effectiveness, and theoretical analysis shows that it is in effect optimizing a more faithful explanation objective in design, which further justifies the proposed approach.Preprint. Under review.

show abstract

“…Graph-SST2 (Yuan et al 2020) and Mol-BACE (Hu et al 2020) are the benchmark in-domain (ID) datasets. To evaluate the generalization ability of GNN models, we build out-of-domain (OOD) datasets by following the principle of DIR (Wu et al 2022). Specifically, Spurious-Motif is an artificially generated dataset.…”

Section: Introductionmentioning

confidence: 99%

Robust Causal Graph Representation Learning against Confounding Effects

Gao¹,

Li²,

Qiang³

et al. 2022

Preprint

View full text Add to dashboard Cite

The prevailing graph neural network models have achieved significant progress in graph representation learning. However, in this paper, we uncover an ever-overlooked phenomenon: the pre-trained graph representation learning model tested with full graphs underperforms the model tested with well-pruned graphs. This observation reveals that there exist confounders in graphs, which may interfere with the model learning semantic information, and current graph representation learning methods have not eliminated their influence. To tackle this issue, we propose Robust Causal Graph Representation Learning (RCGRL) to learn robust graph representations against confounding effects. RCGRL introduces an active approach to generate instrumental variables under unconditional moment restrictions, which empowers the graph representation learning model to eliminate confounders, thereby capturing discriminative information that is causally related to downstream predictions. We offer theorems and proofs to guarantee the theoretical effectiveness of the proposed approach. Empirically, we conduct extensive experiments on a synthetic dataset and multiple benchmark datasets. The results demonstrate that compared with stateof-the-art methods, RCGRL achieves better prediction performance and generalization ability.

show abstract

Discovering Invariant Rationales for Graph Neural Networks

Cited by 9 publications

References 11 publications

Discovering and Explaining the Representation Bottleneck of Graph Neural Networks from Multi-order Interactions

Discovering and Explaining the Representation Bottleneck of Graph Neural Networks from Multi-order Interactions

On Consistency in Graph Neural Network Interpretation

Robust Causal Graph Representation Learning against Confounding Effects

Contact Info

Product

Resources

About