Robust Unsupervised Graph Representation Learning via Mutual Information Maximization

Abstract

Recent studies have shown that GNNs are vulnerable to adversarial attack. Thus, many approaches are proposed to improve the robustness of GNNs against adversarial attacks. Nevertheless, most of these methods measure the model robustness based on label information and thus become infeasible when labels information is not available. Therefore, this paper focuses on robust unsupervised graph representation learning. In particular, to quantify the robustness of GNNs without label information, we propose a robustness measure, named graph representation robustness (GRR), to evaluate the mutual information between adversarially perturbed node representations and the original graph. There are mainly two challenges to estimate GRR: 1) mutual information estimation upon adversarially attacked graphs; 2) high complexity of adversarial attack to perturb node features and graph structure jointly in the training procedure. To tackle these problems, we further propose an effective mutual information estimator with subgraph-level summary and an efficient adversarial training strategy with only feature perturbations. Moreover, we theoretically establish a connection between our proposed GRR measure and the robustness of downstream classifiers, which reveals that GRR can provide a lower bound to the adversarial risk of downstream classifiers. Extensive experiments over several benchmarks demonstrate the effectiveness and superiority of our proposed method.

A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability

“…In contrastive learning, maximizing the representation consistency between the original graphs and the augmented views of edge perturbation [36,218] can also result in a more robust model. Some adversarial graph contrastive learning and variants [56,67,187,210] are developed to further improve the robustness by introducing an adversarial view of graphs.…”

“…Generally, adversarial training simultaneously generate adversarial samples that can fool a classifier and force the classifier to give similar predictions for a clean sample and its perturbed version so as to improve the robustness of the classifier. Adversarial training [38,41,55,187,211] is also investigated to defend against graph adversarial attacks, which can be generally formulated as the following min max game:…”

Low-rank tensor approximation with local structure for multi-view intrinsic subspace clustering

“…Zhao et al [29] maximized the mutual information of different pose representations under varying views to learn the integrative representation. Wang et al [30] utilized the subgraph-level summary to build an effective mutual information estimator, which was optimized to strengthen the robustness of graph representation. Mao et al [31] explored the shared information across modalities via maximizing the mutual information between them.…”