Towards Robust Graph Neural Networks via Adversarial Contrastive Learning

Wang, Shen; Chen, Zhengzhang; Ni, Jingchao; Chen, Haifeng; Yu, Philip S.

doi:10.1109/bigdata55660.2022.10021051

Cited by 13 publications

(37 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One of the most popular approaches is adversarial training (AT) [55], which trains the neural network with the worst-case adversarial examples. Although existing contrastive learning literature has shown boosted performance on the standard generalization, its connection with adversarial robustness has not been studied until recently [45,49]. For detailed review please refer [69].…”

Section: Related Workmentioning

confidence: 99%

“…One of the most popular approaches to mitigate the effect of adversarial perturbation is adversarial training (AT) [55], which trains the neural network with worst-case adversarial examples. Very recently, the connection between self-supervised contrastive learning and adversarial training has been built to develop label-efficient and robust models [45,49].…”

Section: Adversarial Robustness Of Hclmentioning

confidence: 99%

“…• We further propose Supervised Hyperbolic Contrastive Learning (SHCL), a variant of general supervised contrastive loss that yields even better performance on supervised image classification tasks. Though the feature consistency w.r.t.data augmentations introduced by the contrastive loss is effective for standard generalization of CNNs and unsupervised learning [43,45,46,87,88,92], deep learning models also exhibit adversarial fragility [14]. Considering pretrained models from self-supervision is usually used in downstream tasks for faster fine-tuning or better accuracy, it is natural to explore whether pretrained self-supervised models play a similar role for adversarial training as they have for standard training [14].…”

Section: Introductionmentioning

confidence: 99%

“…Considering pretrained models from self-supervision is usually used in downstream tasks for faster fine-tuning or better accuracy, it is natural to explore whether pretrained self-supervised models play a similar role for adversarial training as they have for standard training [14]. The connection between self-supervised contrastive learning and adversarial training has not been built until recently [45,49].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Hyperbolic Contrastive Learning

Yen¹,

Lin²,

Yamada³

et al. 2023

Preprint

View full text Add to dashboard Cite

Learning good image representations that are beneficial to downstream tasks is a challenging task in computer vision. As such, a wide variety of self-supervised learning approaches have been proposed. Among them, contrastive learning has shown competitive performance on several benchmark datasets. The embeddings of contrastive learning are arranged on a hypersphere that results in using the inner (dot) product as a distance measurement in Euclidean space. However, the underlying structure of many scientific fields like social networks, brain imaging, and computer graphics data exhibit highly non-Euclidean latent geometry. We propose a novel contrastive learning framework to learn semantic relationships in the hyperbolic space. Hyperbolic space is a continuous version of trees that naturally owns the ability to model hierarchical structures and is thus beneficial for efficient contrastive representation learning. We also extend the proposed Hyperbolic Contrastive Learning (HCL) to the supervised domain and studied the adversarial robustness of HCL. The comprehensive experiments show that our proposed method achieves better results on selfsupervised pretraining, supervised classification, and higher robust accuracy than baseline methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Adversarial Robustness Of Hclmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Hyperbolic Contrastive Learning

Yen¹,

Lin²,

Yamada³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…They use a pseudo-label generation technique to avoid using labels in adversarial training of downstream tasks. Similarly, Jiang et al [12] considered a linear combination of two contrastive loss functions to study the robustness under different pair selection scenarios.…”

Section: Introductionmentioning

confidence: 99%

On the Role of Contrastive Representation Learning in Adversarial Robustness: An Empirical Study

Ghofrani¹,

Yaghouti²,

Jamshidi³

2023

Preprint

View full text Add to dashboard Cite

Self-supervised contrastive learning has solved one of the significant obstacles in deep learning by alleviating the annotation cost. This advantage comes with the price of false negative-pair selection without any label information. Supervised contrastive learning has emerged as an extension of contrastive learning to eliminate this issue. However, aside from accuracy, there is a lack of understanding about the impacts of adversarial training on the representations learned by these learning schemes. In this work, we utilize supervised learning as a baseline to comprehensively study the robustness of contrastive and supervised contrastive learning under different adversarial training scenarios. Then, we begin by looking at how adversarial training affects the learned representations in hidden layers, discovering more redundant representations between layers of the model. Our results on CIFAR-10 and CIFAR-100 image classification benchmarks demonstrate that this redundancy is highly reduced by adversarial fine-tuning applied to the contrastive learning scheme, leading to more robust representations. However, adversarial fine-tuning is not very effective for supervised contrastive learning and supervised learning schemes. Our code is released at https://github.com/softsys4ai/CL-Robustness.

show abstract

High-level preferences as positive examples in contrastive learning for multi-interest sequential recommendation

Zhu,

Li,

Liu

et al. 2024

World Wide Web

View full text Add to dashboard Cite

The sequential recommendation task based on the multi-interest framework aims to model multiple interests of users from different aspects to predict their future interactions. However, researchers rarely consider the differences in features between the interests generated by the model. In extreme cases, all interest capsules have the same meaning, leading to the failure of modeling users with multiple interests. To address this issue, we propose the High-level Preferences as positive examples in Contrastive Learning for multi-interest Sequence Recommendation framework (HPCL4SR), which uses contrastive learning to distinguish differences in interests based on user item interaction information. In order to find high-quality comparative examples, this paper introduces the category information to construct a global graph, learning the association between categories for high-level preference interest of users. Then, a multi-layer perceptron is used to adaptively fuse the low-level preference interest features of the user's items and the high-level preference interest features of the categories. Finally, user multiinterest contrastive samples are obtained through item sequence information and corresponding categories, which are fed into contrastive learning to optimize model parameters and generate multi-interest representations that are more in line with the user sequence. In addition, when modeling the user's item sequence information, in order to increase the differentiation between item representations, 1 the category of the item is used to supervise the learning process. Extensive experiments on three real datasets demonstrate that our method outperforms existing multi-interest recommendation models.

show abstract

Towards Robust Graph Neural Networks via Adversarial Contrastive Learning

Cited by 13 publications

References 22 publications

Hyperbolic Contrastive Learning

Hyperbolic Contrastive Learning

On the Role of Contrastive Representation Learning in Adversarial Robustness: An Empirical Study

High-level preferences as positive examples in contrastive learning for multi-interest sequential recommendation

Contact Info

Product

Resources

About