Graph Transformer for Recommendation

Abstract

This paper presents a novel approach to representation learning in recommender systems by integrating generative self-supervised learning with graph transformer architecture. We highlight the importance of high-quality data augmentation with relevant selfsupervised pretext tasks for improving performance. Towards this end, we propose a new approach that automates the self-supervision augmentation process through a rationale-aware generative SSL that distills informative user-item interaction patterns. The proposed recommender with Graph TransFormer (GFormer) that offers parameterized collaborative rationale discovery for selective augmentation while preserving global-aware user-item relationships. In GFormer, we allow the rationale-aware SSL to inspire graph collaborative filtering with task-adaptive invariant rationalization in graph transformer. The experimental results reveal that our GFormer has the capability to consistently improve the performance over baselines on different datasets. Several in-depth experiments further investigate the invariant rationale-aware augmentation from various aspects.

Dual Representation Propagation Comparative Learning for Recommendations

“…For example, SGL [8] employs lossy random node/edge operations to generate contrastive learning views, while LightGCL [12] generates contrastive learning views through singular value decomposition (SVD). In addition, some studies [10,11,28] have suggested that manually designed contrastive views may not be able to adapt to different data scenarios and downstream tasks. Therefore, some models [10,29,30] have been improved to address this issue and have proposed methods for automatically generating comparison views.…”

“…However, self-supervised learning relies on manually designed self-supervised tasks or signals to generate training data [10,11]. If an inappropriate self-supervised task is chosen or the supervised signals are not rich enough, the model may not be able to accurately extract global and local information, thus learning irrelevant features or local patterns while ignoring the global structure [12].…”