Dual Contrastive Network for Sequential Recommendation

Lin, Guanyu; Gao, Chen; Li, Yinfeng; Zheng, Yu; Li, Zhiheng; Jin, Depeng

doi:10.1145/3477495.3531918

Cited by 19 publications

(12 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…in Appendix A.4. 1 where Micro-video is a collected industrial dataset and Amazon is the public benchmark dataset which is widely used in existing work for sequential recommendation [19]. The detailed descriptions of them are as below.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Dual-interest Factorization-heads Attention for Sequential Recommendation

Lin,

Gao,

Zheng

et al. 2023

Preprint

View full text Add to dashboard Cite

Accurate user interest modeling is vital for recommendation scenarios. One of the effective solutions is the sequential recommendation that relies on click behaviors, but this is not elegant in the video feed recommendation where users are passive in receiving the streaming contents and return skip or no-skip behaviors. Here skip and no-skip behaviors can be treated as negative and positive feedback, respectively. With the mixture of positive and negative feedback, it is challenging to capture the transition pattern of behavioral sequence. To do so, FeedRec has exploited a shared vanilla Transformer, which may be inelegant because head interaction of multiheads attention does not consider different types of feedback. In this paper, we propose Dual-interest Factorization-heads Attention for Sequential Recommendation (short for DFAR) consisting of feedback-aware encoding layer, dual-interest disentangling layer and prediction layer. In the feedback-aware encoding layer, we first suppose each head of multi-heads attention can capture specific feedback relations. Then we further propose factorization-heads attention which can mask specific head interaction and inject feedback information so as to factorize the relation between different types of feedback. Additionally, we propose a dual-interest disentangling layer to decouple positive and negative interests before performing disentanglement on their representations. Finally, we evolve the positive and negative interests by corresponding towers whose outputs are contrastive by BPR loss. Experiments on two real-world datasets show the superiority of our proposed method against state-of-the-art baselines. Further ablation study and visualization also sustain its effectiveness. We release the source code here: https://github.com/tsinghua-fib-lab/WWW2023-DFAR. CCS CONCEPTS• Information systems → Recommender systems; • Computing methodologies → Neural networks.

show abstract

Section: Methodsmentioning

confidence: 99%

“…Hyper-parameters are generally set following the default settings of baselines. We strictly follow existing work for sequential recommendation [19] and leverage Adam [15] with the learning rate of 0.0001 to weigh the gradients. The embedding sizes of all models are set as 32.…”

Section: 13mentioning

confidence: 99%

Dual-interest Factorization-heads Attention for Sequential Recommendation

Lin,

Gao,

Zheng

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…• Contrastive learning for long-term representation is effective. Compared with the results in the previous SIGIR version [21], improvement for SLi-Rec is promoted under SLi-Rec after introducing contrastive learning for long-term representation, which means though long and short-term modelings have provided SLi-Rec with sufficient prior knowledge our new design still can facilitate it into a better stage when it has been promoted with sufficient prior knowledge; when we achieve this improvement by performing contrastive learning on long-term representation with static embedding (i.e., target user or item embeddings), this observation also sustains our opinion in the previous version: "long term interest can be treated as static interest [21]".…”

Section: Study Of Encoder Backbonementioning

confidence: 99%

“…Therefore, we tend to enhance the sequential learning by incorporating user sequence for auxiliary contrastive training instead of generating sparser augment data by dropout strategy. Besides, we also have some new insights from the item-centric perspective for the overall performance as shown in Table 4 against the previous SIGIR version [21].…”

Section: Overall Performancementioning

confidence: 99%

“…An early version of this work has been published as a 6-page short paper in SIGIR 2022 [21], compared with which we have the following new contributions: (1) We perform a representation transformation layer on the sequential embeddings, transforming the item sequence embeddings to user representation and the user sequence embeddings to item representation before performing dual representation contrastive learning; (2) We approach the problem of sequential recommendation from a more systematic view, further evaluating the proposed method from item-centric view, instead of solely user-centric view; (3) We conduct experiments on additional two benchmark datasets where the results on both item and user recommendations outperform the state-of-the-art models; (4) Further experimental results of the ablation study on additional datasets also well confirm the effectiveness of our designed self-supervision; (5) We also further perform a hyperparameter study on the regularization parameter of contrastive loss.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation