Dual-interest Factorization-heads Attention for Sequential Recommendation

Lin, Guanyu; Gao, Chen; Zheng, Yu; Chang, Jianxin; Niu, Yifan; Song, Yang; Li, Zhiheng; Jin, Depeng; Li, Yong

doi:10.1145/3543507.3583278

Cited by 3 publications

(1 citation statement)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With their success, SR models are actively studied (Qiu et al 2022;Zhou et al 2022;Du et al 2023;Lin et al 2023;Zhou et al 2023;Yue et al 2023;Liu et al 2023;Jiang et al 2023). Recently, contrastive learning has been used as an aid to improve SR performance.…”

Section: Related Work Sequential Recommendationmentioning

confidence: 99%

An Attentive Inductive Bias for Sequential Recommendation beyond the Self-Attention

Shin,

Choi,

et al. 2024

AAAI

View full text Add to dashboard Cite

Sequential recommendation (SR) models based on Transformers have achieved remarkable successes. The self-attention mechanism of Transformers for computer vision and natural language processing suffers from the oversmoothing problem, i.e., hidden representations becoming similar to tokens. In the SR domain, we, for the first time, show that the same problem occurs. We present pioneering investigations that reveal the low-pass filtering nature of self-attention in the SR, which causes oversmoothing. To this end, we propose a novel method called Beyond Self-Attention for Sequential Recommendation (BSARec), which leverages the Fourier transform to i) inject an inductive bias by considering fine-grained sequential patterns and ii) integrate low and high-frequency information to mitigate oversmoothing. Our discovery shows significant advancements in the SR domain and is expected to bridge the gap for existing Transformer-based SR models. We test our proposed approach through extensive experiments on 6 benchmark datasets. The experimental results demonstrate that our model outperforms 7 baseline methods in terms of recommendation performance. Our code is available at https://github.com/yehjin-shin/BSARec.

show abstract

Section: Related Work Sequential Recommendationmentioning

confidence: 99%

An Attentive Inductive Bias for Sequential Recommendation beyond the Self-Attention

Shin,

Choi,

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

Mixed Attention Network for Cross-domain Sequential Recommendation

Lin,

Gao,

Zheng

et al. 2024

Proceedings of the 17th ACM International Conference on Web Search and Data Mining

View full text Add to dashboard Cite

Collaborative Sequential Recommendations via Multi-view GNN-transformers

Luo,

Liu,

Pan

2024

ACM Trans. Inf. Syst.

View full text Add to dashboard Cite

Sequential recommendation systems aim to exploit users’ sequential behavior patterns to capture their interaction intentions and improve recommendation accuracy. Existing sequential recommendation methods mainly focus on modeling the items’ chronological relationships in each individual user behavior sequence, which may not be effective in making accurate and robust recommendations. On one hand, the performance of existing sequential recommendation methods is usually sensitive to the length of a user’s behavior sequence ( i.e. , the list of a user’s historically interacted items). On the other hand, besides the context information in each individual user behavior sequence, the collaborative information among different users’ behavior sequences is also crucial to make accurate recommendations. However, this kind of information is usually ignored by existing sequential recommendation methods. In this work, we propose a new sequential recommendation framework, which encodes the context information in each individual user behavior sequence as well as the collaborative information among the behavior sequences of different users, through building a local dependency graph for each item. We conduct extensive experiments to compare the proposed model with state-of-the-art sequential recommendation methods on five benchmark datasets. The experimental results demonstrate that the proposed model is able to achieve better recommendation performance than existing methods, by incorporating collaborative information.

show abstract

Dual-interest Factorization-heads Attention for Sequential Recommendation

Cited by 3 publications

References 23 publications

An Attentive Inductive Bias for Sequential Recommendation beyond the Self-Attention

An Attentive Inductive Bias for Sequential Recommendation beyond the Self-Attention

Mixed Attention Network for Cross-domain Sequential Recommendation

Collaborative Sequential Recommendations via Multi-view GNN-transformers

Contact Info

Product

Resources

About