Parameter-Efficient Transfer from Sequential Behaviors for User Modeling and Recommendation

Yuan, Fajie; He, Xiangnan; Karatzoglou, Alexandros; Zhang, Liguang

doi:10.1145/3397271.3401156

Cited by 129 publications

(109 citation statements)

References 18 publications

Supporting

Mentioning

109

Contrasting

Order By: Relevance

“…Caser (Tang and Wang, 2018) is a shallow recommendation model which combines vertical and horizontal CNNs to extract user and item representations. BERT4Rec (Sun et al, 2019) FIGURE 2 | The model architecture of Peterrec (Yuan et al, 2020). (A) is the original residual block of the pre-trained model.…”

Section: Task Settings and Baselinesmentioning

confidence: 99%

“…According to the model architecture, we can categorize existing works in recommender systems into two classes: shallow neural networks ( Hu et al, 2018 ; Ni et al, 2018 ) and deep residual neural networks. Existing deep residual neural networks for recommendation can be further divided into BERT-based models ( Chen et al, 2019c ; Sun et al, 2019 ; Yang et al, 2019 ) and parameter-efficient pre-trained convolutional neural networks ( Yuan et al, 2020 ).…”

Section: Fine-tuning Modelsmentioning

confidence: 99%

“…To address this issue, Yuan et al (2020) proposed the Peterrec, which utilizes a grafting neural network in fine-tuning, termed as model patch (MP). By inserting MPs in the pre-trained models, the fine-tuning networks can keep all pre-trained parameters unchanged.…”

Section: Fine-tuning Modelsmentioning

confidence: 99%

“… The model architecture of Peterrec ( Yuan et al, 2020 ). (A) is the original residual block of the pre-trained model.…”

Section: Fine-tuning Modelsmentioning

confidence: 99%

“…For example, if a user is cold in the target domain, it is useful to transfer his/her representation pre-trained in the general domain; if an item is cold, its representation can be estimated by leveraging the pre-trained representations of external information. Peterrec ( Yuan et al, 2020 ) is a good exploration which achieves user cold start with pre-trained models. Knowledge Enhanced Pre-training : knowledge graphs can provide rich domain knowledge, world knowledge, and commonsense knowledge for recommendation.…”

Section: Open Challenges and Future Directionsmentioning

confidence: 99%

See 4 more Smart Citations

Knowledge Transfer via Pre-training for Recommendation: A Review and Prospect

et al. 2021

View full text Add to dashboard Cite

Recommender systems aim to provide item recommendations for users and are usually faced with data sparsity problems (e.g., cold start) in real-world scenarios. Recently pre-trained models have shown their effectiveness in knowledge transfer between domains and tasks, which can potentially alleviate the data sparsity problem in recommender systems. In this survey, we first provide a review of recommender systems with pre-training. In addition, we show the benefits of pre-training to recommender systems through experiments. Finally, we discuss several promising directions for future research of recommender systems with pre-training. The source code of our experiments will be available to facilitate future research.

show abstract

Section: Task Settings and Baselinesmentioning

confidence: 99%

Section: Fine-tuning Modelsmentioning

confidence: 99%

Section: Fine-tuning Modelsmentioning

confidence: 99%

“… The model architecture of Peterrec ( Yuan et al, 2020 ). (A) is the original residual block of the pre-trained model.…”

Section: Fine-tuning Modelsmentioning

confidence: 99%