Knowledge Transfer via Pre-training for Recommendation: A Review and Prospect

Zeng, Zheni; Xiao, Chaojun; Yao, Yuan; Xie, Rong; Liu, Zhiyuan; Lin, Feng; Lin, Leyu; Sun, Maosong

doi:10.3389/fdata.2021.602071

Cited by 28 publications

(23 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Future Modeling in Recommendation. There are some works that indirectly consider future behaviors in sequence-based recommendation [40]. These models are mostly inspired by the masked language model (MLM) of pre-training [7].…”

Section: Related Workmentioning

confidence: 99%

“…There are very few works that consider future information in recommendation. Recently, with the thriving of pre-training, some works introduce the masked language model (MLM) pre-training task in NLP [7] to sequence-based recommendation, which consider bidirectional information to learn better sequential models via selfsupervised learning (SSL) [26,40]. The MLM task randomly masks some items in the user behavior sequence during training, and then attempts to predict them with the remaining unmasked contexts [38,39].…”

mentioning

confidence: 99%

See 1 more Smart Citation

A Peep into the Future: Adversarial Future Encoding in Recommendation

Xie

Zhang

Wang

et al. 2022

Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

Self Cite

View full text Add to dashboard Cite

Personalized recommendation often relies on user historical behaviors to provide items for users. It is intuitive that future information also contains essential messages as supplements to user historical behaviors. However, we cannot directly encode future information into models, since we are unable to get future information in online serving. In this work, we propose a novel adversarial future encoding (AFE) framework to make full use of informative future features in different types of recommendation models. Specifically, AFE contains a future-aware discriminator and a generator. The future-aware discriminator takes both common features and future features as inputs, working as a recommendation prophet to judge user-item pairs. In contrast, the generator is considered as a challenger, which generates items with only common features, aiming to confuse the future-aware prophet. The future-aware discriminator can inspire the generator (to be deployed online) to produce better results. We further conduct a multi-factor optimization to enable a fast and stable model convergence via the direct learning and knowledge distillation losses. Moreover, we have adopted AFE on both a list-wise RL-based ranking model and a point-wise ranking model to verify its universality. In experiments, we conduct sufficient evaluations on two large-scale datasets, achieving significant improvements on both offline and online evaluations. Currently, we have deployed AFE on a real-world system, affecting millions of users. The source code is in https://github.com/modriczhang/AFE.

show abstract

Section: Related Workmentioning

confidence: 99%

mentioning

confidence: 99%

A Peep into the Future: Adversarial Future Encoding in Recommendation

Xie

Zhang

Wang

et al. 2022

Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

Self Cite

View full text Add to dashboard Cite

show abstract

“…Sequence-based recommendation mainly leverages users' sequential behavior to mine users' interests, which focuses on individual information. Recently, various deep neural net-works have been employed for sequence-based recommendation, e.g., RNN [7], memory networks [3], attention mechanisms [23,35,15,30] and mixed models [29,20].…”

Section: Related Workmentioning

confidence: 99%

Multi-view Multi-behavior Contrastive Learning in Recommendation

Wu¹,

Xie²,

Zhu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Multi-behavior recommendation (MBR) aims to jointly consider multiple behaviors to improve the target behavior's performance. We argue that MBR models should: (1) model the coarse-grained commonalities between different behaviors of a user, (2) consider both individual sequence view and global graph view in multi-behavior modeling, and (3) capture the fine-grained differences between multiple behaviors of a user. In this work, we propose a novel Multi-behavior Multi-view Contrastive Learning Recommendation (MMCLR) framework, including three new CL tasks to solve the above challenges, respectively. The multi-behavior CL aims to make different user single-behavior representations of the same user in each view to be similar. The multi-view CL attempts to bridge the gap between a user's sequence-view and graphview representations. The behavior distinction CL focuses on modeling fine-grained differences of different behaviors. In experiments, we conduct extensive evaluations and ablation tests to verify the effectiveness of MMCLR and various CL tasks on two real-world datasets, achieving SOTA performance over existing baselines. Our code will be available on https://github.com/wyqing20/MMCLR

show abstract

“…(2) Text CNN [9] encodes utterances in the current session to learn user preferences by CNN-based model. (3) BERT [8] is a pretraining [34] model that encodes current utterances for recommendation. (4) ReDial [14] is a CRS method which adopts an auto-encoder framework.…”

Section: Datasetsmentioning

confidence: 99%

User-Centric Conversational Recommendation with Multi-Aspect User Modeling

Li,

Xie,

Zhu

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Conversational recommender systems (CRS) aim to provide highquality recommendations in conversations. However, most conventional CRS models mainly focus on the dialogue understanding of the current session, ignoring other rich multi-aspect information of the central subjects (i.e., users) in recommendation. In this work, we highlight that the user's historical dialogue sessions and look-alike users are essential sources of user preferences besides the current dialogue session in CRS. To systematically model the multi-aspect information, we propose a User-Centric Conversational Recommendation (UCCR) model, which returns to the essence of user preference learning in CRS tasks. Specifically, we propose a historical session learner to capture users' multi-view preferences from knowledge, semantic, and consuming views as supplements to the current preference signals. A multi-view preference mapper is conducted to learn the intrinsic correlations among different views in current and historical sessions via self-supervised objectives. We also design a temporal look-alike user selector to understand users via their similar users. The learned multi-aspect multi-view user preferences are then used for the recommendation and dialogue generation. In experiments, we conduct comprehensive evaluations on both Chinese and English CRS datasets. The significant improvements over competitive models in both recommendation and dialogue generation verify the superiority of UCCR. The source code will be available on https://github.com/lisk123/UCCR.

show abstract

Knowledge Transfer via Pre-training for Recommendation: A Review and Prospect

Cited by 28 publications

References 52 publications

A Peep into the Future: Adversarial Future Encoding in Recommendation

A Peep into the Future: Adversarial Future Encoding in Recommendation

Multi-view Multi-behavior Contrastive Learning in Recommendation

User-Centric Conversational Recommendation with Multi-Aspect User Modeling

Contact Info

Product

Resources

About