Lifelong Sequential Modeling with Personalized Memorization for User Response Prediction

Ren, Kan; Qin, Jiarui; Yu, Fang; Zhang, Weinan; Zheng, Lei; Bian, Weijie; Zhou, Guorui; Xu, Jian; Yu, Yong; Zhu, Xiaoqiang; Gai, Kun

doi:10.1145/3331184.3331230

Cited by 80 publications

(19 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For all datasets, we aggregate each user's interaction records and sort them by the action timestamps in chronological order. For evaluation purpose, we adopt the data split strategy in [33], [53]. Specifically, suppose a user has L historical behaviors sorted by time, the behavior sequence [1, L − 3] is used for training and predicts whether she/he will interact with the (L−2)-th item.…”

Section: Methodsmentioning

confidence: 99%

MISS: Multi-Interest Self-Supervised Learning Framework for Click-Through Rate Prediction

Guo¹,

Zhang²,

He³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

CTR prediction is essential for modern recommender systems. Ranging from early factorization machines to deep learning based models in recent years, existing CTR methods focus on capturing useful feature interactions or mining important behavior patterns. Despite the effectiveness, we argue that these methods suffer from the risk of label sparsity (i.e., the user-item interactions are highly sparse with respect to the feature space), label noise (i.e., the collected user-item interactions are usually noisy), and the underuse of domain knowledge (i.e., the pairwise correlations between samples). To address these challenging problems, we propose a novel Multi-Interest Self-Supervised learning (MISS) framework which enhances the feature embeddings with interest-level self-supervision signals.With the help of two novel CNN-based multi-interest extractors, self-supervision signals are discovered with full considerations of different interest representations (point-wise and union-wise), interest dependencies (short-range and long-range), and interest correlations (inter-item and intra-item). Based on that, contrastive learning losses are further applied to the augmented views of interest representations, which effectively improves the feature representation learning. Furthermore, our proposed MISS framework can be used as an "plug-in" component with existing CTR prediction models and further boost their performances. Extensive experiments on three large-scale datasets show that MISS significantly outperforms the state-of-the-art models, by up to 13.55% in AUC, and also enjoys good compatibility with representative deep CTR models.

show abstract

Section: Methodsmentioning

confidence: 99%

MISS: Multi-Interest Self-Supervised Learning Framework for Click-Through Rate Prediction

Guo¹,

Zhang²,

He³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In point-wise approaches, many SOTA models focus on feature interactions design, such as DeepFM [26], PNN [27], DCN [28], xDeepFM [29], DCN-M [6] and AutoInt [5]. Other models try to capture users' sequential interest patterns, including GRU4Rec [30], Caser [31], SAS-Rec [32], MIMN [33], HPMN [34], DIN [35] and DIEN [36]. In pair-wise approaches, SVMRank [37] is the pioneer to transform ranking tasks to classification tasks.…”

Section: Related Work a Learning-to-rank Approachesmentioning

confidence: 99%

PeerRank: Robust Learning to Rank With Peer Loss Over Noisy Labels

Liu

Qin

et al. 2022

IEEE Access

Self Cite

View full text Add to dashboard Cite

User-generated data are extensively utilized in learning to rank as they are easy to collect and up-to-date. However, the data inevitably contain noisy labels attributed to users' annotation mistakes, lack of domain knowledge, system failure, etc., making building a robust model challenging. On account of the remarkable nature of deep neural networks in fitting datasets, the noisy labels significantly degrade the performance of learning-to-rank algorithms. To cope with this problem, previous studies have put forward several methods for label de-noising. However, they are either susceptible to the noise distribution on datasets, raising the demand for clean data or incurring more computational costs. Moreover, most of them are tough to extend to different scenarios. This paper proposes a simple yet effective framework named PeerRank that can be applied in broad applications such as click-through rate prediction and commercial web search in learning-to-rank tasks. PeerRank is a robust, effective, and adaptable framework that can couple with numerous models with theoretical guarantees. Extensive experiments on three public real-world datasets with thirteen point-wise base models and four semi-synthetic generation datasets with four pairwise base models show the consistent improvement of PeerRank. The results comparing PeerRank with seven classic and state-of-the-art de-noising methods validate the advantages of PeerRank framework for learning to rank over noisy labels.INDEX TERMS Learning to rank, Noisy labels, Robustness.

show abstract

“…And similarly the Recurrent Recommender Network (RRN) [62] proposed to employ LSTM [22] to capture the dynamics of both users and items. Complementary to the RNN-based methods, Huang et al [25] and Ren et al [44] alternatively adopted memory networks [13], [55] to learn the short-term patterns, together with item attributes. In the meanwhile, the Sequential Hierarchical Attention Network (SHAN) [64] and [26], [67] introduced the attention mechanism to automatically assign different influences of items in a user's long-term set so that the dynamic properties can be captured, then relied on another attention layer to couple user sequential behavior with long-term representation.…”

Section: Related Workmentioning

confidence: 99%

“…However, existing RNN-based methods [62], [65], [68], which exhibit the advantage by capturing temporal or sequential user behaviors, may not be suitable for modeling and capturing the long-term impact of previously purchased items on the future one [9]. To remedy this, many works [9], [25], [26], [44], [60], [64], [67] proposed to use external memory networks (EMN) [55], [61] and attention mechanism [11], [57], where shorter path between any positions in the sequence makes the long-term impact easier to learn. Despite their effectiveness, the ordinal information of historical items is usually not explicitly considered in these works, which may lead to suboptimal performance because the sequential patterns contained in the sequence of user behaviors may be neglected.…”

Section: Introductionmentioning

confidence: 99%

Modeling Dynamic User Preference via Dictionary Learning for Sequential Recommendation

Chen,

Li,

Yan

et al. 2022

Preprint

View full text Add to dashboard Cite

Capturing the dynamics in user preference is crucial to better predict user future behaviors because user preferences often drift over time. Many existing recommendation algorithms -including both shallow and deep ones -often model such dynamics independently, i.e., user static and dynamic preferences are not modeled under the same latent space, which makes it difficult to fuse them for recommendation. This paper considers the problem of embedding a user's sequential behavior into the latent space of user preferences, namely translating sequence to preference. To this end, we formulate the sequential recommendation task as a dictionary learning problem, which learns: 1) a shared dictionary matrix, each row of which represents a partial signal of user dynamic preferences shared across users; and 2) a posterior distribution estimator using a deep autoregressive model integrated with Gated Recurrent Unit (GRU), which can select related rows of the dictionary to represent a user's dynamic preferences conditioned on his/her past behaviors. Qualitative studies on the Netflix dataset demonstrate that the proposed method can capture the user preference drifts over time and quantitative studies on multiple real-world datasets demonstrate that the proposed method can achieve higher accuracy compared with state-of-the-art factorization and neural sequential recommendation methods. The code is available at https://github.com/cchao0116/S2PNM-TKDE2021.

show abstract

Lifelong Sequential Modeling with Personalized Memorization for User Response Prediction

Cited by 80 publications

References 52 publications

MISS: Multi-Interest Self-Supervised Learning Framework for Click-Through Rate Prediction

MISS: Multi-Interest Self-Supervised Learning Framework for Click-Through Rate Prediction

PeerRank: Robust Learning to Rank With Peer Loss Over Noisy Labels

Modeling Dynamic User Preference via Dictionary Learning for Sequential Recommendation

Contact Info

Product

Resources

About