Towards Multi-Interest Pre-training with Sparse Capsule Network

Tang, Zhu; Wang, Lin; Zou, Lixin; Zhang, Xiaolu; Zhou, Jun; Li, Chenliang

doi:10.1145/3539618.3591778

Cited by 5 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They introduce the variance regularizer on the routing weights to eliminate sparsity and effectively address the problem. MIRACLE [19] forces interest capsules to satisfy orthogonality, which clearly provides each user with K unrelated interests. However, such K interests can cause unnecessary item recommendations for users, which goes against our common sense that there may be implicit correlations between interests.…”

Section: Contrastive Learningmentioning

confidence: 99%

“…Finally, we analyze and consider the contribution of differences in interests to multi-interest sequence recommendation. We not only attempt to replace the multi-interest contrastive learning module with Capsule Regulation [19], denoted as HPCL4SR(w CR), but also analyze the impact of the lack of additive angle margin m, denoted as HPCL4SR(w/o m). The experimental results on three data sets are shown in Table 3.…”

Section: Ablation Studymentioning

confidence: 99%

“…When modeling user interests using the above structure, it is common to represent user interests as low dimensional embeddings, which contradicts the fact that each user may have multiple interests in reality [16]. Some studies [3,[17][18][19] propose capturing multiple user interest vectors from different aspects instead of a single vector. These methods explicitly generate users' diverse interest representations from their behavior sequences, breaking the representation bottleneck of using a single generic user embedding.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

High-level preferences as positive examples in contrastive learning for multi-interest sequential recommendation

Zhu,

Li,

Liu

et al. 2024

World Wide Web

View full text Add to dashboard Cite

The sequential recommendation task based on the multi-interest framework aims to model multiple interests of users from different aspects to predict their future interactions. However, researchers rarely consider the differences in features between the interests generated by the model. In extreme cases, all interest capsules have the same meaning, leading to the failure of modeling users with multiple interests. To address this issue, we propose the High-level Preferences as positive examples in Contrastive Learning for multi-interest Sequence Recommendation framework (HPCL4SR), which uses contrastive learning to distinguish differences in interests based on user item interaction information. In order to find high-quality comparative examples, this paper introduces the category information to construct a global graph, learning the association between categories for high-level preference interest of users. Then, a multi-layer perceptron is used to adaptively fuse the low-level preference interest features of the user's items and the high-level preference interest features of the categories. Finally, user multiinterest contrastive samples are obtained through item sequence information and corresponding categories, which are fed into contrastive learning to optimize model parameters and generate multi-interest representations that are more in line with the user sequence. In addition, when modeling the user's item sequence information, in order to increase the differentiation between item representations, 1 the category of the item is used to supervise the learning process. Extensive experiments on three real datasets demonstrate that our method outperforms existing multi-interest recommendation models.

show abstract

Section: Contrastive Learningmentioning

confidence: 99%

Section: Ablation Studymentioning

confidence: 99%