On Sampling Strategies for Neural Network-based Collaborative Filtering

Abstract

Recent advances in neural networks have inspired people to design hybrid recommendation algorithms that can incorporate both (1) user-item interaction information and (2) content information including image, audio, and text. Despite their promising results, neural network-based recommendation algorithms pose extensive computational costs, making it challenging to scale and improve upon. In this paper, we propose a general neural network-based recommendation framework, which subsumes several existing stateof-the-art recommendation algorithms, and address the efficiency issue by investigating sampling strategies in the stochastic gradient descent training for the framework. We tackle this issue by first establishing a connection between the loss functions and the useritem interaction bipartite graph, where the loss function terms are defined on links while major computation burdens are located at nodes. We call this type of loss functions "graph-based" loss functions, for which varied mini-batch sampling strategies can have different computational costs. Based on the insight, three novel sampling strategies are proposed, which can significantly improve the training efficiency of the proposed framework (up to ×30 times speedup in our experiments), as well as improving the recommendation performance. Theoretical analysis is also provided for both the computational cost and the convergence. We believe the study of sampling strategies have further implications on general graphbased loss functions, and would also enable more research under the neural network-based recommendation framework.

Deeply supervised model for click-through rate prediction in sponsored search

“…Recently, search2vec model for learning with implicit negative signals from sponsored search sessions was proposed [12] with improved performance and speed of the algorithm. Furthermore, [3] have confirmed this approach and applied it on the special case of bipartite graphs. We exploit implicit negatives in our model and consider comparing to search2vec algorithm in Section 4.2.…”

“…In the past, learning such implicit relations between queries and ads has shown great benefit in sponsored search ad recommendations [11], while its computational benefits were supported in [3]. In this study, unlike in [3], implicit negative samples naturally occur as signals from the users, furthermore they do not consider that complete ground-truth bipartite graph is needed to obtain the good working model, as artificial negative samples can be harmful if a pair is semantically related. The later issue is leveraged with matching tensor layer, while matching loss merely plays a role of discriminativeness enforcing regularizer.…”

Co-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity Alignment

“…collocating). To facilitate the sampling-based approximation, we use the stratified negative sharing technique [Chen et al, 2017c]. That is to say, we sample batches of ILLs into B d .…”

Attribute-Aware Recommender System Based on Collaborative Filtering: Survey and Classification

“…Embed. 4 [17] 2017 CVAE [61] 2017 entity2rec [77] 2017 NFM [38] 2017 MFM [66] 2017 Focused FM [7] 2017 GB-CENT [121] 2017 CML [43] 2017 ATRank [123] 2018 Div-HeteRec [73] 2018 HeteLearn [48] 2018 RNNLatentCross [8] 2018 DDL [119] 2018…”