MARS: Memory Attention-Aware Recommender System

Zheng, Lei; Lu, Chun-Ta; He, Lifang; Xie, Sihong; Noroozi, Vahid; Huang, He; Yu, Philip S.

doi:10.48550/arxiv.1805.07037

Cited by 5 publications

(4 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Occasionally, we nd that results can be con icting and relative positions change very frequently. For example, the scores of NCF in [201] is relatively ranked very low as compared to the original paper that proposed the model [53]. is makes the relative benchmark of new neural models extremely challenging.…”

Section: The Field Needs Be Er More Unified and Harder Evaluationmentioning

confidence: 95%

Deep Learning based Recommender System: A Survey and New Perspectives

Zhang,

Yao,

Sun

et al. 2017

Preprint

View full text Add to dashboard Cite

With the ever-growing volume of online information, recommender systems have been an e ective strategy to overcome such information overload. e utility of recommender systems cannot be overstated, given its widespread adoption in many web applications, along with its potential impact to ameliorate many problems related to over-choice. In recent years, deep learning has garnered considerable interest in many research elds such as computer vision and natural language processing, owing not only to stellar performance but also the a ractive property of learning feature representations from scratch. e in uence of deep learning is also pervasive, recently demonstrating its e ectiveness when applied to information retrieval and recommender systems research. Evidently, the eld of deep learning in recommender system is ourishing. is article aims to provide a comprehensive review of recent research e orts on deep learning based recommender systems. More concretely, we provide and devise a taxonomy of deep learning based recommendation models, along with providing a comprehensive summary of the state-of-the-art. Finally, we expand on current trends and provide new perspectives pertaining to this new exciting development of the eld.

show abstract

Section: The Field Needs Be Er More Unified and Harder Evaluationmentioning

confidence: 95%

Deep Learning based Recommender System: A Survey and New Perspectives

Zhang,

Yao,

Sun

et al. 2017

Preprint

View full text Add to dashboard Cite

show abstract

“…A similar architecture is proposed to handle both textual and visual question answering tasks (Xiong, Merity, and Socher 2016), where visual inputs are fed into a deep convolutional network and high-level features are extracted and processed into an input sequence for the attention network. If we further extend memory and query representation to fields beyond question answering, memory-based attention techniques are also used in aspect and opinion term mining (Wang et al 2017) where query is represented as aspect prototypes, in recommender systems (Zheng et al 2018) where users become the memory component and items become queries, in topic modelings (Zeng et al 2018) where latent topic representation extracted from a deep network constitutes the memory, etc.…”

Section: Memory-based Attentionmentioning

confidence: 99%

An Introductory Survey on Attention Mechanisms in NLP Problems

2019

Advances in Intelligent Systems and Computing

172

View full text Add to dashboard Cite

First derived from human intuition, later adapted to machine translation for automatic token alignment, attention mechanism, a simple method that can be used for encoding sequence data based on the importance score each element is assigned, has been widely applied to and attained significant improvement in various tasks in natural language processing, including sentiment classification, text summarization, question answering, dependency parsing, etc. In this paper, we survey through recent works and conduct an introductory summary of the attention mechanism in different NLP problems, aiming to provide our readers with basic knowledge on this widely used method, discuss its different variants for different tasks, explore its association with other techniques in machine learning, and examine methods for evaluating its performance.

show abstract

“…The Facebook DLRM network (Naumov et al, 2019) mimics factorization machines more directly by passing the pairwise dot product between different embeddings into a multilayer perceptron (MLP). More sophisticated techniques that incorporate trees, memory, and (self-)attention mechanisms (to capture sequential user behavior) have also been proposed (Zheng et al, 2018;Zhou et al, 2018a;b;Zhu et al, 2018).…”

Section: Related Workmentioning

confidence: 99%

Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems

Shi

Mudigere

Naumov

et al. 2020

Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery &Amp; Data Mining

View full text Add to dashboard Cite

Modern deep learning-based recommendation systems exploit hundreds to thousands of different categorical features, each with millions of different categories ranging from clicks to posts. To respect the natural diversity within the categorical data, embeddings map each category to a unique dense representation within an embedded space. Since each categorical feature could take on as many as tens of millions of different possible categories, the embedding tables form the primary memory bottleneck during both training and inference. We propose a novel approach for reducing the embedding size in an end-to-end fashion by exploiting complementary partitions of the category set to produce a unique embedding vector for each category without explicit definition. By storing multiple smaller embedding tables based on each complementary partition and combining embeddings from each table, we define a unique embedding for each category at smaller cost. This approach may be interpreted as using a specific fixed codebook to ensure uniqueness of each category's representation. Our experimental results demonstrate the effectiveness of our approach over the hashing trick for reducing the size of the embedding tables in terms of model loss and accuracy, while retaining a similar reduction in the number of parameters.

show abstract

MARS: Memory Attention-Aware Recommender System

Cited by 5 publications

References 29 publications

Deep Learning based Recommender System: A Survey and New Perspectives

Deep Learning based Recommender System: A Survey and New Perspectives

An Introductory Survey on Attention Mechanisms in NLP Problems

Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems

Contact Info

Product

Resources

About