A Study of Smoothing Methods for Relevance-Based Language Modelling of Recommender Systems

Abstract

Language Models have been traditionally used in several fields like speech recognition or document retrieval. It was only recently when their use was extended to collaborative Recommender Systems. In this field, a Language Model is estimated for each user based on the probabilities of the items. A central issue in the estimation of such Language Model is smoothing, i.e., how to adjust the maximum likelihood estimator to compensate for rating sparsity. This work is devoted to explore how the classical smoothing approaches (Absolute Discounting, Jelinek-Mercer and Dirichlet priors) perform in the field of Recommender Systems. We tested the different methods under the recently presented Relevance-Based Language Models for collaborative filtering, and compared how the smoothing techniques behave in terms of precision and stability. We found that Absolute Discounting is practically insensitive to the parameter value being an almost parameter-free method and, at the same time, its performance is similar to Jelinek-Mercer and Dirichlet priors.

Axiomatic Analysis of Language Modelling of Recommender Systems

“…When using language models for recommendation, we compute the probability of an item given a user p(i|u) by smoothing the maximum likelihood estimate (MLE) p ml (i|u) of a multinomial distribution: 17,24…”

“…Previous work on using relevance models for recommendation has computed neighbourhoods using k-NN algorithm with Pearson or cosine similarities. 17,24,25 In general, cosine similarity provides better results than Pearson in terms of accuracy metrics for top-N recommendation. 5 Additionally, for relevance models, this also holds.…”

“…However, a posterior axiomatic analysis of RM2 for collaborative filtering showed that the IDF effect is related to item novelty in recommendation advocating for the use of additive smoothing in this setting. 24…”

A Study of Priors for Relevance-Based Language Modelling of Recommender Systems

“…The rationale behind this decision is that this smoothing method models the user bias yielding better recommendations. The reason is that the amount of smoothing applied from the background collection p(i|C) is inversely proportional to the average rating of the user [10]:…”

“…Relevance-Based Language Models (RM) were conceived for expanding queries automatically [6]. However, they can be effectively applied to CF recommendation [8,2,10]. The task of recommending items to a user can be assimilated to the task of expanding a query with new terms.…”