Predicting performance in recommender systems

Abstract

Performance prediction has gained growing attention in the Information Retrieval field since the late nineties and has become an established research topic in the field. Our work restates the problem in the area of Recommender Systems, where it has barely been researched so far, despite being an appealing problem, as it enables an array of strategies for deciding when to deliver or hold back recommendations based on their foreseen accuracy. We investigate the adaptation and definition of different performance predictors based on the available user and item features. The properties of the predictor are empirically studied by checking the correlation of the predictor output with a performance measure. Then, we propose to introduce the performance predictor in a recommender system to produce a dynamic strategy. Depending on how the predictor is introduced we analyze two different problems: dynamic neighbor weighting in collaborative filtering and dynamic weighting of ensemble recommenders.

The Magic Barrier of Recommender Systems – No Magic, Just Ratings

“…By drawing from Information Retrieval related quantities, Bellogín et al present in [3,4] a family of performance predictors for users. Correlations found between ranking-based metrics and such predictors are strong, and the authors propose to exploit them in at least two applications: dynamic neighbourhood building and dynamic ensemble recommendation, where the weights for the neigbours or the recommenders would dynamically change depending on the predicted performance of each variable.…”

Improving recommender systems

“…The KL divergence is widely used to measure the similarity of any two distributions of data on various domains such as in detecting topics [16], building a user recommendation framework in social tagging system [22], and (most relevant to my research) predicting recommender system performances [4].…”

“…One user is removed from this analysis because he did not rate more than 15 movies, the minimum number of ratings required for the cold start process 4. At this moment, I am not able to report the consumed memory since Lenskit does not generate a report for this metric.…”

Offline optimization for user-specific hybrid recommender systems

“…Automatically adjusting weights (through a learning component) according to user feedback was however deferred to future work. The topic of automatically adjusting user-specific weights for dynamic ensembles has been touched by Bellogín et al [6,7]. From an information retrieval perspective, they proposed adaptations of query performance techniques to define performance predictors in recommender systems.…”

“…We define this problem in the form of an optimization task. We adopt the notation from related work [6] where a dynamic ensemble recommender was defined as…”