Ensemble contextual bandits for personalized recommendation

Abstract

The cold-start problem has attracted extensive attention among various online services that provide personalized recommendation. Many online vendors employ contextual bandit strategies to tackle the so-called exploration/exploitation dilemma rooted from the cold-start problem. However, due to high-dimensional user/item features and the underlying characteristics of bandit policies, it is often difficult for service providers to obtain and deploy an appropriate algorithm to achieve acceptable and robust economic profit.In this paper, we explore ensemble strategies of contextual bandit algorithms to obtain robust predicted click-through rate (CTR) of web objects. The ensemble is acquired by aggregating different pulling policies of bandit algorithms, rather than forcing the agreement of prediction results or learning a unified predictive model. To this end, we employ a meta-bandit paradigm that places a hyper bandit over the base bandits, to explicitly explore/exploit the relative importance of base bandits based on user feedbacks. Extensive empirical experiments on two real-world data sets (news recommendation and online advertising) demonstrate the effectiveness of our proposed approach in terms of CTR.

Using Contextual Information to Understand Searching and Browsing Behavior

“…Tang et al [225] propose a context-aware recommender system, implemented as a contextual multi-armed bandits problem. Although the authors report extensive offline evaluation (log based and simulation based) with acceptable CTR, no comparison is made from a cold-start problem standpoint.…”

Algorithmic and Ethical Aspects of Recommender Systems in e-Commerce

“…The resulting cumulative reward is considered "the unique good metric to evaluate the recommendation algorithm" [74]. It is common to extend this setting to a contextual bandit problem [117,37,192] [114]. This approach is known as ranked bandits.…”

“…This type of MAB is called multiple-play bandit and has been studied in [109,123]. Another emerging approach is based on ensemble learning, where the bandit algorithm decides which recommendation model to choose for filling the slots in the top-N ranking via exploring the potential of untested models and exploiting the predictive power of the already tested ones [192,62,31].…”

“…The success of these approaches has been witnessed in recommendation competitions such as the Netflix Prize contest or KDD cups [192].…”

“…The idea behind this method is to train a number of component recommenders, whose individual outputs are aggregated for the final prediction. The aggregation is typically done by taking the average or the (weighted) majority vote [192]. This allows to reduce the variance of learning algorithms on unseen data.…”