An Adaptive Similarity-Measuring-Based CMAB Model for Recommendation System

Abstract

Online context-based domains such as recommendation systems strive to promptly suggest the appropriate items to users according to the information about items and users. However, such contextual information may be not available in practical, where the only information we can utilize is users' interaction data. Furthermore, the lack of clicked records, especially for the new users, worsens the performance of the system. To address the issues, similarity measuring, one of the key techniques in collaborative filtering, as well as the online context-based multiple armed bandit mechanism, are combined. The similarity between the context of a selected item and any candidate item is calculated and weighted. An adaptive method for adjusting the weights according to the passed time from clicking is proposed. The weighted similarity is then multiplied with the action value to decide which action is optimal or the poorest. Additionally, we come up with an exploration probability equation by introducing the selected times for the poorest action and the variance of the action values, to balance the exploration and exploitation. The regret analysis is given and the upper bound of the regret is proved. Empirical studies on three benchmarks, random dataset, Yahoo!R6A, and MovieLens, demonstrate the effectiveness of the proposed method.

A multi‐agent system for itinerary suggestion in smart environments

“…Nevertheless, supporting the itinerary decisions of unknown users is especially important in scenarios where users have only few interactions with the RS, for instance, the visitors of a museum. An emerging way [7][8][9] to deal with this challenging problem lies in adopting a reinforcement learning strategy, considering the two-faced aim of a RS: on the one hand, it aims to maximise user satisfaction with the immediate itinerary suggestion (exploitation of known knowledge), on the other hand, it wants to gain new knowledge about user preferences (exploration of new information) to improve long-term reward, although with some effect on short-term satisfaction. This dilemma is typically conjugated in the context of RSs as a Multi-Armed Bandit problem [7,10].…”