Multiobjective Deep Reinforcement Learning for Recommendation Systems

Abstract

Most existing recommendation systems (RSs) are primarily concerned about the accuracy of rating prediction and only recommending popular items. However, other non-accuracy metrics such as novelty and diversity should not be overlooked. Existing multi-objective (MO) RSs employed collaborative filtering and combined with evolutionary algorithms to handle bi-objective optimization. Besides cold-start problem from collaborative filtering, it also vulnerable to highly sparse environment, while the evolutionary algorithm suffers from premature convergence and curse of dimensionality. These limitations have prompted this work to propose deep reinforcement learning (DRL) approaches for MO optimization in RSs. Several works in DRL are available but none has addressed MO RS problems. In this study, the performances of proposed DRL approaches that based on Deep Q-Network in MO recommendation problem were investigated. The approaches were evaluated with movie recommendation dataset by using three conflicting metrics, namely precision, novelty, and diversity. The results demonstrated that deep reinforcement learning approaches has superiority performance in MO optimization, and its capability of recommending precise item along with achieving high novelty and diversity against the benchmark that using probabilistic based multiobjective approach based on evolutionary algorithm (PMOEA). Although PMOEA algorithm secured higher average value in precision, it has lower values of novelty and diversity than the proposed DRL approaches. The DRL approaches surpassed the benchmark results in average of maximum novelty and the average of mean diversity metrics, the optimization between accuracy and non-accuracy metrics is inevitable. In addition, the experiments revealed that incorporation of user latent features enhanced the recommendation quality.

Cinematic Curator: A Machine Learning Approach to Personalized Movie Recommendations

“…The review concludes by presenting future trends and challenges, providing valuable insights for researchers in this field. E. Y. Keat [13], this work addresses the limitations of existing recommendation systems (RSs) that primarily focus on rating prediction accuracy and popularity, neglecting metrics like novelty and diversity. To overcome challenges in multiobjective optimization, the study proposes two deep reinforcement learning (DRL) approaches, DQNMORS and Radnor's, for RSs.…”

A Graph Pointer Network-Based Multi-Objective Deep Reinforcement Learning Algorithm for Solving the Traveling Salesman Problem

“…For example, in [27], a pointer network-based multi-objective deep reinforcement learning model was introduced to assist service function chains placement needed for 5G (or beyond) network services' strict quality of service requirements (e.g., minimizing E2E latency and minimizing computing resource congestion among all nodes). In [28], a deep Q-network-based multi-objective deep reinforcement learning model was introduced to recommend movies based on three conflicting metrics, namely, precision, novelty, and diversity. MODRL/D-EL [29] introduced a hybrid solution to improve MODRL/D-AM from [30], and a new evolutionary learning model to solve a multi-objective vehicle routing problem with time windows (MO-VRPTW) [29].…”

Deep Learning-Based Recommendation System: Systematic Review and Classification

“…Deep reinforcement involves incorporating reinforcement learning in deep learning models, while deep neural networks model complex relationships between user and item features [268]. For instance, Liu et al proposed a hybrid neural recommendation model that uses ratings and reviews to learn deep representations for users and items [269].…”