Balancing Between Accuracy and Fairness for Interactive Recommendation with Reinforcement Learning

Abstract

Fairness in recommendation has attracted increasing attention due to bias and discrimination possibly caused by traditional recommenders. In Interactive Recommender Systems (IRS), user preferences and the system's fairness status are constantly changing over time. Existing fairness-aware recommenders mainly consider fairness in static settings. Directly applying existing methods to IRS will result in poor recommendation. To resolve this problem, we propose a reinforcement learning based framework, FairRec, to dynamically maintain a longterm balance between accuracy and fairness in IRS. User preferences and the system's fairness status are jointly compressed into the state representation to generate recommendations. FairRec aims at maximizing our designed cumulative reward that combines accuracy and fairness. Extensive experiments validate that FairRec can improve fairness, while preserving good recommendation quality.

A Survey on Reinforcement Learning for Recommender Systems

“…Interactive Recommendation UWR [46] Value-function Q-Learning Offline DRCGR [47] DQN Offline UDQN [48] DQN Offline FeedRec [49] DQN Offline PDQ [50] DQN Offline KGQR [51] DQN Offline SL+RL [5] Policy Search REINFORCE Offline RCR [52] REINFORCE Online & Offline TPGR [53] REINFORCE Offline FairRec [54] Actor-Critic DPG Offline Attacks&Detection [8] AC Offline SDAC [55] AC Offline AAMRL [56] AC Online DRR [ [65] Actor-Critic A3C Offline Sequential Recommendation SQN [44] Value-function Q-learning Offline DEERS [66] DQN Online & Offline RLradio [67] R-Learning Online & Offline KERL [68] Policy Search…”

“…This fairness concern is extremely challenging, since it is hard to deal with the conflict between fairness and accuracy. To address the challenge in interactive recommender systems, an RL-based framework (i.e., FairRec) [54] is proposed to dynamically achieve a fairness-accuracy balance, in which the fairness status of the system and user's preferences combine to form the state representation to generate recommendations. Besides, a twofold reward is constructed in terms of both fairness and accuracy.…”

Fairness in Recommender Systems: Research Landscape and Future Directions

“…• Price and brand [35,72], geographical region [73,48] Target: Interaction-oriented fairness -sensitive attribute based on the interactions observed on items e.g., popularity.…”

FAIR: Fairness-Aware Information Retrieval Evaluation

“…The closest work to us is Liu et al(2020), which proposed to combine accuracy and fairness in the reward function for reinforcement learning, aiming to maintain the accuracy-fairness trade-off in interactive recommendation. Yet the reward function is more focused on optimization approach rather than an evaluation metric that can be directly applied to measure a given rank list.…”