User-centered Evaluation of Popularity Bias in Recommender Systems

Abdollahpouri, Himan; Mansoury, Masoud; Burke, Robin; Mobasher, Bamshad; Malthouse, Edward C.

doi:10.1145/3450613.3456821

Cited by 63 publications

(39 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Popularity bias has been largely investigated from the item-centered perspective, that is, how frequently popular items appear in recommendation lists. The item-centered perspective study ignores users' interest in popular or less popular items, which causes a limitation as the popularity bias does not affect users equally [3]. Recently, Abdollahpouri et al [2] have conducted a research study to look at the popularity bias from a different perspective: the users'.…”

Section: Introductionmentioning

confidence: 99%

The Unfairness of Popularity Bias in Book Recommendation

Naghiaei¹,

Rahmani²,

Dehghan³

2022

Preprint

View full text Add to dashboard Cite

Recent studies have shown that recommendation systems commonly suffer from popularity bias. Popularity bias refers to the problem that popular items (i.e., frequently rated items) are recommended frequently while less popular items are recommended rarely or not at all. Researchers adopted two approaches to examining popularity bias: (i) from the users' perspective, by analyzing how far a recommendation system deviates from user's expectations in receiving popular items, and (ii) by analyzing the amount of exposure that long-tail items receive, measured by overall catalog coverage and novelty. In this paper, we examine the first point of view in the book domain, although the findings may be applied to other domains as well. To this end, we analyze the well-known Book-Crossing dataset and define three user groups based on their tendency towards popular items (i.e., Niche, Diverse, Bestsellerfocused). Further, we evaluate the performance of nine state-of-the-art recommendation algorithms and two baselines (i.e., Random, MostPop) from both the accuracy (e.g., NDCG, Precision, Recall) and popularity bias perspectives. Our results indicate that most state-of-the-art recommendation algorithms suffer from popularity bias in the book domain, and fail to meet users' expectations with Niche and Diverse tastes despite having a larger profile size. Conversely, Bestseller-focused users are more likely to receive high-quality recommendations, both in terms of fairness and personalization. Furthermore, our study shows a tradeoff between personalization and unfairness of popularity bias in recommendation algorithms for users belonging to the Diverse and Bestseller groups, that is, algorithms with high capability of personalization suffer from the unfairness of popularity bias. Finally, across the models, our results show that WMF and VAECF can provide a higher quality recommendation when considering both accuracy and fairness perspectives.

show abstract

Section: Introductionmentioning

confidence: 99%

The Unfairness of Popularity Bias in Book Recommendation

Naghiaei¹,

Rahmani²,

Dehghan³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Then, they proposed a fair re-ranking model by introducing constraints based on the 0-1 integer programming to reduce this gap. Abdollahpouri et al [3] further researched user-centered evaluation of popularity bias. They proposed a user-centered evaluation method that can effectively tackle popularity bias for different user groups while accounting for users' tolerance towards popularity bias using Jensen divergence.…”

Section: Related Workmentioning

confidence: 99%

“…Furthermore, they analyzed whether such algorithmic popularity bias affects users of different genders. Contrary to the works mentioned above that focus on consumers [22,3,2], or provider fairness [1,20,38], our work studies the fairness of recommended items from both perspectives (i.e., CP-Fairness). Mehrotra et al [26] showed that blindly optimizing for consumer relevance might hurt supplier fairness.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

The Unfairness of Active Users and Popularity Bias in Point-of-Interest Recommendation

Rahmani¹,

Deldjoo²,

Tourani³

et al. 2022

Preprint

View full text Add to dashboard Cite

Point-of-Interest (POI) recommender systems provide personalized recommendations to users and help businesses attract potential customers. Despite their success, recent studies suggest that highly datadriven recommendations could be impacted by data biases, resulting in unfair outcomes for different stakeholders, mainly consumers (users) and providers (items). Most existing fairness-related research works in recommender systems treat user fairness and item fairness issues individually, disregarding that RS work in a two-sided marketplace. This paper studies the interplay between (i) the unfairness of active users, (ii) the unfairness of popular items, and (iii) the accuracy (personalization) of recommendation as three angles of our study triangle. We group users into advantaged and disadvantaged levels to measure user fairness based on their activity level. For item fairness, we divide items into short-head, mid-tail, and long-tail groups and study the exposure of these item groups into the top-k recommendation list of users. Experimental validation of eight different recommendation models commonly used for POI recommendation (e.g., contextual, CF) on two publicly available POI recommendation datasets, Gowalla and Yelp, indicate that most well-performing models suffer seriously from the unfairness of popularity bias (provider unfairness). Furthermore, our study shows that most recommendation models cannot satisfy both consumer and producer fairness, indicating a trade-off between these variables possibly due to natural biases in data. We choose the POI recommendation as our test scenario; however, the insights should be trivially extendable on other domains.

show abstract

“…Though long-tail SBR models like TailNet and NISER can provide more tail items, they do not consider how many tail items should be included in a recommendation list. As proposed in [30], these methods mainly take the item-centered perspective but ignore users' different preferences on tail items. To this end, we mitigate the popularity bias in SBR from the user perspective.…”

Section: Introductionmentioning

confidence: 99%

Long-Tail Session-based Recommendation from Calibration

Chen¹,

Wang²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Accurate prediction in session-based recommendation has progressed, but a few studies focused on skewed recommendation lists caused by popularity bias. Existing models on mitigating the popularity bias attempted to reduce the over-concentration on popular items by simply amplifying scores of less popular items. However, they normally ignore the users' different preferences towards tail items. To this end, we incorporate calibration, where calibrated recommendations reflect the user's interests with appropriate proportions, to mitigate the popularity bias from the user's perspective. Concretely, we add a calibration module to predict the ratio of tail items in the recommendation list from the session representation, and align it to the ongoing session. Meanwhile, we propose a two-stage curriculum training strategy to improve the prediction in the calibration module. Experiments on benchmark datasets show that our model can both achieve the competitive accuracy of recommendation and provide more tail items.

show abstract

User-centered Evaluation of Popularity Bias in Recommender Systems

Cited by 63 publications

References 37 publications

The Unfairness of Popularity Bias in Book Recommendation

The Unfairness of Popularity Bias in Book Recommendation

The Unfairness of Active Users and Popularity Bias in Point-of-Interest Recommendation

Long-Tail Session-based Recommendation from Calibration

Contact Info

Product

Resources

About