A method for evaluating discoverability and navigability of recommendation algorithms

Lamprecht, Daniel; Strohmaier, Markus; Helić, Denis

doi:10.1186/s40649-017-0045-3

Cited by 4 publications

(3 citation statements)

References 39 publications

(24 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As it was shown in [6], beyond-mainstream music listeners tend to have larger user profile sizes than users interested in mainstream music, which means that they provide a substantial amount of listening interaction data for services such as Last.fm and Spotify. We assume that improving the recommendation quality for this active user group also leads to another effect, namely a more prominent exposure of (long-tail) music artists due to a better-connected recommendation network [80]. We leave such investigations to future work.…”

Section: Discussionmentioning

confidence: 99%

Support the underground: characteristics of beyond-mainstream music listeners

et al. 2021

View full text Add to dashboard Cite

Music recommender systems have become an integral part of music streaming services such as Spotify and Last.fm to assist users navigating the extensive music collections offered by them. However, while music listeners interested in mainstream music are traditionally served well by music recommender systems, users interested in music beyond the mainstream (i.e., non-popular music) rarely receive relevant recommendations. In this paper, we study the characteristics of beyond-mainstream music and music listeners and analyze to what extent these characteristics impact the quality of music recommendations provided. Therefore, we create a novel dataset consisting of Last.fm listening histories of several thousand beyond-mainstream music listeners, which we enrich with additional metadata describing music tracks and music listeners. Our analysis of this dataset shows four subgroups within the group of beyond-mainstream music listeners that differ not only with respect to their preferred music but also with their demographic characteristics. Furthermore, we evaluate the quality of music recommendations that these subgroups are provided with four different recommendation algorithms where we find significant differences between the groups. Specifically, our results show a positive correlation between a subgroup’s openness towards music listened to by members of other subgroups and recommendation accuracy. We believe that our findings provide valuable insights for developing improved user models and recommendation approaches to better serve beyond-mainstream music listeners.

show abstract

Section: Discussionmentioning

confidence: 99%

Support the underground: characteristics of beyond-mainstream music listeners

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Researchers of algorithm auditing [55] aim to produce methodologies that enable regulators to examine black-box systems, and understand their impact on different stakeholders. Mechanisms have been developed for auditing bias in ad-delivery systems [1,56], dynamic pricing [14], personalization and performance of search engines [26,43,54], information segregation [11], radicalization, and reachability by recommendation systems [17,19,36,53] to mention a few. However, none of the prior works have studied the biases toward entities having special relationship with the marketplace, especially in the context of product recommendations.…”

Section: Related Workmentioning

confidence: 99%

When the Umpire is also a Player: Bias in Private Label Product Recommendations on E-commerce Marketplaces

Dash,

Chakraborty,

Ghosh

et al. 2021

Preprint

View full text Add to dashboard Cite

Algorithmic recommendations mediate interactions between millions of customers and products (in turn, their producers and sellers) on large e-commerce marketplaces like Amazon. In recent years, the producers and sellers have raised concerns about the fairness of black-box recommendation algorithms deployed on these marketplaces. Many complaints are centered around marketplaces biasing the algorithms to preferentially favor their own 'private label' products over competitors. These concerns are exacerbated as marketplaces increasingly de-emphasize or replace 'organic' recommendations with ad-driven 'sponsored' recommendations, which include their own private labels. While these concerns have been covered in popular press and have spawned regulatory investigations, to our knowledge, there has not been any public audit of these marketplace algorithms. In this study, we bridge this gap by performing an end-to-end systematic audit of related item recommendations on Amazon. We propose a network-centric framework to quantify and compare the biases across organic and sponsored related item recommendations. Along a number of our proposed bias measures, we find that the sponsored recommendations are significantly more biased toward Amazon private label products compared to organic recommendations. While our findings are primarily interesting to producers and sellers on Amazon, our proposed bias measures are generally useful for measuring link formation bias in any social or content networks. 1 CCS CONCEPTS• Human-centered computing → Empirical studies in collaborative and social computing.

show abstract

“…At a more theoretical level, Lamprecht et al [26] propose new evaluation measures for the concepts of discoverability (helping users to reach items) and navigability (helping users to explore a collection). Navigability in particular, is measured by simulating the user behavior, modeled through information seeking models where users move from item to item using links, such as "related" movies in a movie web-page.…”

Section: Making the Case For Simulationsmentioning

confidence: 99%

Siren

Bountouridis

Harambam

Makhortykh

et al. 2019

Proceedings of the Conference on Fairness, Accountability, and Transparency

View full text Add to dashboard Cite

The growing volume of digital data stimulates the adoption of recommender systems in di erent socioeconomic domains, including ecommerce, music, and news industries. While news recommenders help consumers deal with information overload and increase their engagement and satisfaction, their use also raises an increasing number of societal concerns, such as "Matthew e ects", " lter bubbles", and the overall lack of transparency. We argue that focusing on transparency for content-providers is an under-explored avenue. As such, we designed a simulation framework called SIREN 1 (SImulating Recommender E ects in online News environments), that allows news content providers to (i) select and parameterize di erent recommenders and (ii) analyze and visualize their e ects with respect to two diversity metrics with little costs. Taking the U.S. news media as a case study, we present an analysis on the recommender e ects with respect to long-tail novelty and unexpectedness using SIREN. Our analysis o ers a number of interesting ndings, such as the similar potential of certain algorithmically simple (item-based k-Nearest Neighbour) and sophisticated strategies (based on Bayesian Personalized Ranking) to increase diversity over time. Overall, we argue that simulating the e ects of recommender systems can help content providers to make more informed decisions when choosing algorithmic recommenders, and as such can help mitigate the aforementioned societal concerns.

show abstract

A method for evaluating discoverability and navigability of recommendation algorithms

Cited by 4 publications

References 39 publications

Support the underground: characteristics of beyond-mainstream music listeners

Support the underground: characteristics of beyond-mainstream music listeners

When the Umpire is also a Player: Bias in Private Label Product Recommendations on E-commerce Marketplaces

Siren

Contact Info

Product

Resources

About