Peeking into the other half of the glass : handling polarization in recommender systems.

Abstract

This dissertation is about filtering and discovering information online while using recommender systems. In the first part of our research, we study the phenomenon of polarization and its impact on filtering and discovering information. Polarization is a social phenomenon, with serious consequences, in real-life, particularly on social media. Thus it is important to understand how machine learning algorithms, especially recommender systems, behave in polarized environments. We study polarization within the context of the users' interactions with a space of items and how this affects recommender systems. We first formalize the concept of polarization based on item ratings and then relate it to the item reviews, when available. We then propose a domain independent data science pipeline to automatically detect polarization using the ratings rather than the properties, typically used to detect polarization, such as item's content or social network topology. We perform an extensive comparison of polarization measures on several benchmark data sets and show that our polarization detection framework can detect different degrees of polarization and outperforms existing measures in capturing an intuitive notion of polarization. We also investigate and uncover certain peculiar patterns that are characteristic of environments where polarization emerges: A machine learning algorithm finds it easier to learn discriminating models in polarized environments: The models will quickly learn to v keep each user in the safety of their preferred viewpoint, essentially, giving rise to filter bubbles and making them easier to learn. After quantifying the extent of polarization in current recommender system benchmark data, we propose new counter-polarization approaches for existing collaborative filtering recommender systems, focusing particularly on the state of the art models based on Matrix Factorization. Our work represents an essential step toward the new research area concerned with quantifying, detecting and counteracting polarization in human-generated data and machine learning algorithms.We also make a theoretical analysis of how polarization affects learning latent factor models, and how counter-polarization affects these models. In the second part of our dissertation, we investigate the problem of discovering related information by recommendation of tags on social media micro-blogging platforms. Real-time micro-blogging services such as Twitter have recently witnessed exponential growth, with millions of active web users who generate billions of micro-posts to share information, opinions and personal viewpoints, daily. However, these posts are inherently noisy and unstructured because they could be in any format, hence making them difficult to organize for the purpose of retrieval of relevant information. One way to solve this problem is using hashtags, which are quickly becoming the standard approach for annotation of various information on social media, such that varied posts about the same or related topic are annotated wi...

Improving the Estimation of Tail Ratings in Recommender System with Multi-Latent Representations

“…Similar to our observations, they showed that predicting ratings for controversial items is much worse than for other items. Badami surveyed state-of-the-art research on the polarization [2], finding that many trust-based RS attempts to improve recommendation for controversial items by defining a trusted network for each user, e.g., [11,27,30,35]. Recently, Beutel et al proposed a focused learning model to improve the recommendation quality for a specified subset of items, through hyper-parameter optimization and a customized matrix factorization objective [4].…”

“…While the importance of the distribution of ratings on RS has been long recognized, e.g., [1,2,15,36], many popular methods based on latent factor models and recently introduced neural variants [3,14,20,22,25,39] optimize for the head of these distributions, potentially leading to large estimation errors for tail ratings. As we will show in Section 3, these tail estimation errors are common across multiple domains and datasets, leading to large overestimations of the ratings of items with very low ratings, and large under-estimations of the ratings of items with very high ratings.…”

Theoretical Modeling of the Iterative Properties of User Discovery in a Collaborative Filtering Recommender System

“…Our interactions with recommender systems naturally generate new data which is then used to update or retrain the next iteration of recommender system models, thus effectively creating a closed feedback loop [25,28] that affects the stream of information visible to us, and consequently, our capacity to discover information online. Possible impacts of limited discovery include the creation of filter bubbles and polarization [6][7][8]. Previous research has focused on dealing with this problem by aiming at increasing the diversity of the recommendation through several post-processing and preprocessing techniques [10,36] .…”

Extracting Information from Negative Interactions in Multiplex Networks Using Mutual Information

“…This recent work has shown that negative links have significant added value over positive links for various analytical tasks. For example, a small number of negative links improves the performance of recommender systems in arXiv:1804.07210v4 [cs.SI] 16 Aug 2018 social media [4][5][6][7][8]. Similarly, trust and distrust relations in Epinions can help users find high-quality and reliable reviews [9].…”

“…social media [4][5][6][7][8]. Similarly, trust and distrust relations in Epinions can help users find high-quality and reliable reviews [9].…”