Understanding and Evaluating User Satisfaction with Music Discovery

Garcia-Gathright, Jean; Thomas, B.S.; Hosey, Christine; Nazari, Zahra; Dı́az, Fernando

doi:10.1145/3209978.3210049

Cited by 46 publications

(18 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The quantitative literature does not often pay attention to the possible existence of distinct classes of users when it comes to their behaviors and listening habits on the platform, classes for which the function and effect of recommendation may differ significantly [17]. Many studies report aggregate effects averaged over binary categories of users, for instance depending on a heavy vs. limited use of recommendation [13,31] or categorical variables such as gender [41] or age [3].…”

Section: Modes Of Access and User Behavior Classesmentioning

confidence: 99%

“…Most of this literature works at the aggregate level without distinguishing populations of users who may differently use or respond to algorithmic guidance. Several studies nonetheless expressly differentiate users who are eager for recommendation [31], diversity [30], or exploration [17,21]. This hints at the existence of different user behaviors and expectations towards recommendation [24] prior to it influencing users.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Follow the guides: disentangling human and algorithmic curation in online music consumption

Villermet

Poiroux

Moussallam³

et al. 2021

Fifteenth ACM Conference on Recommender Systems

View full text Add to dashboard Cite

The role of recommendation systems in the diversity of content consumption on platforms is a much-debated issue. The quantitative state of the art often overlooks the existence of individual attitudes toward guidance, and eventually of different categories of users in this regard. Focusing on the case of music streaming, we analyze the complete listening history of about 9k users over one year and demonstrate that there is no blanket answer to the intertwinement of recommendation use and consumption diversity: it depends on users. First we compute for each user the relative importance of different access modes within their listening history, introducing a trichotomy distinguishing so-called 'organic' use from algorithmic and editorial guidance. We thereby identify four categories of users.We then focus on two scales related to content diversity, both in terms of dispersion -how much users consume the same content repeatedly -and popularity -how popular is the content they consume. We show that the two types of recommendation offered by music platforms -algorithmic and editorial -may drive the consumption of more or less diverse content in opposite directions, depending also strongly on the type of users. Finally, we compare users' streaming histories with the music programming of a selection of popular French radio stations during the same period. While radio programs are usually more tilted toward repetition than users' listening histories, they often program more songs from less popular artists. On the whole, our results highlight the nontrivial effects of platform-mediated recommendation on consumption, and lead us to speak of 'filter niches' rather than 'filter bubbles'. They hint at further ramifications for the study and design of recommendation systems. CCS Concepts: • Human-centered computing → Empirical studies in HCI; • Information systems → Multimedia streaming; Recommender systems.

show abstract

Section: Modes Of Access and User Behavior Classesmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Follow the guides: disentangling human and algorithmic curation in online music consumption

Villermet

Poiroux

Moussallam³

et al. 2021

Fifteenth ACM Conference on Recommender Systems

View full text Add to dashboard Cite

show abstract

“…This assumption has been challenged recently [6,9,17,24]. In order to better understand and improve user experience on the platform, recommender systems have started to rely more on surveys in which users are explicitly asked to rate their experience on the platform, or specific items they have recently consumed [7,8,13,16]. For simplicity, we will focus on the latter kind of surveys, which request explicit point-wise feedback on items.…”

Section: Introductionmentioning

confidence: 99%

Deconfounding User Satisfaction Estimation from Response Rate Bias

Christakopoulou

Traverse

Potter

et al. 2020

Fourteenth ACM Conference on Recommender Systems

View full text Add to dashboard Cite

Improving user satisfaction is at the forefront of industrial recommender systems. While significant progress has been made by utilizing logged implicit data of user-item interactions (i.e., clicks, dwell/watch time, and other user engagement signals), there has been a recent surge of interest in measuring and modeling user satisfaction, as provided by orthogonal data sources. Such data sources typically originate from responses to user satisfaction surveys, which explicitly ask users to rate their experience with the system and/or specific items they have consumed in the recent past. This data can be valuable for measuring and modeling the degree to which a user has had a satisfactory experience on the recommendation platform, since what users do (engagement) does not always align with what users say they want (satisfaction as measured by surveys). We focus on a large-scale industrial system trained on user survey responses to predict user satisfaction. The predictions of the satisfaction model for each user-item pair, combined with the predictions of the other models (e.g., engagement-focused ones), are fed into the ranking component of a real-world recommender system in deciding items to present to the user. It is therefore imperative that the satisfaction model does an equally good job on imputing user satisfaction across slices of users and items, as it would directly impact which items a user is exposed to. However, the data used for training satisfaction models is biased in that users are more likely to respond to a survey when they will respond that they are more satisfied. When the satisfaction survey responses in slices of data with high response rate follow a different distribution than those with low response rate, response rate becomes a confounding factor for user satisfaction estimation. We find positive correlation between response rate and ratings in a large-scale survey dataset collected in our case study. To address this inherent response rate bias in the satisfaction data, we propose an inverse propensity weighting approach within a multi-task learning framework. We extend a simple feed-forward neural network architecture predicting user satisfaction to a shared-bottom multi-task learning architecture with two tasks: the user satisfaction estimation task, and the response rate estimation task. We Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

show abstract

“…In the last years, the music industry has been experiencing many important changes as a result of new user requirements and the wide range of possibilities offered by emerging devices and technologies [12]. These technologies allow users to access huge databases of musical pieces through different kind of applications.…”

Section: Introductionmentioning

confidence: 99%

Emotional classification of music using neural networks with the MediaEval dataset

Medina

Beltrán

2020

Pers Ubiquit Comput

View full text Add to dashboard Cite

The proven ability of music to transmit emotions provokes the increasing interest in the development of new algorithms for music emotion recognition (MER). In this work, we present an automatic system of emotional classification of music by implementing a neural network. This work is based on a previous implementation of a dimensional emotional prediction system in which a Multilayer Perceptron (MLP) was trained with the freely available MediaEval database. Although these previous results are good in terms of the metrics of the prediction values, they are not good enough to obtain a classification by quadrant based on the valence and arousal values predicted by the neural network, mainly due to the imbalance between classes in the dataset. To achieve better classification values a pre-processing phase was implemented to stratify and balance the dataset. Three different classifiers have been compared: Linear Support Vector Machine (SVM), Random Forest and MLP. The best results are obtained with the MLP. An averaged F-measure of 50% is obtained in a four-quadrant classification schema. Two binary classification approaches are also presented: one vs. rest (OvR) approach in four-quadrants and binary classifier in valence and arousal. The OvR approach has an average F-measure of 69%, and the second one obtained F-measure of 73% and 69% in valence and arousal respectively. Finally, a dynamic classification analysis with different time windows was performed using the temporal annotation data of the MediaEval database. The results obtained show that the classification F-measures in four quadrants are practically constant, regardless of the duration of the time window. Also, this work reflects some limitations related to the characteristics of the dataset, including: size, class balance, quality of the annotations and the sound features available.

show abstract

Understanding and Evaluating User Satisfaction with Music Discovery

Cited by 46 publications

References 21 publications

Follow the guides: disentangling human and algorithmic curation in online music consumption

Follow the guides: disentangling human and algorithmic curation in online music consumption

Deconfounding User Satisfaction Estimation from Response Rate Bias

Emotional classification of music using neural networks with the MediaEval dataset

Contact Info

Product

Resources

About