User experience with commercial music services: An empirical exploration

Bainbridge

et al. 2015

Self Cite

Music Information Retrieval (MIR) evaluation has traditionally focused on system-centered approaches where components of MIR systems are evaluated against predefined data sets and golden answers (i.e., ground truth). There are two major limitations of such systemcentered evaluation approaches: (a) The evaluation focuses on subtasks in music information retrieval, but not on entire systems and (b) users and their interactions with MIR systems are largely excluded. This article describes the first implementation of a holistic userexperience evaluation in MIR, the MIREX Grand Challenge, where complete MIR systems are evaluated, with user experience being the single overarching goal. It is the first time that complete MIR systems have been evaluated with end users in a realistic scenario. We present the design of the evaluation task, the evaluation criteria and a novel evaluation interface, and the datacollection platform. This is followed by an analysis of the results, reflection on the experience and lessons learned, and plans for future directions.

Section: The Criteriamentioning

confidence: 99%

Section: User-centered Evaluation In Mirmentioning

confidence: 99%

Section: The Criteriamentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The MIREX grand challenge: A framework of holistic user‐experience evaluation in music information retrieval

Bainbridge

et al. 2015

Self Cite

“…Schedl, Flexer, and Urbano (2013), Schedl, G omez, and Urbano (2014), and Schedl and Hauger (2015) have argued in a series of articles that usercentric inquiry, rather than top-down system design, is crucial to improving features such as recommendation tools and collection organization/visualization interfaces. Lee and Price (2016) also call for a user-centered approach in evaluating MIR systems, suggesting a set of evaluation criteria.…”

Section: Music User Behaviormentioning

confidence: 99%

Understanding users of cloud music services: Selection factors, management and access behavior, and perceptions

Wishkoski

Aase

et al. 2016

Self Cite

Recent, rapid changes in technology have resulted in a proliferation of choices for music storage and access. Portable, web‐enabled music devices are widespread, and listeners now enjoy a plethora of options regarding formats, devices, and access methods. Yet in this mobile music environment, listeners' access and management strategies for music collections are poorly understood, because behaviors surrounding the organization and retrieval of music collections have received little formal study. Our current research seeks to enrich our knowledge of people's music listening and collecting behavior through a series of systematic user studies. In this paper we present our findings from interviews involving 20 adult and 20 teen users of commercial cloud music services. Our results contribute to theoretical understandings of users' music information behavior in a time of upheaval in music usage patterns, and more generally, the purposes and meanings users ascribe to personal media collections in cloud‐based systems. The findings suggest improvements to the future design of cloud‐based music services, as well as to any information systems and services designed for personal media collections, benefiting both commercial entities and listeners.

Users' music information needs and behaviors: Design implications for music information retrieval systems

Cho

Kim

2015

Self Cite

User studies in the music information retrieval (MIR) domain tend to be exploratory and qualitative in nature, involving a small number of users, which makes it difficult to derive broader implications for system design. In order to fill this gap, we conducted a large-scale user survey questioning various aspects of people's music information needs and behaviors. In particular, we investigate if general music users' needs and behaviors have significantly changed over time by comparing our current survey result with a similar survey conducted in 2004. In this paper, we present the key findings from the survey data and discuss 4 emergent themes-(a) the shift in access and use of personal music collections; (b) the growing need for tools to support collaborative music seeking, listening, and sharing; (c) the importance of "visual" music experiences; and (d) the need for ontologies for providing rich contextual information. We conclude by making specific recommendations for improving the design of MIR systems and services.