The Scientific Evaluation of Music Information Retrieval Systems: Foundations and Future

Downie, J. Stephen

doi:10.1162/014892604323112211

Cited by 61 publications

(44 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Reinforcing this shift, some key work has also been done on user needs using grounded theory analysis of music queries (Bainbridge et al, 2003) and ethnographic examinations of personal music collections (Cunningham et al 2004). The MIR community is currently focussed on creating a TREC-like evaluation scenario to give credibility and focus to user studies in this area (Lee and Downie 2004, Downie 2003a, Downie 2003b, Cunningham 2002. It is not easy to measure the effectiveness of Music Information Retrieval (MIR) systems because of the affective nature of the documents which are being sought.…”

Section: Mir Evaluation and Relevancementioning

confidence: 99%

A study of the information needs of the users of a folk music library and the implications for the design of a digital library system

Inskip

Butterworth

MacFarlane

2008

Information Processing & Management

View full text Add to dashboard Cite

Citation: Inskip, C., Butterworth, R. & MacFarlane, A. (2008). A study of the information needs of the users of a folk music library and the implications for the design of a digital library system. Information Processing and Management, 44(2), pp. 647-662. doi: 10.1016/j.ipm.2007.05.004 This is the unspecified version of the paper.This version of the publication may differ from the final published version. (2000) is used as an analytical tool to achieve this end. The demographics of the users were examined in order to establish four user groups: PermanentPerformer, Academic, Professional and Enthusiast. Important information needs were found to be based on social interaction, and key resources of the library were its staff, the concentration of the collection and the library's social nature. A collection of broad design requirements are proposed based on the analysis and this study also provided some insights into the issue of musical relevance, which are discussed.

show abstract

Section: Mir Evaluation and Relevancementioning

confidence: 99%

A study of the information needs of the users of a folk music library and the implications for the design of a digital library system

Inskip

Butterworth

MacFarlane

2008

Information Processing & Management

View full text Add to dashboard Cite

show abstract

“…(e.g. Downie, 2003;Downie, 2004;Lee and Downie, 2004;Casey et al, 2008, see also Laplante, 2008 A good example of the latter is Downie's (2003, pp. 293-301) specification of seven facets of music information, which consists of F o r P e e r R e v i e w pitch, temporal, harmonic, timbral, editorial, textual and bibliographic facets.…”

Section: Literature Reviewmentioning

confidence: 99%

A typology of music information for studies on information seeking

Rousi

Savolainen

Vakkari

2016

Journal of Documentation

View full text Add to dashboard Cite

Purpose – A need to renew music-related information notions arises from both information-seeking models and literature of musical semiotics. The purpose of this paper is to create a music information typology, which aims at facilitating the examination of music information types at varying levels of abstraction in the context of information seeking. Design/methodology/approach – Literature of musical semiotics and information seeking are juxtaposed to develop a novel approach to music-related information. The grounding concepts are Bruner’s enactive, iconic and symbolic modes of representation. The modes of representation offer a universal scheme of knowledge that is applied to the domain of music by defining their content through Tarasti’s Theory of Musical Semiotics. Findings – This conceptual paper results in a music information typology ranging from the enactive music information representations to the abstract ones as follows. Music making as the first mode of enactive representations; music listening as the second mode of enactive representations; iconic representations of music; technological models of music as the first mode of symbolic representations; and ideological models of music as the second mode of symbolic representations. Originality/value – The present paper develops a music information typology that encompasses broadly different music information facets by categorizing music information sources according to their level of abstraction. When applied into empirical research, the typology opens a new window into the perceived roles of music information types in the context of information seeking.

show abstract

“…[1] This approach has been taken on board by a number of MIREX participants, who gather ground truth data from human volunteers who participate in evaluations (including, for example, [25], [26], [27], [28]), generating subjective ground truth sets which are required when there is no objective ground truth available. Although this can be time consuming and expensive, and can be inaccurate and corrupted [23], it does indicate a worthy attempt to solve this problem and indicates a move towards user-oriented evaluations.…”

Section: Evaluation Of Music Information Retrieval Systemsmentioning

confidence: 99%

“…Music is a complex concept and, although there are some similarities between music and text in terms of search, there are also some significant differences caused by this complexity [1]. Although known item searching for music can be dealt with by searching metadata using existing text search techniques, human subjectivity and variability within the music itself make it very difficult to search for unknown items.…”

Section: Introductionmentioning

confidence: 99%

Creative professional users’ musical relevance criteria

Inskip

MacFarlane

Rafferty

2010

Journal of Information Science

View full text Add to dashboard Cite

This is the unspecified version of the paper.This version of the publication may differ from the final published version. Permanent repository link Andy MacFarlane Department of Information Science, City University London, UK Pauline Rafferty Department of Information Studies, University of Aberystwyth, Wales AbstractAlthough known item searching for music can be dealt with by searching metadata using existing text search techniques, human subjectivity and variability within the music itself make it very difficult to search for unknown items. This paper examines these problems within the context of text retrieval and music information retrieval. The focus is on ascertaining a relationship between music relevance criteria and those relating to relevance judgements in text retrieval. A data-rich collection of relevance judgements by creative professionals searching for unknown musical items to accompany moving images using real world queries is analysed. The participants in our observations are found to take a socio-cognitive approach and use a range of content and context based criteria. These criteria correlate strongly with those arising from previous text retrieval studies despite the many differences between music and text in their actual content.

show abstract

The Scientific Evaluation of Music Information Retrieval Systems: Foundations and Future

Cited by 61 publications

References 11 publications

A study of the information needs of the users of a folk music library and the implications for the design of a digital library system

A study of the information needs of the users of a folk music library and the implications for the design of a digital library system

A typology of music information for studies on information seeking

Creative professional users’ musical relevance criteria

Contact Info

Product

Resources

About