A Comparison of Approaches to Timbre Descriptors in Music Information Retrieval and Music Psychology

Siedenburg, Kai; Fujinaga, Ichiro; McAdams, Stephen

doi:10.1080/09298215.2015.1132737

Cited by 52 publications

(38 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, machine learning techniques that best recognize musical instruments, styles, etc. are not based on a few recurrent dimensions: they use and create whichever pieces statistical information fit a given classification task, relying on high-dimensional representations [94–97]. Another important result is that prior exposure to complex, abstract noise textures with little structure improves the performance at a subsequent repetition detection task [98].…”

Section: Discussionmentioning

confidence: 99%

Vocal Imitations of Non-Vocal Sounds

et al. 2016

View full text Add to dashboard Cite

Imitative behaviors are widespread in humans, in particular whenever two persons communicate and interact. Several tokens of spoken languages (onomatopoeias, ideophones, and phonesthemes) also display different degrees of iconicity between the sound of a word and what it refers to. Thus, it probably comes at no surprise that human speakers use a lot of imitative vocalizations and gestures when they communicate about sounds, as sounds are notably difficult to describe. What is more surprising is that vocal imitations of non-vocal everyday sounds (e.g. the sound of a car passing by) are in practice very effective: listeners identify sounds better with vocal imitations than with verbal descriptions, despite the fact that vocal imitations are inaccurate reproductions of a sound created by a particular mechanical system (e.g. a car driving by) through a different system (the voice apparatus). The present study investigated the semantic representations evoked by vocal imitations of sounds by experimentally quantifying how well listeners could match sounds to category labels. The experiment used three different types of sounds: recordings of easily identifiable sounds (sounds of human actions and manufactured products), human vocal imitations, and computational “auditory sketches” (created by algorithmic computations). The results show that performance with the best vocal imitations was similar to the best auditory sketches for most categories of sounds, and even to the referent sounds themselves in some cases. More detailed analyses showed that the acoustic distance between a vocal imitation and a referent sound is not sufficient to account for such performance. Analyses suggested that instead of trying to reproduce the referent sound as accurately as vocally possible, vocal imitations focus on a few important features, which depend on each particular sound category. These results offer perspectives for understanding how human listeners store and access long-term sound representations, and sets the stage for the development of human-computer interfaces based on vocalizations.

show abstract

Section: Discussionmentioning

confidence: 99%

Vocal Imitations of Non-Vocal Sounds

et al. 2016

View full text Add to dashboard Cite

show abstract

“…Yet, questions regarding the most perceptually relevant acoustical descriptors of the spectral envelope prevail (cf. Siedenburg et al, 2016;Almeida et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

Timbral Shepard-illusion reveals ambiguity and context sensitivity of brightness perception

Siedenburg

2018

The Journal of the Acoustical Society of America

Self Cite

View full text Add to dashboard Cite

Recent research has described strong effects of prior context on the perception of ambiguous pitch shifts of Shepard tones [Chambers, Akram, Adam, Pelofi, Sahani, Shamma, and Pressnitzer (2017). Nat. Commun. 8, 15027]. Here, similar effects are demonstrated for brightness shift judgments of harmonic complexes with cyclic spectral envelope components and fixed fundamental frequency. It is shown that frequency shifts of the envelopes are perceived as systematic shifts of brightness. Analogous to the work of Chambers et al., the perceptual ambiguity of half-octave shifts resolves with the presentation of prior context tones. These results constitute a context effect for the perceptual processing of spectral envelope shifts and indicate so-far unknown commonalities between pitch and timbre perception.

show abstract

“…Consistent with this, a recent paper reviews the disparities in outlook and purpose (even towards timbre) between music psychology and other fields, and suggests approaches to bring them together productively [34]. The present study begins to address such disparities by investigating predictors amongst acoustic factors, particularly spectral features, which might model and potentially explain listeners' perceptions.…”

Section: Introductionmentioning

confidence: 65%

What Constitutes a Phrase in Sound-Based Music? A Mixed-Methods Investigation of Perception and Acoustics

2016

View full text Add to dashboard Cite

Phrasing facilitates the organization of auditory information and is central to speech and music. Not surprisingly, aspects of changing intensity, rhythm, and pitch are key determinants of musical phrases and their boundaries in instrumental note-based music. Different kinds of speech (such as tone- vs. stress-languages) share these features in different proportions and form an instructive comparison. However, little is known about whether or how musical phrasing is perceived in sound-based music, where the basic musical unit from which a piece is created is commonly non-instrumental continuous sounds, rather than instrumental discontinuous notes. This issue forms the target of the present paper. Twenty participants (17 untrained in music) were presented with six stimuli derived from sound-based music, note-based music, and environmental sound. Their task was to indicate each occurrence of a perceived phrase and qualitatively describe key characteristics of the stimulus associated with each phrase response. It was hypothesized that sound-based music does elicit phrase perception, and that this is primarily associated with temporal changes in intensity and timbre, rather than rhythm and pitch. Results supported this hypothesis. Qualitative analysis of participant descriptions showed that for sound-based music, the majority of perceived phrases were associated with intensity or timbral change. For the note-based piano piece, rhythm was the main theme associated with perceived musical phrasing. We modeled the occurrence in time of perceived musical phrases with recurrent event ‘hazard’ analyses using time-series data representing acoustic predictors associated with intensity, spectral flatness, and rhythmic density. Acoustic intensity and timbre (represented here by spectral flatness) were strong predictors of perceived musical phrasing in sound-based music, and rhythm was only predictive for the piano piece. A further analysis including five additional spectral measures linked to timbre strengthened the models. Overall, results show that even when little of the pitch and rhythm information important for phrasing in note-based music is available, phrasing is still perceived, primarily in response to changes of intensity and timbre. Implications for electroacoustic music composition and music recommender systems are discussed.

show abstract

A Comparison of Approaches to Timbre Descriptors in Music Information Retrieval and Music Psychology

Cited by 52 publications

References 72 publications

Vocal Imitations of Non-Vocal Sounds

Vocal Imitations of Non-Vocal Sounds

Timbral Shepard-illusion reveals ambiguity and context sensitivity of brightness perception

What Constitutes a Phrase in Sound-Based Music? A Mixed-Methods Investigation of Perception and Acoustics

Contact Info

Product

Resources

About