Using listener-based perceptual features as intermediate representations in music information retrieval

Abstract

The notion of perceptual features is introduced for describing general music properties based on human perception. This is an attempt at rethinking the concept of features, aiming to approach the underlying human perception mechanisms. Instead of using concepts from music theory such as tones, pitches, and chords, a set of nine features describing overall properties of the music was selected. They were chosen from qualitative measures used in psychology studies and motivated from an ecological approach. The perceptual features were rated in two listening experiments using two different data sets. They were modeled both from symbolic and audio data using different sets of computational features. Ratings of emotional expression were predicted using the perceptual features. The results indicate that (1) at least some of the perceptual features are reliable estimates; (2) emotion ratings could be predicted by a small combination of perceptual features with an explained variance from 75% to 93% for the emotional dimensions activity and valence; (3) the perceptual features could only to a limited extent be modeled using existing audio features. Results clearly indicated that a small number of dedicated features were superior to a "brute force" model using a large number of general audio features.

Predicting the perception of performed dynamics in music audio with ensemble learning

“…The resulting reliability of the mean estimate across listeners was high [Cronbach's alpha (CA) ¼ 0.94-0.95 for both groups]. For more details about the datasets and procedure we refer to the studies by Friberg et al (2014), , and .…”

“…The overall perceived performed dynamics (as well as eight other perceptual features) was rated for both datasets on a quasi-continuous scale by two groups of 20 and 21 listeners, respectively, in two previous experiments (Friberg et al, 2014). In each experiment, the performed dynamics (along with several other perceptual features) of each musical example was rated on a scale ranging from soft (1) to loud (10).…”

“…Previously, we have studied performed dynamics as part of a larger investigation of the concept of perceptual features; both regarding basic perception and for computational modeling in MIR applications Friberg et al, 2014). The ground truth annotations consisted of listener ratings of overall performed dynamics for a variety of music examples.…”

“…Two different datasets of music audio recordings were used in the study, for which perceptual ratings of performed dynamics had been collected in two previous experiments (Friberg et al, 2014). The first dataset contains 100 audio examples of popular music (average length 30 s) that were originally produced in the MIDI format and then converted to audio.…”

“…II. In the previous study, 25 audio features were used for the prediction (Friberg et al, 2014). The features were calculated using the MIRToolbox (Lartillot and Toiviainen, 2007), (number of features in parenthesis): MFCCs (13), zero crossings (1), brightness (3), spectral centroid (1), spectral spread, skewness, kurtosis, and flatness (4), spectral roll-off (2), and spectral flux (SF) (1).…”