Novel Audio Features for Music Emotion Recognition

Abstract

This work advances the music emotion recognition state-of-the-art by proposing novel emotionally-relevant audio features. 5We reviewed the existing audio features implemented in well-known frameworks and their relationships with the eight commonly 6 defined musical concepts. This knowledge helped uncover musical concepts lacking computational extractors, to which we propose 7 algorithms -namely related with musical texture and expressive techniques. To evaluate our work, we created a public dataset of 900 8 audio clips, with subjective annotations following Russell's emotion quadrants. The existent audio features (baseline) and the proposed 9 features (novel) were tested using 20 repetitions of 10-fold cross-validation. Adding the proposed features improved the F1-score to 10 76.4 percent (by 9 percent), when compared to a similar number of baseline-only features. Moreover, analysing the features relevance 11 and results uncovered interesting relations, namely the weight of specific features and musical concepts to each emotion quadrant, and 12 warrant promising new directions for future research in the field of music emotion recognition, interactive media, and novel music 13 interfaces.

Playing Technique Recognition by Joint Time–Frequency Scattering

“…Hidden Markov models (HMMs) were used in [9] for detecting glissando and in [7] to recognise portamento. Rule-based features introduced in [10] were specifically for glissando detection. Patterns of regularity across playing techniques motivate us to build a generic model for music playing technique recognition.…”

Pop Music Highlighter: Marking the Emotion Keypoints

“…In addition, considering up to 190 emotion classes makes it difficult to carefully investigate the dependency between emotions and song highlights. In the future, we hope to validate the emotion labels with human validation (e.g., as Panda et al (2018) did), and to reduce the number of emotion classes to facilitate in-depth analysis. For example, depending on the emotion a song tries to express, the highlight of the song may not be the chorus but somewhere else.…”

Come on Baby, Light My Fire: Sparking Further Research in Socio-Affective Mechanisms of Music Using Computational Advancements

“…However, computational methods for music emotion recognition have tended to favor certain features over others (e.g., timbre accounting for over 60%; Yang et al, 2019 ). Recently, researchers have begun to develop software packages for emotion recognition in music, which include fine-grained features such as specific textural shifts and articulations (Panda et al, 2018 ). Such software could provide important contextual affective information in existing joint music-making paradigms.…”