Automated Music Emotion Recognition: A Systematic Evaluation

Abstract

Automated music emotion recognition (MER) is a challenging task in Music Information Retrieval (MIR) with wide-ranging applications. Some recent studies pose MER as a continuous regression problem in the ArousalValence (AV) plane. These consist of variations on a common architecture having a universal model of emotional response, a common repertoire of low-level audio features, a bag-of-frames approach to audio analysis, and relatively small data sets. These approaches achieve some success at MER and suggest that further improvements are possible with current technology.Our contribution to the state of the art is to examine just how far one can go within this framework, and to investigate what the limitations of this framework are. We present the results of a systematic study conducted in an attempt to maximize the prediction performance of an automated MER system using the architecture described. We begin with a carefully constructed data set, emphasizing quality over quantity. We address affect induction rather than affect attribution. We consider a variety of algorithms at each stage of the training process, from preprocessing to feature selection and model selection, and we report the results of extensive testing.We found that: (1) none of the variations we considered leads to a substantial improvement in performance, which we present as evidence of a limit on what is achievable under this architecture, and (2) the size of the small data sets that are commonly used in the MER literature limits the possibility of improving the set of features used in MER due to the phenomenon of Subset Selection Bias. We conclude with some proposals for advancing the state of the art.

Exploiting online music tags for music emotion classification

“…In addition to the size of the training dataset (which is related to the generality of the training instances), another factor that impedes the progress of MEC is the so-called semantic gap between the object feature level and the human cognitive level of emotion perception Huq et al 2010;Yang and Chen 2011a]. Existing audio features can only characterize the power spectrum of a music piece but fail to represent the emotion perceived by humans, which makes emotion classification even more challenging.…”

Bridge the semantic gap between pop music acoustic feature and emotion: Build an interpretable model

“…Although they got a slight improvement by using RReliefF, it may have been a subject to Subset Selection Bias (SSB) [35]. In Huq et al's [2] work, feature selection algorithms including filter methods (RReliefF and CFS) and wrapper methods (biased FS and biased BE) were applied to select a subset of the 160 features to improve regression performance. They discovered that above feature selection algorithms do not improve the performance in his system due to the phenomenon of SSB, which make train model tend to overfit.…”

“…The categorical approach describes mood using adjective. Plenty of work [2,3] is based on Hevner's checklist [4], which contains 67 adjectives. The adjectives are not treated individually, but are arranged into eight groups.…”

Feature selection and feature learning in arousal dimension of music emotion by using shrinkage methods

“…Although he gets a slight improvement from using RReliefF, it may be subject to Subset Selection Bias (SSB) [10]. In Arefin Huq's [11] work, feature selection algorithms including filter methods (RReliefF and CFS) and wrapper methods (biased FS and biased BE) are applied to select a subset of the 160 features to improve regression performance. He discovers that above feature selection algorithms do not improve the performance in his system due to the phenomenon of SSB, which make train model tend to over-fitting.…”