Emotional classification of music using neural networks with the MediaEval dataset

Abstract

The proven ability of music to transmit emotions provokes the increasing interest in the development of new algorithms for music emotion recognition (MER). In this work, we present an automatic system of emotional classification of music by implementing a neural network. This work is based on a previous implementation of a dimensional emotional prediction system in which a Multilayer Perceptron (MLP) was trained with the freely available MediaEval database. Although these previous results are good in terms of the metrics of the prediction values, they are not good enough to obtain a classification by quadrant based on the valence and arousal values predicted by the neural network, mainly due to the imbalance between classes in the dataset. To achieve better classification values a pre-processing phase was implemented to stratify and balance the dataset. Three different classifiers have been compared: Linear Support Vector Machine (SVM), Random Forest and MLP. The best results are obtained with the MLP. An averaged F-measure of 50% is obtained in a four-quadrant classification schema. Two binary classification approaches are also presented: one vs. rest (OvR) approach in four-quadrants and binary classifier in valence and arousal. The OvR approach has an average F-measure of 69%, and the second one obtained F-measure of 73% and 69% in valence and arousal respectively. Finally, a dynamic classification analysis with different time windows was performed using the temporal annotation data of the MediaEval database. The results obtained show that the classification F-measures in four quadrants are practically constant, regardless of the duration of the time window. Also, this work reflects some limitations related to the characteristics of the dataset, including: size, class balance, quality of the annotations and the sound features available.

The Contribution of Lyrics and Acoustics to Collaborative Understanding of Mood

“…Recently, there have been a body of works that applied deep neural network models to capture the association of mood/emotion and song by taking advantage of audio features (Saari et al 2013;Panda 2019;Korzeniowski et al 2020;Panda, Malheiro, and Paiva 2020;Medina, Beltrán, and Baldassarri 2020), lyrics features (Fell et al 2019;Hrustanović, Kavšek, and Tkalčič 2021) as well as both lyrics and audio (Delbouys et al 2018;Parisi et al 2019;Wang, Syu, and Wongchaisuwat 2021;Bhattacharya and Kadambari 2018) features. Delbouys et al classify mood of a song to either 'arousal' or 'valence' by utilizing a 100-dimensional word2vec embedding vector that is trained on 1.6 million lyrics in several different neural architectures such as GRU, LSTM, Convolutional Networks for their lyrics-based model.…”

Predicting the Preference for Sad Music: The Role of Gender, Personality, and Audio Features

“…MER, a field investigating computational models for automatically recognizing the perceptual emotion of music, has made great progress in recent decades [91]- [94]. Kim et al noted that MER usually constitutes a process of extracting music features from original music, forming the relations between music features and perceived emotions, and predicting the emotion of untagged music [95].…”

Automatic Music Mood Classification using Multi-class Support Vector Machine based on Hybrid Spectral Features

“…Medina [13] developed an emotional classification of music using neural networks with the MediaEval dataset resulted the values of valence and arousal showed imbalanced classification results. However, the required characteristics of the dataset that are: size, class balance and quality of the annotations, still needed more improvement for achieving good performance in terms of accuracy.…”