A Multimodal Music Emotion Classification Method Based on Multifeature Combined Network Classifier

Abstract

Aiming at the shortcomings of single network classification model, this paper applies CNN-LSTM (convolutional neural networks-long short-term memory) combined network in the field of music emotion classification and proposes a multifeature combined network classifier based on CNN-LSTM which combines 2D (two-dimensional) feature input through CNN-LSTM and 1D (single-dimensional) feature input through DNN (deep neural networks) to make up for the deficiencies of original single feature models. The model uses multiple convolution kernels in CNN for 2D feature extraction, BiLSTM (bidirectional LSTM) for serialization processing and is used, respectively, for audio and lyrics single-modal emotion classification output. In the audio feature extraction, music audio is finely divided and the human voice is separated to obtain pure background sound clips; the spectrogram and LLDs (Low Level Descriptors) are extracted therefrom. In the lyrics feature extraction, the chi-squared test vector and word embedding extracted by Word2vec are, respectively, used as the feature representation of the lyrics. Combining the two types of heterogeneous features selected by audio and lyrics through the classification model can improve the classification performance. In order to fuse the emotional information of the two modals of music audio and lyrics, this paper proposes a multimodal ensemble learning method based on stacking, which is different from existing feature-level and decision-level fusion methods, the method avoids information loss caused by direct dimensionality reduction, and the original features are converted into label results for fusion, effectively solving the problem of feature heterogeneity. Experiments on million song dataset show that the audio classification accuracy of the multifeature combined network classifier in this paper reaches 68%, and the lyrics classification accuracy reaches 74%. The average classification accuracy of the multimodal reaches 78%, which is significantly improved compared with the single-modal.

Emotion Label Enhancement via Emotion Wheel and Lexicon

“…Given a sentence with its logical emotion label, LLE extracts affective words from text and attaches the corresponding emotional information to the logical label, which is used to generate the final emotion distribution. Compared to the label enhancement method without using affective words 4 Mathematical Problems in Engineering [23], experimental results demonstrated that the emotion distribution generated by LLE has better performance [24]. In summary, the lack of textual emotional datasets with annotated emotion distributions is a distinct obstacle to the development of EDL.…”

“…Text emotion classification (recognition) is an important research topic with many promising novel applications [1], such as emotional human-computer interaction [2], intelligent customer service [3], music emotion classification [4], anticipating corporate financial performance [5], and online product review analysis [6]. e goal of text emotion recognition is to find out the writers' emotional states contained in sentences [1].…”

“…e goal of text emotion recognition is to find out the writers' emotional states contained in sentences [1]. In recent years, researchers have proposed a lot of effective and fruitful work in the field of text emotion classification [2][3][4][5][6].…”

Research on Intelligent Recognition and Classification Algorithm of Music Emotion in Complex System of Music Performance

“…Shan et al [11] established a recommendation model based on music emotion, mainly studying the emotion conveyed by movie music. Chen and Li [12] used a continuous emotional psychological model and regression model to predict the emotional value of music and used two fuzzy classifiers to measure the emotional intensity to identify the emotional content of music. Rajib Sarkar et al [13] proposed to use convolution neural network to identify music models and compared it with commonly used classifiers such as BP neural network.…”

Using Machine Learning Algorithm to Describe the Connection between the Types and Characteristics of Music Signal

“…But, as the music data are increasing, the error rate of the music classification model based on the neural network to construct a classification of low efficiency is higher and easier to fall into local minimum problem [2]; therefore, the neural network structure should be optimized to establish a music based on particle swarm algorithm to optimize the neural network classification model, speed up the neural network optimization ability, and improve the effect and the music classification accuracy. e music classification model based on particle swarm optimization neural network needs to extract music features, and the extracted music features can describe the music information [3] to effectively identify the music types.…”