Emotion classification from speech signal based on empirical mode decomposition and non-linear features

Krishnan, Palani Thanaraj; Raj, Alex Noel Joseph; Rajangam, Vijayarajan

doi:10.1007/s40747-021-00295-z

Cited by 54 publications

(15 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The sentiment and emotion analysis of multi-modal dialogue such as video, audio, image attracts more and more attention [7,[22][23][24]. To improve the performance of the classifier, the image-text pair in the existing unimodal data is usually used as a multi-modal comprehensive data set [25].…”

Section: Multimodal Sentiment Analysismentioning

confidence: 99%

A multi-modal and multi-scale emotion-enhanced inference model based on fuzzy recognition

Qiu

Yan

2021

Complex Intell. Syst.

View full text Add to dashboard Cite

Only the label corresponding to the maximum value of the fully connected layer is used as the output category when a neural network performs classification tasks. When the maximum value of the fully connected layer is close to the sub-maximum value, the classification obtained by considering only the maximum value and ignoring the sub-maximum value is not completely accurate. To reduce the noise and improve classification accuracy, combining the principles of fuzzy reasoning, this paper integrates all the output results of the fully connected layer with the emotional tendency of the text based on the dictionary to establish a multi-modal fuzzy recognition emotion enhancement model. The provided model considers the enhancement effect of negative words, degree adverbs, exclamation marks, and question marks based on the smallest subtree on the emotion of emotional words, and defines the global emotional membership function of emojis based on the corpus. Through comparing the results of CNN, LSTM, BiLSTM and GRU on Weibo and Douyin, it is shown that the provided model can effectively improve the text emotion recognition when the neural network output result is not clear, especially for long texts.

show abstract

Section: Multimodal Sentiment Analysismentioning

confidence: 99%

A multi-modal and multi-scale emotion-enhanced inference model based on fuzzy recognition

Qiu

Yan

2021

Complex Intell. Syst.

View full text Add to dashboard Cite

show abstract

“…In addition, this preliminary work adds another path for future development and applications of IA and IF in different domains where time frequency analysis is required. The EMD method has been used in speech recognition systems [ 30 , 31 , 32 ] and human emotion recognition system [ 33 ]. The EMD method has also been used to perform classification of respiratory sound in conjunction with FFT to extract features [ 34 ].…”

Section: Introductionmentioning

confidence: 99%

Empirical Mode Decomposition-Based Feature Extraction for Environmental Sound Classification

Ahmed

Serrestou

Raoof

et al. 2022

Sensors

View full text Add to dashboard Cite

In environment sound classification logs, Mel band energies (MBEs) are considered as the most successful and commonly used features for classification. The underlying algorithm, fast Fourier transform (FFT), is valid under certain restrictions. In this study, we address these limitations of Fourier transform and propose a new method to extract log Mel band energies using amplitude modulation and frequency modulation. We present a comparative study between traditionally used log Mel band energy features extracted by Fourier transform and log Mel band energy features extracted by our new approach. This approach is based on extracting log Mel band energies from estimation of instantaneous frequency (IF) and instantaneous amplitude (IA), which are used to construct a spectrogram. The estimation of IA and IF is made by associating empirical mode decomposition (EMD) with the Teager–Kaiser energy operator (TKEO) and the discrete energy separation algorithm. Later, a Mel filter bank is applied to the estimated spectrogram to generate EMD-TKEO-based MBEs, or simply, EMD-MBEs. In addition, we employ the EMD method to remove signal trends from the original signal and generate another type of MBE, called S-MBEs, using FFT and a Mel filter bank. Four different datasets were utilised and convolutional neural networks (CNN) were trained using features extracted from Fourier transform-based MBEs (FFT-MBEs), EMD-MBEs, and S-MBEs. In addition, CNNs were trained with an aggregation of all three feature extraction techniques and a combination of FFT-MBEs and EMD-MBEs. Individually, FFT-MBEs achieved higher accuracy compared to EMD-MBEs and S-MBEs. In general, the system trained with the combination of all three features performed slightly better compared to the system trained with the three features separately.

show abstract

“…Extract features from these five frequency bands. These features usually include power spectral density (PSD) [ 11 ], differential entropy (DE) [ 12 , 13 ], differential asymmetry (DASM) [ 14 ], and rational asymmetry (RASM) [ 15 , 16 ]. EEG signal is a non-stationary and non-linear random signal [ 17 ].…”

Section: Introductionmentioning

confidence: 99%

Multidimensional Feature in Emotion Recognition Based on Multi-Channel EEG Signals

Liu

et al. 2022

Entropy

View full text Add to dashboard Cite

As a major daily task for the popularization of artificial intelligence technology, more and more attention has been paid to the scientific research of mental state electroencephalogram (EEG) in recent years. To retain the spatial information of EEG signals and fully mine the EEG timing-related information, this paper proposes a novel EEG emotion recognition method. First, to obtain the frequency, spatial, and temporal information of multichannel EEG signals more comprehensively, we choose the multidimensional feature structure as the input of the artificial neural network. Then, a neural network model based on depthwise separable convolution is proposed, extracting the input structure’s frequency and spatial features. The network can effectively reduce the computational parameters. Finally, we modeled using the ordered neuronal long short-term memory (ON-LSTM) network, which can automatically learn hierarchical information to extract deep emotional features hidden in EEG time series. The experimental results show that the proposed model can reasonably learn the correlation and temporal dimension information content between EEG multi-channel and improve emotion classification performance. We performed the experimental validation of this paper in two publicly available EEG emotional datasets. In the experiments on the DEAP dataset (a dataset for emotion analysis using EEG, physiological, and video signals), the mean accuracy of emotion recognition for arousal and valence is 95.02% and 94.61%, respectively. In the experiments on the SEED dataset (a dataset collection for various purposes using EEG signals), the average accuracy of emotion recognition is 95.49%.

show abstract

Emotion classification from speech signal based on empirical mode decomposition and non-linear features

Cited by 54 publications

References 46 publications

A multi-modal and multi-scale emotion-enhanced inference model based on fuzzy recognition

A multi-modal and multi-scale emotion-enhanced inference model based on fuzzy recognition

Empirical Mode Decomposition-Based Feature Extraction for Environmental Sound Classification

Multidimensional Feature in Emotion Recognition Based on Multi-Channel EEG Signals

Contact Info

Product

Resources

About