Emotion recognition with convolutional neural network and EEG-based EFDMs

Abstract

Electroencephalogram (EEG), as a direct response to brain activity, can be used to detect mental states and physical conditions. Among various EEG-based emotion recognition studies, due to the nonlinear, non-stationary and the individual difference of EEG signals, traditional recognition methods still have the disadvantages of complicated feature extraction and low recognition rates. Thus, this paper first proposes a novel concept of electrode-frequency distribution maps (EFDMs) with short-time Fourier transform (STFT). Residual block based deep convolutional neural network (CNN) is proposed for automatic feature extraction and emotion classification with EFDMs. Aim at the shortcomings of the small amount of EEG samples and the challenge of differences in individual emotions, which makes it difficult to construct a universal model, this paper proposes a cross-datasets emotion recognition method of deep model transfer learning. Experiments carried out on two publicly available datasets. The proposed method achieved an average classification score of 90.59% based on a short length of EEG data on SEED, which is 4.51% higher than the baseline method. Then, the pre-trained model was applied to DEAP through deep model transfer learning with a few samples, resulted an average accuracy of 82.84%. Finally, this paper adopts the gradient weighted class activation mapping (Grad-CAM) to get a glimpse of what features the CNN has learned during training from EFDMs and concludes that the high frequency bands are more favorable for emotion recognition. Proposed a novel concept of EFDMs with STFT based on multiple channel EEG signals. Constructed four residual blocks based CNN for emotion recognition. Performed cross-datasets emotion recognition based on deep model transfer learning. Studied the number of training samples used for cross-datasets emotion recognition. Obtained the key EEG information automatically based on EFDMs and Grad-CAM.

Emotion Recognition From EEG Signal Focusing on Deep Learning and Shallow Learning Techniques

“…However, we discussed shallow and deep learning-based systems with a proper comparison, including the application of recently developed novel deep learning models. After all, we conducted a review that comprises 12 recent sources of the dataset with eight information features Among many deep machine learning-based reviewed systems, most of the systems are developed either direct CNN based [98]- [100], [102], [106] or by modified CNN based system. The modified CNN methods are like DE-CNN [97], HCNN [90], MC-CNN [104], PCRNN [107], RA-CNN [101].…”

High-wearable EEG-Based Detection of Emotional Valence for Scientific Measurement of Emotions

“…Samples with number of subjects below this threshold were considered not statistically significant. Studies claiming the best accuracy on emotional valence assessment are based on public EEG signal datasets: SEED [24][25][26][27][28][29] , DEAP [25][26][27][28][30][31][32][33][34][35][36][37][38][39][40] , and DREAMER 29,37,38 .…”

Research on Emotion Recognition Based on Deep Learning for Mental Health

“…The CNN model was simulated and analyzed using MATLAB software [17]. The experiment was carried out on a laboratory server.…”