Human Emotion Recognition with Electroencephalographic Multidimensional Features by Hybrid Deep Neural Networks

Abstract

Featured Application: The method presented in this study can be applied in many fields, such as mental health care, entertainment consumption behavior, society safety, and so on. For example, in the mental health care field, an automatic emotion analysis system can be constructed with our method to monitor the emotional variation of the subjects. With accurate and objective emotion analysis results from EEG signals, our method can provide useful treatment effect information to the medical staff.Abstract: The aim of this study is to recognize human emotions by electroencephalographic (EEG) signals. The innovation of our research methods involves two aspects: First, we integrate the spatial characteristics, frequency domain, and temporal characteristics of the EEG signals, and map them to a two-dimensional image. With these images, we build a series of EEG Multidimensional Feature Image (EEG MFI) sequences to represent the emotion variation with EEG signals. Second, we construct a hybrid deep neural network to deal with the EEG MFI sequences to recognize human emotional states where the hybrid deep neural network combined the Convolution Neural Networks (CNN) and Long Short-Term-Memory (LSTM) Recurrent Neural Networks (RNN). Empirical research is carried out with the open-source dataset DEAP (a Dataset for Emotion Analysis using EEG, Physiological, and video signals) using our method, and the results demonstrate the significant improvements over current state-of-the-art approaches in this field. The average emotion classification accuracy of each subject with CLRNN (the hybrid neural networks that we proposed in this study) is 75.21%.

Improved Deep Feature Learning by Synchronization Measurements for Multi-Channel EEG Emotion Recognition

“…If the order of two variables is changed, then the value of MIC remains unchanged. [13,24]. To characterize the spatial information and global synchronization information of all EEG channels, an MIC gray image is constructed as the feature for all EEG channels.…”

“…From equation (13), the PCA network offers low computational complexity. Compared with the PCA network, the filters of CNN require a numerical optimization solver during the training phase, which significantly increases the computational complexity.…”

“…erefore, investigating the relationship between physiological signals, such as electroencephalogram (EEG) and emotion states, has garnered considerable attention [11][12][13]. A critical step in the EEG-based emotion recognition task is to extract features related to human emotional states from multichannel EEG signals.…”

Emotion Classification through Nonlinear EEG Analysis Using Machine Learning Methods

“…There are several ways to stimulate emotions such as images, video clips, memories, music, etc however, based on previous works the audio-visual elicitation has its advantages. [40][41][42][43] Since videos contain both scene and audio, individuals are more exposed to real and strong emotional changes. Therefore, we employed video clips to stimulate participants' feelings.…”

Smart Healthcare

“…In [11], Li et al reported their work on detecting human emotion from EEG signals via hybrid deep neural networks. They did so in two steps.…”