Core-Brain-Network-Based Multilayer Convolutional Neural Network for Emotion Recognition

Gao, Zhong-Ke; Li, Rumei; Ma, Chao; Rui, Lin-Ge; Sun, Xinlin

doi:10.1109/tim.2021.3090164

Cited by 23 publications

(7 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on different EEG rhythms [29] (δ (1-3 Hz), θ (4-7 Hz), α (8-13 Hz), β (14-30 Hz), and γ (31-50 Hz)), the wavelet coefficients can be divided into five frequency bands. For each EEG rhythm, the wavelet coefficients can be aggregated according to the boundaries of each frequency band by the following equation [30]:…”

Section: B Brain Network Construction Blockmentioning

confidence: 99%

“…For two aggregated coefficients sequences of two EEG channels within same frequency band, they are first sorted according to the same sorting rules to obtain two ranking sets X and Y . A difference set D of ranked elements can be obtained by subtracting X and Y in element-wise order, in which the k th element in D is calculated as [30]: Then the Spearman correlation coefficient ρ can be obtained by the following equation [30]:…”

Section: B Brain Network Construction Blockmentioning

confidence: 99%

See 1 more Smart Citation

A Brain Network Analysis-Based Double Way Deep Neural Network for Emotion Recognition

Niu

Sun

et al. 2023

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Constructing reliable and effective models to recognize human emotional states has become an important issue in recent years. In this article, we propose a double way deep residual neural network combined with brain network analysis, which enables the classification of multiple emotional states. To begin with, we transform the emotional EEG signals into five frequency bands by wavelet transform and construct brain networks by inter-channel correlation coefficients. These brain networks are then fed into a subsequent deep neural network block which contains several modules with residual connection and enhanced by channel attention mechanism and spatial attention mechanism. In the second way of the model, we feed the emotional EEG signals directly into another deep neural network block to extract temporal features. At the end of the two ways, the features are concatenated for classification. To verify the effectiveness of our proposed model, we carried out a series of experiments to collect emotional EEG from eight subjects. The average accuracy of the proposed model on our emotional dataset is 94.57%. In addition, the evaluation results on public databases SEED and SEED-IV are 94.55% and 78.91%, respectively, demonstrating the superiority of our model in emotion recognition tasks.

show abstract

Section: B Brain Network Construction Blockmentioning

confidence: 99%

Section: B Brain Network Construction Blockmentioning

confidence: 99%

A Brain Network Analysis-Based Double Way Deep Neural Network for Emotion Recognition

Niu

Sun

et al. 2023

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

show abstract

“…Further, they used them to identify human emotions by a classifier. Recently, the research of emotion recognition based on EEG has also begun to use deep learning models to extract features [7,18]. For example, Li et al [8] used various deep learning models to extract features from EEG signals for emotion recognition.…”

Section: Related Workmentioning

confidence: 99%

“…The deep learning model can automatically extract features compared to traditional emotion recognition methods [6]. Gao et al [7] proposed an emotion classification method based on a multilayer convolutional neural network (CNN) and combining differential entropy (DE) and brain networks, which achieved an average accuracy of 91.45%. Li et al [8] proposed a multi-domain adaptive graph convolutional network (MD-AGCN) method, which combines the frequency and time domains to utilize the complementary information of electroencephalogram (EEG) signals.…”

Section: Introductionmentioning

confidence: 99%

Multimodal Emotion Recognition From EEG Signals and Facial Expressions

Wang

Zhang

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Emotion recognition has attracted a lot of attention in recent years. It is widely used in health care, teaching, human-computer interaction, and other fields. Various human emotional features are often used to recognize different emotions. Currently, there is more and more research on multimodal emotion recognition based on the fusion of multiple features. This paper proposes a deep learning model of multimodal emotion recognition based on the fusion of electroencephalogram (EEG) signals and facial expressions to achieve an excellent classification effect. First, the pre-trained convolution neural network (CNN) is used to extract the facial features from the facial expressions. Next, the attention mechanism is introduced to extract more critical facial frame features. Then, we apply CNNs to extract spatial features from original EEG signals, which use a local convolution kernel and a global convolution kernel to learn the features of left and right hemispheres channels and all EEG channels. After feature-level fusion, the fusion features of the facial expression features and EEG features are fed into the classifier for emotion recognition. This paper conducted experiments on the DEAP and MAHNOB-HCI datasets to evaluate the performance of the proposed model. The accuracy of valence dimension classification is 96.63%, and arousal dimension classification is 97.15% on the DEAP dataset, while 96.69% and 96.26% on the MAHNOB-HCI dataset. The experimental results show that the proposed model can effectively carry out emotion recognition.

show abstract

“…Meanwhile, Williams' investigation on deep learning and transfer learning used multi-layer perceptrons, convolutional neural networks (CNN), Long short-term memory (LSTM), and LSTM-fully convolutional networks to classify positive/negative emotions and achieved maximum accuracy of 64.36% [29]. Gao et al used a core-brain-network-based convolutional neural network with differential entropy and timefrequency representation of the EEG data to classify emotional states for 15 subjects of the SJTU emotion EEG dataset (SEED) [30] yielding an average accuracy of 91.45% [31].…”

Section: Introductionmentioning

confidence: 99%

Discriminating Fake and Real Smiles Using Electroencephalogram Signals With Convolutional Neural Networks

et al. 2022

View full text Add to dashboard Cite

Genuineness of smiles is of particular interest in the field of human emotions and social interactions. In this work, we develop an experimental protocol to elicit genuine and fake smile expressions on 28 healthy subjects. Then, we assess the type of smile expressions using electroencephalogram (EEG) signals with convolutional neural networks (CNNs). Five different architectures (CNN1, CNN2, CNN3, CNN4, and CNN5) were examined to differentiate between fake and real smiles. We transform the temporal EEG signals into normalized gray-scale images and perform three-way classification to classify fake smiles, genuine smiles, and neutral expressions in the form of subject-dependent classification. We achieved the highest classification accuracy of 90.4% using CNN1 for the full EEG spectrum. Likewise, we achieved classification accuracies of 87.4%, 88.3%, 89.7%, and 90.0% using Beta, Alpha, Theta, and Delta EEG bands respectively. This paper suggests that CNNs models, widely used in image classification problems, can provide an alternative approach for smile detection from physiological signals such as the EEG.

show abstract

Core-Brain-Network-Based Multilayer Convolutional Neural Network for Emotion Recognition

Cited by 23 publications

References 37 publications

A Brain Network Analysis-Based Double Way Deep Neural Network for Emotion Recognition

A Brain Network Analysis-Based Double Way Deep Neural Network for Emotion Recognition

Multimodal Emotion Recognition From EEG Signals and Facial Expressions

Discriminating Fake and Real Smiles Using Electroencephalogram Signals With Convolutional Neural Networks

Contact Info

Product

Resources

About