Emotion recognition using spatial-temporal EEG features through convolutional graph attention network

Abstract

Objective. Constructing an efficient human emotion recognition model based on electroencephalogram (EEG) signals is of great significance for realizing emotional brain computer interaction and improving machine intelligence. Approach. In this paper, we present a spatial-temporal feature fused convolutional graph attention network (STFCGAT) model based on multi-channel EEG signals for human emotion recognition. First, we combined the single-channel differential entropy (DE) feature with the cross-channel functional connectivity (FC) feature to extract both the temporal variation and spatial topological information of EEG. After that, a novel convolutional graph attention network was used to fuse the DE and FC features and further extract higher-level graph structural information with sufficient expressive power for emotion recognition. Furthermore, we introduced a multi-headed attention mechanism in graph neural networks to improve the generalization ability of the model. Main results. We evaluated the emotion recognition performance of our proposed model on the public SEED and DEAP datasets, which achieved a classification accuracy of 99.11±0.83% and 94.83±3.41% in subject-dependent and subject-independent experiments on SEED dataset, and achieved an accuracy of 91.19±1.24% and 92.03±4.57% for discrimination of arousal and valence in subject-independent experiments on DEAP dataset. Notably, our model achieved state-of-the-art (SOTA) performance on cross-subject emotion recognition task for both datasets. In addition, we gained an insight into the proposed frame by both the ablation experiments and the analysis of spatial patterns of FC and DE features. Significance. All these results prove the effectiveness of the STFCGAT architecture for emotion recognition and also indicate that there are significant differences in the spatial-temporal characteristics of the brain under different emotional states.

A Comparative Study on Prominent Connectivity Features for Emotion Recognition From EEG

“…Tang et al used PCC and NMI to extract feature and investigated a DL model (combining a compact convolutional network and an auxiliary fully connected network) which can effectively reconstruct high-density EEG and holds potentials in EEG big data applications [56]. Convolutional graph attention network was used by Li et al [57], where PCC was used to extract functional connectivity feature. The study [57] indicates that functional connection strength under high Valence is higher than that under low Valence, i.e., the positive emotion shows stronger functional connectivity than negative emotions.…”

“…Convolutional graph attention network was used by Li et al [57], where PCC was used to extract functional connectivity feature. The study [57] indicates that functional connection strength under high Valence is higher than that under low Valence, i.e., the positive emotion shows stronger functional connectivity than negative emotions.…”

“…Recently, EEG channel connectivity features have been widely used in ER, and several studies have demonstrated the effectiveness of connectivity features for ER. Among the functional connectivity, PCC, PLV, and MI were used frequently in the literature, such as PCC was used by the study [7], [8], [21], [23], [41], [43], [49], [51], [52], [56], [57], [58],…”

“…In the pre-processed version of the dataset, the EEG channels are ordered as listed in Table V. [7], [8], [21], [23], [41], [43], [56], [58], [49], [51], [52], [57] Functional non-linear (Phase-based measures) PLV [43] 2015 [7], [21], [23], [25], [58]- [64], [51] Functional non-linear (Information theoretic measures) MI [39] 2010 [24], [39], [42], [43], [46] NMI [45] 2019 [45], [56] Effective non-linear (Information theoretic measures) TE [7] 2020 [5], [7], [23], [51], [65], [66]…”

LSTM-enhanced multi-view dynamical emotion graph representation for EEG signal recognition

“…To this end, graph convolutional neural networks [15][16][17] were used to deal with irregular and non-Euclidean data. It is an extension of the traditional convolutional neural network.…”