Linking Multi-Layer Dynamical GCN With Style-Based Recalibration CNN for EEG-Based Emotion Recognition

Abstract

Electroencephalography (EEG)-based emotion computing has become one of the research hotspots of human-computer interaction (HCI). However, it is difficult to effectively learn the interactions between brain regions in emotional states by using traditional convolutional neural networks because there is information transmission between neurons, which constitutes the brain network structure. In this paper, we proposed a novel model combining graph convolutional network and convolutional neural network, namely MDGCN-SRCNN, aiming to fully extract features of channel connectivity in different receptive fields and deep layer abstract features to distinguish different emotions. Particularly, we add style-based recalibration module to CNN to extract deep layer features, which can better select features that are highly related to emotion. We conducted two individual experiments on SEED data set and SEED-IV data set, respectively, and the experiments proved the effectiveness of MDGCN-SRCNN model. The recognition accuracy on SEED and SEED-IV is 95.08 and 85.52%, respectively. Our model has better performance than other state-of-art methods. In addition, by visualizing the distribution of different layers features, we prove that the combination of shallow layer and deep layer features can effectively improve the recognition performance. Finally, we verified the important brain regions and the connection relationships between channels for emotion generation by analyzing the connection weights between channels after model learning.

Emotion Classification from Multi-Band Electroencephalogram Data Using Dynamic Simplifying Graph Convolutional Network and Channel Style Recalibration Module

“…Lu et al performed target detection on video-induced EEG signals, extracted EEG spatio-temporal features with graph convolution, and improved the SRM to select features with larger contributions [ 26 ]. Bao et al added an SRM to a CNN to extract deep features and select features with high correlation with emotion and obtained improved results in a subject-depended experiment on the SEED dataset (95.08%) [ 27 ]. Therefore, the introduction of the SRM to adaptively recalibrate the intermediate features learned from sub-bands can incorporate them into the feature maps, thereby minimizing the loss of information and improving the feature extraction ability of the network [ 27 ].…”

Graph Neural Network-Based EEG Classification: A Survey

“…Despite most of the surveyed papers being relatively recent, a wide range of GNN-based methods has already been proposed to classify EEG signals in a diverse set of tasks, such as emotion recognition, brain-computer interfaces, and psychological and neurodegenerative disorders and diseases [46], [53], [54], [56], [58], [61], [70], [72], [75], [83], [89], [106] Chebyshev Graph Convolution ✗ ✓ ✗ [49], [51], [55], [57], [59], [66], [67], [69], [71], [74], [76]- [78], [80], [82], [85], [90], [97], [99], [104] Graph Attention Network ✓ ✗ ✗ [60], [62], [73], [84], [88], [94], [98] This survey categorises the proposed GNN models in terms of their inputs and modules. Specifically, these are brain graph structure, node features and their preprocessing, GCN layers, node pooling mechanisms, and formation of graph embeddings.…”

“…These methods can be generally categorised as learnable or pre-defined. Multiple/Combined graph definitions -- [47], [49], [53], [54], [57]- [59], [61]- [64], [67], [69], [72], [79], [79], [81], [82], [87], [92], [102] [51], [53], [55], [57], [71], [72], [75], [78], [81], [82], [87], [89], [90], [92], [93], [93], [95]- [99], [101], [102] Raw signal ✓ ✗ ✗…”

A Comparative Study on Prominent Connectivity Features for Emotion Recognition From EEG

“…Recently, EEG channel connectivity features have been widely used in ER, and several studies have demonstrated the effectiveness of connectivity features for ER. Among the functional connectivity, PCC, PLV, and MI were used frequently in the literature, such as PCC was used by the study [7], [8], [21], [23], [41], [43], [49], [51], [52], [56], [57], [58],…”

“…In the pre-processed version of the dataset, the EEG channels are ordered as listed in Table V. [7], [8], [21], [23], [41], [43], [56], [58], [49], [51], [52], [57] Functional non-linear (Phase-based measures) PLV [43] 2015 [7], [21], [23], [25], [58]- [64], [51] Functional non-linear (Information theoretic measures) MI [39] 2010 [24], [39], [42], [43], [46] NMI [45] 2019 [45], [56] Effective non-linear (Information theoretic measures) TE [7] 2020 [5], [7], [23], [51], [65], [66]…”