STSNet: a novel spatio-temporal-spectral network for subject-independent EEG-based emotion recognition

Li, Rui; Ren, Chao; Zhang, S.L.; Yang, Yikun; Zhao, Qida; Hou, Kechen; Yuan, Weiqi; Zhang, Xiaowei; Hu, Bin

doi:10.1007/s13755-023-00226-x

Cited by 9 publications

(4 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…BiSMSM [67] 61.88 ± 8.76 64.25 ± 6.86 MSD-SS-SAN [68] 74.08 72.74 SSTD [69] 76.81 81.64 STSNet [70] 78.26 82.37 MTLFuseNet [71] 80. 43…”

Section: Model Valence Arousalmentioning

confidence: 99%

Emotion Recognition from Brain Connectivity Based on Multi-Head Attention Transformer and Autoencoder Network

Saffari,

Schmidt,

Buscema

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…BiSMSM [67] 61.88 ± 8.76 64.25 ± 6.86 MSD-SS-SAN [68] 74.08 72.74 SSTD [69] 76.81 81.64 STSNet [70] 78.26 82.37 MTLFuseNet [71] 80. 43…”

Section: Model Valence Arousalmentioning

confidence: 99%

Emotion Recognition from Brain Connectivity Based on Multi-Head Attention Transformer and Autoencoder Network

Saffari,

Schmidt,

Buscema

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…These bands include Delta (δ), Theta (θ), Alpha (α), Beta (β), and Gamma (γ) [13]. Building upon these neuroscience findings, several pioneering studies (e.g., [14][15][16]) have focused on investigating frequency features through frequency decomposition. For instance, in study [14], Yang et al utilized the DE as EEG features and combined it with CNN for classification purposes.…”

Section: Introductionmentioning

confidence: 99%

“…Extending this work, [15] employed a 2D-CNN combined with LSTM to extract information. Lastly, in [16], Li et al introduced a combined model based on ManifoldNet and Bi-LSTM, named STSNet. This model extracts the time-frequency-spatial features of EEG.…”

Section: Introductionmentioning

confidence: 99%

Multi-scale 3D-CRU for EEG emotion recognition*

Dong,

Zhou,

Fan

et al. 2024

Biomed. Phys. Eng. Express

View full text Add to dashboard Cite

In this paper, we propose a novel multi-scale 3D-CRU model, with the goal of extracting more discriminative emotion feature from EEG signals. By concurrently exploiting the relative electrode locations and different frequency subbands of EEG signals, a three-dimensional feature representation is reconstructed wherein the Delta (δ) frequency pattern is included. We employ a multi-scale approach, termed 3D-CRU, to concurrently extract frequency and spatial features at varying levels of granularity within each time segment. In the proposed 3D-CRU, we introduce a multi-scale 3D Convolutional Neural Network (3D-CNN) to effectively capture discriminative information embedded within the 3D feature representation. To model the temporal dynamics across consecutive time segments, we incorporate a Gated Recurrent Unit (GRU) module to extract temporal representations from the time series of combined frequency-spatial features. Ultimately, the 3D-GRU model yields a global feature representation, encompassing comprehensive information across time, frequency, and spatial domains. Numerous experimental assessments conducted on publicly available DEAP and SEED databases provide empirical evidence supporting the enhanced performance of our proposed model in the domain of emotion recognition. These findings underscore the eﬀicacy of the features extracted by the proposed multi- scale 3D-GRU model, particularly with the incorporation of the Delta (δ) frequency pattern. Specifically, on the DEAP dataset, the accuracy of Valence and Arousal are 93.12% and 94.31%, respectively, while on the SEED dataset, the accuracy is 92.25%.

show abstract

“…Classification accuracy higher than is reported with a combination of a Deep CNN (DCNN) and a Support Vector Machines [ 30 ]. With a spatio-temporal-spectral network, an accuracy of is obtained [ 21 ]. Finally, Xu et al [ 38 ] reported an accuracy of using a combination of Gated Recurrent Unit (GRU) cells and a CNN.…”

Section: Introductionmentioning

confidence: 99%

Sneaky emotions: impact of data partitions in affective computing experiments with brain-computer interfacing

Moreno-Alcayde,

Traver,

Leiva

2023

Biomed. Eng. Lett.

View full text Add to dashboard Cite

Brain-Computer Interfacing (BCI) has shown promise in Machine Learning (ML) for emotion recognition. Unfortunately, how data are partitioned in training/test splits is often overlooked, which makes it difficult to attribute research findings to actual modeling improvements or to partitioning issues. We introduce the “data transfer rate” construct (i.e., how much data of the test samples are seen during training) and use it to examine data partitioning effects under several conditions. As a use case, we consider emotion recognition in videos using electroencephalogram (EEG) signals. Three data splits are considered, each representing a relevant BCI task: subject-independent (affective decoding), video-independent (affective annotation), and time-based (feature extraction). Model performance may change significantly (ranging e.g. from 50% to 90%) depending on how data is partitioned, in classification accuracy. This was evidenced in all experimental conditions tested. Our results show that (1) for affective decoding, it is hard to achieve performance above the baseline case (random classification) unless some data of the test subjects are considered in the training partition; (2) for affective annotation, having data from the same subject in training and test partitions, even though they correspond to different videos, also increases performance; and (3) later signal segments are generally more discriminative, but it is the number of segments (data points) what matters the most. Our findings not only have implications in how brain data are managed, but also in how experimental conditions and results are reported.

show abstract

STSNet: a novel spatio-temporal-spectral network for subject-independent EEG-based emotion recognition

Cited by 9 publications

References 52 publications

Emotion Recognition from Brain Connectivity Based on Multi-Head Attention Transformer and Autoencoder Network

Emotion Recognition from Brain Connectivity Based on Multi-Head Attention Transformer and Autoencoder Network

Multi-scale 3D-CRU for EEG emotion recognition*

Sneaky emotions: impact of data partitions in affective computing experiments with brain-computer interfacing

Contact Info

Product

Resources

About