Multiscale Temporal Self-Attention and Dynamical Graph Convolution Hybrid Network for EEG-Based Stereogram Recognition

Abstract

Stereopsis is the ability of human beings to get the 3D perception on real scenarios. The conventional stereopsis measurement is based on subjective judgment for stereograms, leading to be easily affected by personal consciousness. To alleviate the issue, in this paper, the EEG signals evoked by dynamic random dot stereograms (DRDS) are collected for stereogram recognition, which can help the ophthalmologists diagnose strabismus patients even without real-time communication. To classify the collected EEG signals, a novel multi-scale temporal self-attention and dynamical graph convolution hybrid network (MTS-DGCHN) is proposed, including multi-scale temporal self-attention module, dynamical graph convolution module and classification module. Firstly, the multi-scale temporal self-attention module is employed to learn time continuity information, where the temporal self-attention block is designed to highlight the global importance of each time segments in one EEG trial, and the multi-scale convolution block is developed to further extract advanced temporal features in multiple receptive fields. Meanwhile, the dynamical graph convolution module is utilized to capture spatial functional relationships between different EEG electrodes, in which the adjacency matrix of each GCN layer is adaptively tuned to explore the optimal intrinsic relationship. Finally, the temporal and spatial features are fed into the classification module to obtain prediction results. Extensive experiments are conducted on collected datasets i.e., SRDA and SRDB, and the results demonstrate the proposed MTS-DGCHN achieves outstanding classification performance compared with the other methods. The datasets are available at https://github.com/YANGeeg/ TJU-SRD-datasets and the code is at https://github. com/YANGeeg/MTS-DGCHN.

Electroencephalography recognition based on encephalic region and temporal sequence transformer

“…This paper adopts the public two EEG datasets SRDA [12] and SRDB [3] to verify the proposed TER-TSAN. In the data processing stage, the sampling frequency is down-sampled from 1000 Hz to 256 Hz, as in [3]. The SGD optimizer was used to optimize the model backpropagation in the training process.…”

“…After 100 iterations of each training, the optimal experimental results are saved. The other details are the same as [3].…”

“…In addition, stereoscopic vision research can employ precise acquisition equipment to obtain the subject's physiological electrical signals, including electrooculoscopy, electromyography, electrocardiography, and electroencephalography (EEG). Among the above signals, EEG is frequently adopted in stereoscopic vision recognition [3] because of its precise and high time resolution.…”

“…Wang et al [10] proposed an AMCNN-DGCN for detecting driver fatigue. In a previous study, we proposed the MTS-DGCHN [3] for DRDS-EEG recognition. These above methods employ the GCN to capture the spatial connections of EEG.…”

A Novel Deep Learning Method Based on an Overlapping Time Window Strategy for Brain–Computer Interface-Based Stroke Rehabilitation

“…The aim of graph neural networks (GNNs) is to use graph structure data and node features as input to learn a representation of the node (or graph) for relevant tasks [ 23 ]. Because EEG data are easily converted to graph structure data, several studies have investigated GNNs applied to EEG signal-based tasks [ 17 , 24 , 25 , 26 , 27 ]. An important aspect of using a GNN to classify EEG signals is building graph data, the original data first need to be converted into graph structure data.…”