EEGDnet: Fusing non-local and local self-similarity for EEG signal denoising with transformer

Pu, Xiaorong; Peng, Yi; Chen, Kecheng; Ma, Zhaoqi; Zhao, Di; Ren, Yazhou

doi:10.1016/j.compbiomed.2022.106248

Cited by 14 publications

(3 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To better separate and recover the original shipradiated noise before distortion and mixing, an end-to-end nonlinear BSS network based on an attention mechanism is proposed in this paper. Due to the fact that the Transformer has a shortcoming in capturing local self-dependency and performs well in learning long-term or global dependencies, while convolutional neural networks (CNN) and RNN behave in the opposite way [52][53][54][55][56], an end-to-end network is utilized combining an RNN and multi-head self-attention, i.e., recurrent attention neural networks (RANN). The recurrent attention mechanism is used in image aesthetics, target detection, flow forecasting and time series forecasting [57][58][59][60][61], but it has not been used in nonlinear BSS yet.…”

Section: Introductionmentioning

confidence: 99%

Underwater Acoustic Nonlinear Blind Ship Noise Separation Using Recurrent Attention Neural Networks

Song,

Feng,

Wang

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

Ship-radiated noise is the main basis for ship detection in underwater acoustic environments. Due to the increasing human activity in the ocean, the captured ship noise is usually mixed with or covered by other signals or noise. On the other hand, due to the softening effect of bubbles in the water generated by ships, ship noise undergoes non-negligible nonlinear distortion. To mitigate the nonlinear distortion and separate the target ship noise, blind source separation (BSS) becomes a promising solution. However, underwater acoustic nonlinear models are seldom used in research for nonlinear BSS. This paper is based on the hypothesis that the recovery and separation accuracy can be improved by considering this nonlinear effect in the underwater environment. The purpose of this research is to explore and discover a method with the above advantages. In this paper, a model is used in underwater BSS to describe the nonlinear impact of the softening effect of bubbles on ship noise. To separate the target ship-radiated noise from the nonlinear mixtures, an end-to-end network combining an attention mechanism and bidirectional long short-term memory (Bi-LSTM) recurrent neural network is proposed. Ship noise from the database ShipsEar and line spectrum signals are used in the simulation. The simulation results show that, compared with several recent neural networks used for linear and nonlinear BSS, the proposed scheme has an advantage in terms of the mean square error, correlation coefficient and signal-to-distortion ratio.

show abstract

Section: Introductionmentioning

confidence: 99%

Underwater Acoustic Nonlinear Blind Ship Noise Separation Using Recurrent Attention Neural Networks

Song,

Feng,

Wang

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Liu et al [29] proposed a novel recurrent neural networkbased EEG state estimation model, which automatically replaces the detected noises with the estimated values to achieve a robust EEG perception. Pu et al [30] proposed a 1-D EEG signal denoising network EEGDNet with a 2D Transformer, which can significantly reduce the negative effects of noise and outliers by fusing the nonlocal self-similarity in the selfattention module and the local self-similarity in the feedforward module.…”

Section: Introductionmentioning

confidence: 99%

SQI-DOANet: electroencephalogram-based deep neural network for estimating signal quality index and depth of anaesthesia

Yu,

Zhou,

et al. 2024

J. Neural Eng.

View full text Add to dashboard Cite

Objective:Monitoring the depth of anaesthesia (DOA) during surgery is of critical importance. However, accurate and real-time estimation of DOA remains a challenging task. In this paper, we proposed a signal quality index (SQI) network (SQINet) for assessing the electroencephalogram (EEG) signal quality and a DOA network (DOANet) for analyzing EEG signals to precisely estimate DOA. The two networks are termed SQI-DOANet.Approach:The SQINet contained a shallow convolutional neural network to quickly determine the quality of the EEG signal. The DOANet comprised a feature extraction module for extracting features, a dual attention module for fusing multi-channel and multi-scale information, and a gated multilayer perceptron module for extracting temporal information. The performance of the SQI-DOANet model was validated by training and testing the model on the large VitalDB database, with the bispectral index (BIS) as the reference standard.Main results:The proposed DOANet yielded a Pearson correlation coefficient with the BIS score of 0.88 in the 5-fold cross-validation, with a mean absolute error (MAE) of 4.81. The mean Pearson correlation coefficient of SQI-DOANet with the BIS score in the 5-fold cross-validation was 0.82, with an MAE of 5.66.Significance:The SQI-DOANet model outperformed three compared methods. The proposed SQI-DOANet may be used as a new deep learning method for DOA estimation. The code of the SQI-DOANet will be made available publicly at https://github.com/YuRui8879/SQI-DOANet.

show abstract

“…The Self-Attention (SA) structure and parallel computing mode allow the transfer extract global information without multiple convolutions and pooling calculations. In the BCI field, the transformer is adopted to handle signals in the applications such as person identification (Du et al, 2022 ), emotion recognition (Li et al, 2022 ), visual stimulus classification (Bagchi and Bathula, 2022 ) and signal denoising (Pu et al, 2022 ). For MI-EEG decoding, Ma et al (Ma et al, 2022 ) proposed a hybrid CNN-Transformer model to weigh spatial features and frequency signals by employing the attention mechanism.…”

Section: Introductionmentioning

confidence: 99%

Local and global convolutional transformer-based motor imagery EEG classification

Zhang,

Li,

Yang

et al. 2023

Front. Neurosci.

View full text Add to dashboard Cite

Transformer, a deep learning model with the self-attention mechanism, combined with the convolution neural network (CNN) has been successfully applied for decoding electroencephalogram (EEG) signals in Motor Imagery (MI) Brain-Computer Interface (BCI). However, the extremely non-linear, nonstationary characteristics of the EEG signals limits the effectiveness and efficiency of the deep learning methods. In addition, the variety of subjects and the experimental sessions impact the model adaptability. In this study, we propose a local and global convolutional transformer-based approach for MI-EEG classification. The local transformer encoder is combined to dynamically extract temporal features and make up for the shortcomings of the CNN model. The spatial features from all channels and the difference in hemispheres are obtained to improve the robustness of the model. To acquire adequate temporal-spatial feature representations, we combine the global transformer encoder and Densely Connected Network to improve the information flow and reuse. To validate the performance of the proposed model, three scenarios including within-session, cross-session and two-session are designed. In the experiments, the proposed method achieves up to 1.46%, 7.49% and 7.46% accuracy improvement respectively in the three scenarios for the public Korean dataset compared with current state-of-the-art models. For the BCI competition IV 2a dataset, the proposed model also achieves a 2.12% and 2.21% improvement for the cross-session and two-session scenarios respectively. The results confirm that the proposed approach can effectively extract much richer set of MI features from the EEG signals and improve the performance in the BCI applications.

show abstract

EEGDnet: Fusing non-local and local self-similarity for EEG signal denoising with transformer

Cited by 14 publications

References 21 publications

Underwater Acoustic Nonlinear Blind Ship Noise Separation Using Recurrent Attention Neural Networks

Underwater Acoustic Nonlinear Blind Ship Noise Separation Using Recurrent Attention Neural Networks

SQI-DOANet: electroencephalogram-based deep neural network for estimating signal quality index and depth of anaesthesia

Local and global convolutional transformer-based motor imagery EEG classification

Contact Info

Product

Resources

About