EEG-based intention recognition with deep recurrent-convolution neural network: Performance and channel selection by Grad-CAM

Li, Yurong; Yang, Hao; Li, Jixiang; Chen, Dongyi; Du, Min

doi:10.1016/j.neucom.2020.07.072

Cited by 64 publications

(28 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Without adjusting the hyper-parameters determined using Dataset-A, the model was then tested on Dataset-B, as explained in Section 2.1. The results of this generalization test are presented in Table 2, and show that the proposed method outperformed other baseline models and a Gradient-Class Activation Mapping (Grad-CAM) channel selection method proposed in [11]. Performance was close to, although slightly less than, that of a cascaded CRNN approach proposed in [8].…”

Section: Resultsmentioning

confidence: 79%

“…The activities included eyes closed, imagining opening/closing left fist, imagining opening/closing right fist, imagining opening/closing both fists, and imagining opening/closing both feet. The data from 108 participants (subject #89 was excluded as suggested in previous works [8,11]), were modeled using the proposed framework by randomly selecting 75% for the training and the remaining 25% for testing, as employed in [11].…”

Section: Datasetmentioning

confidence: 99%

See 1 more Smart Citation

A Deep Spatio-Temporal Model for EEG-Based Imagined Speech Recognition

Kumar

Scheme

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Automatic speech recognition interfaces are becoming increasingly pervasive in daily life as a means of interacting with and controlling electronic devices. Current speech interfaces, however, are infeasible for a variety of users and use cases, such as patients who suffer from locked-in syndrome or those who need privacy. In these cases, an interface that works based on envisioned speech, the idea of imagining what one wants to say, could be of benefit. Consequently, in this work, we propose an imagined speech Brain-Computer-Interface (BCI) using Electroencephalogram (EEG) signals. EEG signals are processed using a deep spatio-temporal learning architecture with 1D Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM), respectively. LSTM units are implemented in a many-to-many fashion to produce a time series of imagined speech outputs. Using this series, the performance of the system is boosted using majority vote (MV) post-processing to further improve results. The performance is evaluated on two publicly available datasets; one to test the performance of the tuned model, and another to test its generalization to a new dataset. The proposed architecture outperforms previous results with improvements of up to 23.7%.

show abstract

Section: Resultsmentioning

confidence: 79%

Section: Datasetmentioning

confidence: 99%

A Deep Spatio-Temporal Model for EEG-Based Imagined Speech Recognition

Kumar

Scheme

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…[37][38][39]. The EEG signals can be divided into four rhythms according to the frequency band distribution: δ (1-4 Hz), θ (4-8 Hz), α (8-12 Hz) and β (12)(13)(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30). The multitaper spectral estimation method based on PSD is used to calculate the average PSDs in one epoch of each channel [40,41], namely p f req,ch (i).…”

Section: Data Preprocessingmentioning

confidence: 99%

“…Alyasser et al promoted an EEG channel selection method based on the binary flower pollination algorithm (FPA) and β-hill climbing for personal identification [26], which showed that half of the channel numbers can achieve high accuracy. Li et al applied the Gradient Class Activation Mapping (Grad-CAM) visualization technology on raw EEG signals to the channel selection [27]. The results achieved an optimal tradeoff between performance and the number of channels for EEG intention decoding.…”

Section: Introductionmentioning

confidence: 99%

Sensitive Channel Selection for Mental Workload Classification

Jin

Qu²,

Pang³

et al. 2022

Mathematics

View full text Add to dashboard Cite

Mental workload (MW) assessment has been widely studied in various human–machine interaction tasks. The existing researches on MW classification mostly use non-invasive electroencephalography (EEG) caps to collect EEG signals and identify MW levels. However, the activation region of the brain stimulated by MW tasks is not the same for every subject. It may be inappropriate to use EEG signals from all electrode channels to identify MW. In this paper, an EEG rhythm energy heatmap is first established to visually show the change trends in the energy of four EEG rhythms with time, EEG channels and MW levels. It can be concluded from the presented heatmaps that this change trend varies with subjects, rhythms and channels. Based on the analysis, a double threshold method is proposed to select sensitive channels for MW assessment. The EEG signals of personalized selected channels, named positive sensitive channels (PSCs) and negative sensitive channels (NSCs), are used for MW classification using the Support Vector Machine (SVM) algorithm. The results show that the selection of personalized sensitive channels generally contributes to improving the performance of MW classification.

show abstract

“…Still, the approaches for building class activation mapping-based (CAM) visualizations have increased interest in MI research, which performs a weighted sum of the feature maps of the last convolutional layer for each class using and a structural regularizer for preventing overfitting during training [32,33]. Specifically, the visualizations generated by gradient-based methods such as GradCam provide explanations with fine-grained details of the predicted class [34][35][36]. However, the CNN-learned features to be highlighted for interpretation purposes must be compatible with the neurophysiological principle of MI [37,38].…”

Section: Introductionmentioning

confidence: 99%

Image-Based Learning Using Gradient Class Activation Maps for Enhanced Physiological Interpretability of Motor Imagery Skills

Collazos-Huertas

Álvarez-Meza

Castellanos-Domínguez

2022

Applied Sciences

View full text Add to dashboard Cite

Brain activity stimulated by the motor imagery paradigm (MI) is measured by Electroencephalography (EEG), which has several advantages to be implemented with the widely used Brain–Computer Interfaces (BCIs) technology. However, the substantial inter/intra variability of recorded data significantly influences individual skills on the achieved performance. This study explores the ability to distinguish between MI tasks and the interpretability of the brain’s ability to produce elicited mental responses with improved accuracy. We develop a Deep and Wide Convolutional Neuronal Network fed by a set of topoplots extracted from the multichannel EEG data. Further, we perform a visualization technique based on gradient-based class activation maps (namely, GradCam++) at different intervals along the MI paradigm timeline to account for intra-subject variability in neural responses over time. We also cluster the dynamic spatial representation of the extracted maps across the subject set to come to a deeper understanding of MI-BCI coordination skills. According to the results obtained from the evaluated GigaScience Database of motor-evoked potentials, the developed approach enhances the physiological explanation of motor imagery in aspects such as neural synchronization between rhythms, brain lateralization, and the ability to predict the MI onset responses and their evolution during training sessions.

show abstract

EEG-based intention recognition with deep recurrent-convolution neural network: Performance and channel selection by Grad-CAM

Cited by 64 publications

References 25 publications

A Deep Spatio-Temporal Model for EEG-Based Imagined Speech Recognition

A Deep Spatio-Temporal Model for EEG-Based Imagined Speech Recognition

Sensitive Channel Selection for Mental Workload Classification

Image-Based Learning Using Gradient Class Activation Maps for Enhanced Physiological Interpretability of Motor Imagery Skills

Contact Info

Product

Resources

About