Short time Fourier transformation and deep neural networks for motor imagery brain computer interface recognition

Wang, Zijian; Cao, Lei; Zhang, Zuo; Gong, Xiaoliang; Sun, Yaoru; Wang, Haoran

doi:10.1002/cpe.4413

Cited by 73 publications

(40 citation statements)

References 32 publications

(37 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Methods vary across these studies and include grid search [47,48], Bayesian methods [49][50][51] (one fails to report the specific approach [49]), trial and error [24,52], and unstated approaches, likely indicating trial and error [53,54]. In six of these studies, only partial results are reported in relation to HPs [24,48,50,51,53,54]. For example, in an otherwise excellent paper [50], only present optimal values for structural parameters were tested and the authors completely fail to report on the effects of optimizing learning rate and learning rate decay.…”

Section: Introductionmentioning

confidence: 99%

Evaluation of Hyperparameter Optimization in Machine and Deep Learning Methods for Decoding Imagined Speech EEG

Cooney

Korik

Folli

et al. 2020

Sensors

View full text Add to dashboard Cite

Classification of electroencephalography (EEG) signals corresponding to imagined speech production is important for the development of a direct-speech brain–computer interface (DS-BCI). Deep learning (DL) has been utilized with great success across several domains. However, it remains an open question whether DL methods provide significant advances over traditional machine learning (ML) approaches for classification of imagined speech. Furthermore, hyperparameter (HP) optimization has been neglected in DL-EEG studies, resulting in the significance of its effects remaining uncertain. In this study, we aim to improve classification of imagined speech EEG by employing DL methods while also statistically evaluating the impact of HP optimization on classifier performance. We trained three distinct convolutional neural networks (CNN) on imagined speech EEG using a nested cross-validation approach to HP optimization. Each of the CNNs evaluated was designed specifically for EEG decoding. An imagined speech EEG dataset consisting of both words and vowels facilitated training on both sets independently. CNN results were compared with three benchmark ML methods: Support Vector Machine, Random Forest and regularized Linear Discriminant Analysis. Intra- and inter-subject methods of HP optimization were tested and the effects of HPs statistically analyzed. Accuracies obtained by the CNNs were significantly greater than the benchmark methods when trained on both datasets (words: 24.97%, p < 1 × 10–7, chance: 16.67%; vowels: 30.00%, p < 1 × 10–7, chance: 20%). The effects of varying HP values, and interactions between HPs and the CNNs were both statistically significant. The results of HP optimization demonstrate how critical it is for training CNNs to decode imagined speech.

show abstract

Section: Introductionmentioning

confidence: 99%

Evaluation of Hyperparameter Optimization in Machine and Deep Learning Methods for Decoding Imagined Speech EEG

Cooney

Korik

Folli

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…As future work, to enhance the impact of tested Deep Learning models, we plan to employ datasets that hold more labeled MI tasks, fusing CNNs with different characteristics and architectures is also to be considered to learn more complex relationships between spatial patterns and extracted t-f representations, making the learned CNN weights more accessible to interpret [53,54].…”

Section: Discussion and Concluding Remarksmentioning

confidence: 99%

CNN-based framework using spatial dropping for enhanced interpretation of neural activity in motor imagery classification

et al. 2020

View full text Add to dashboard Cite

Interpretation of brain activity responses using motor imagery (MI) paradigms is vital for medical diagnosis and monitoring. Assessed by machine learning techniques, identification of imagined actions is hindered by substantial intra-and inter-subject variability. Here, we develop an architecture of Convolutional Neural Networks (CNN) with an enhanced interpretation of the spatial brain neural patterns that mainly contribute to the classification of MI tasks. Two methods of 2D-feature extraction from EEG data are contrasted: Power Spectral Density and Continuous Wavelet Transform. For preserving the spatial interpretation of extracting EEG patterns, we project the multi-channel data using a topographic interpolation. Besides, we include a spatial dropping algorithm to remove the learned weights that reflect the localities not engaged with the elicited brain response. We evaluate two labeled scenarios of MI tasks: bi-class and three-class. Obtained results in an MI database show that the thresholding strategy combined with Continuous Wavelet Transform improves the accuracy and enhances the interpretability of CNN architecture, showing that the highest contribution clusters over the sensorimotor cortex with a differentiated behavior of rhythms µ and β.

show abstract

Section: A02tmentioning

confidence: 99%

CNN-based Framework using Spatial Dropping for Enhanced Interpretation of Neural Activity in Motor Imagery Classification

Collazos-Huertas

Meza

Domínguez

2020

Preprint

View full text Add to dashboard Cite

Interpretation of brain activity responses using Motor Imagery (MI) paradigms is vital for medical diagnosis and monitoring. Assessed by machine learning techniques, identification of imagined actions is hindered by substantial intra and inter subject variability. Here, we develop an architecture of Convolutional Neural Networks (CNN) with enhanced interpretation of the spatial brain neural patterns that mainly contribute to the classification of MI tasks. Two methods of 2D-feature extraction from EEG data are contrasted: Power Spectral Density and Continuous Wavelet Transform. For preserving the spatial interpretation of extracting EEG patterns, we project the multi-channel data using a topographic interpolation. Besides, we include a spatial dropping algorithm to remove the learned weights that reflect the localities not engaged with the elicited brain response. Obtained results in a bi-task MI database show that the thresholding strategy in combination with Continuous Wavelet Transform improves the accuracy and enhances the interpretability of CNN architecture, showing that the highest contribution clusters over the sensorimotor cortex with differentiated behavior between μ and β rhythms.

show abstract

Short time Fourier transformation and deep neural networks for motor imagery brain computer interface recognition

Cited by 73 publications

References 32 publications

Evaluation of Hyperparameter Optimization in Machine and Deep Learning Methods for Decoding Imagined Speech EEG

Evaluation of Hyperparameter Optimization in Machine and Deep Learning Methods for Decoding Imagined Speech EEG

CNN-based framework using spatial dropping for enhanced interpretation of neural activity in motor imagery classification

CNN-based Framework using Spatial Dropping for Enhanced Interpretation of Neural Activity in Motor Imagery Classification

Contact Info

Product

Resources

About