Decoding Imagined Speech From EEG Using Transfer Learning

Panachakel, Jerrin Thomas; Ganesan, R.

doi:10.1109/access.2021.3116196

Cited by 17 publications

(5 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As shown in tables 4 and 5, the proposed cross-task transfer learning model achieves competitive performance accuracy of 82.35% for the target KaraOne model and 89.01% for the target FEIS model for multiclass classification, in comparison to prior studies with other transfer learning models such as Cooney et al [28] crosssubject learning of accuracy of 35.68%, Vorontsova et al [29] cross-domain learning of accuracy of 84.5%, Tamm et al [30] cross-subject learning of accuracy of 24.77%, and Panachakel et al [31] cross-domain learning of accuracy of 79.7%-95.5%. The results show that transfer learning procedures are significant for the generalizability of decoding imagined speech EEG signals, despite the difficulties of comparison across state-ofthe-art transfer learning research where the investigations were with distinct datasets.…”

Section: Comparison With the State-of-the-artmentioning

confidence: 65%

“…Tamm et al [30] explored with imagined speech five vowels and six words using the classifier transfer learning approach of inter-subject, yielding a noticeably lower performance. In experiments on intra-subject knowledge transfer by Panachakel et al [31], the ResNet50 transfer learning classifier was applied to the five words and three vowels of imagined speech. The majority of recent research has been focused on the decoding of imagined speech prompts employing cross-subject or crosssession based knowledge sharing.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Decoding of imagined speech electroencephalography neural signals using transfer learning method

Mahapatra,

Bhuyan

2023

J. Phys. Commun.

View full text Add to dashboard Cite

The use of brain-computer interfaces to produce imagined speech from brain waves has the potential to assist individuals with difficulty producing speech or communicating silently. The decoding of covert speech has been observed to have limited efficacy due to the diverse nature of the associated measured brain waves and the limited number of covert speech databases. As a result, traditional machine learning algorithms for learning and inference are challenging, and one of the real alternatives could be to leverage transfer of learning. The main goals of this research were to create a new deep learning (DL) framework for decoding imagined speech electroencephalography (EEG) signals tasks using transfer learning and to transfer the model learning of the source task of an imagined speech EEG dataset to the model training on the target task of another imagined speech EEG dataset, essentially the cross-task learning transfer of discriminative characteristics of the source task to the target task of imagined speech. The experiment was carried out using two distinct open-access EEG datasets, FEIS and KaraOne, that recorded the imagined speech classes of neural signals from multiple individuals. The target FEIS model and the target KaraOne model for multiclass classification exhibit overall accuracy of 80.65% and 81.32%, respectively, according to the proposed transfer learning. The experiment results indicate that the cross-task deep transfer learning design reliably classifies the imagined speech EEG signals by applying the source task learning to the target task learning. The findings suggest the feasibility of a consistent strategy for classifying multiclass imagined speech with transfer learning, which could thereby open up the possibility of future investigation into cross-task imagined speech classification knowledge usability for generalization of new imagined speech prompts.

show abstract

Section: Comparison With the State-of-the-artmentioning

confidence: 65%

Section: Introductionmentioning

confidence: 99%

Decoding of imagined speech electroencephalography neural signals using transfer learning method

Mahapatra,

Bhuyan

2023

J. Phys. Commun.

View full text Add to dashboard Cite

show abstract

“… Cooney et al (2020) used relative wavelet energy features and several CNNs (shallow CNN, deep CNN, and EEGNet) with different hyperparameters to classify two different imagined speech datasets. Panachakel and Ganesan (2021b) used ResNet50 to classify imagined vowels (two classes) and short-long words (three classes) and obtained classification accuracy of 86.28 and 92.8%, respectively. Lee et al (2020) achieved 13 class (12 words/phrases and rest state) classification accuracy of 39.73% using frequency band spectral features and SVM with RBF kernel classifier.…”

Section: Introductionmentioning

confidence: 99%

Multiclass classification of imagined speech EEG using noise-assisted multivariate empirical mode decomposition and multireceptive field convolutional neural network

Park

Lee

2023

Front. Hum. Neurosci.

View full text Add to dashboard Cite

IntroductionIn this study, we classified electroencephalography (EEG) data of imagined speech using signal decomposition and multireceptive convolutional neural network. The imagined speech EEG with five vowels /a/, /e/, /i/, /o/, and /u/, and mute (rest) sounds were obtained from ten study participants.Materials and methodsFirst, two different signal decomposition methods were applied for comparison: noise-assisted multivariate empirical mode decomposition and wavelet packet decomposition. Six statistical features were calculated from the decomposed eight sub-frequency bands EEG. Next, all features obtained from each channel of the trial were vectorized and used as the input vector of classifiers. Lastly, EEG was classified using multireceptive field convolutional neural network and several other classifiers for comparison.ResultsWe achieved an average classification rate of 73.09 and up to 80.41% in a multiclass (six classes) setup (Chance: 16.67%). In comparison with various other classifiers, significant improvements for other classifiers were achieved (p-value < 0.05). From the frequency sub-band analysis, high-frequency band regions and the lowest-frequency band region contain more information about imagined vowel EEG data. The misclassification and classification rate of each vowel imaginary EEG was analyzed through a confusion matrix.DiscussionImagined speech EEG can be classified successfully using the proposed signal decomposition method and a convolutional neural network. The proposed classification method for imagined speech EEG can contribute to developing a practical imagined speech-based brain-computer interfaces system.

show abstract

“…Sarmiento et al [20] used the CNN-based model to classify five English vowels. Panachakel and Ganesan [21] used sliding window data augmentation and TL on the underlying ResNet50 model to classify imagined spoken words and vowels.…”

Section: Introductionmentioning

confidence: 99%

Multiclass Classification of Imagined Speech Vowels and Words of Electroencephalography Signals Using Deep Learning

Mahapatra

Bhuyan

2022

Advances in Human-Computer Interaction

View full text Add to dashboard Cite

The paper’s emphasis is on the imagined speech decoding of electroencephalography (EEG) neural signals of individuals in accordance with the expansion of the brain-computer interface to encompass individuals with speech problems encountering communication challenges. Decoding an individual’s imagined speech from nonstationary and nonlinear EEG neural signals is a complex task. Related research work in the field of imagined speech has revealed that imagined speech decoding performance and accuracy require attention to further improve. The evolution of deep learning technology increases the likelihood of decoding imagined speech from EEG signals with enhanced performance. We proposed a novel supervised deep learning model that combined the temporal convolutional networks and the convolutional neural networks with the intent of retrieving information from the EEG signals. The experiment was carried out using an open-access dataset of fifteen subjects’ imagined speech multichannel signals of vowels and words. The raw multichannel EEG signals of multiple subjects were processed using discrete wavelet transformation technique. The model was trained and evaluated using the preprocessed signals, and the model hyperparameters were adjusted to achieve higher accuracy in the classification of imagined speech. The experiment results demonstrated that the multiclass imagined speech classification of the proposed model exhibited a higher overall accuracy of 0.9649 and a classification error rate of 0.0350. The results of the study indicate that individuals with speech difficulties might well be able to leverage a noninvasive EEG-based imagined speech brain-computer interface system as one of the long-term alternative artificial verbal communication mediums.

show abstract

Decoding Imagined Speech From EEG Using Transfer Learning

Cited by 17 publications

References 52 publications

Decoding of imagined speech electroencephalography neural signals using transfer learning method

Decoding of imagined speech electroencephalography neural signals using transfer learning method

Multiclass classification of imagined speech EEG using noise-assisted multivariate empirical mode decomposition and multireceptive field convolutional neural network

Multiclass Classification of Imagined Speech Vowels and Words of Electroencephalography Signals Using Deep Learning

Contact Info

Product

Resources

About