Sonification and textification: Proposing methods for classifying unspoken words from EEG signals

González-Castañeda, Erick Fernando; Torres-García, Alejandro A.; Reyes-García, Carlos A.; Villaseñor-Pineda, Luis

doi:10.1016/j.bspc.2016.10.012

Cited by 47 publications

(50 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although the area of BCI based speech intent recognition has received increasing attention within the research community in the past few years, most research has focused on classification of individual speech categories in terms of discrete vowels, phonemes and words [5][6][7][8][9][10][11][12][13]. This includes categorization of imagined EEG signal into binary vowel categories like /a/, /u/ and rest [5][6][7]; binary syllable classes like /ba/ and /ku/ [8][9][10]14]; a handful of control words like 'up', 'down', 'left', 'right' and 'select' [13] or others like 'water', 'help', 'thanks', 'food', 'stop' [11], Chinese characters [12], etc. Such works mostly involve traditional signal processing or manual feature handcrafting along with linear classifiers (e.g., SVMs).…”

Section: Introductionmentioning

confidence: 99%

SPEAK YOUR MIND! Towards Imagined Speech Recognition with Hierarchical Deep Learning

Saha¹,

Abdul-Mageed²,

Fels³

2019

Interspeech 2019

View full text Add to dashboard Cite

Speech-related Brain Computer Interface (BCI) technologies provide effective vocal communication strategies for controlling devices through speech commands interpreted from brain signals. In order to infer imagined speech from active thoughts, we propose a novel hierarchical deep learning BCI system for subject-independent classification of 11 speech tokens including phonemes and words. Our novel approach exploits predicted articulatory information of six phonological categories (e.g., nasal, bilabial) as an intermediate step for classifying the phonemes and words, thereby finding discriminative signal responsible for natural speech synthesis. The proposed network is composed of hierarchical combination of spatial and temporal CNN cascaded with a deep autoencoder. Our best models on the KARA database achieve an average accuracy of 83.42% across the six different binary phonological classification tasks, and 53.36% for the individual token identification task, significantly outperforming our baselines. Ultimately, our work suggests the possible existence of a brain imagery footprint for the underlying articulatory movement related to different sounds that can be used to aid imagined speech decoding.

show abstract

Section: Introductionmentioning

confidence: 99%

SPEAK YOUR MIND! Towards Imagined Speech Recognition with Hierarchical Deep Learning

Saha¹,

Abdul-Mageed²,

Fels³

2019

Interspeech 2019

View full text Add to dashboard Cite

show abstract

“…Discrete Wavelet Transform (DWT) has been applied in several EEG studies. For example, epileptic seizure detection [15], unspoken speech recognition [4], [9], emotion recognition [16], [17]. DWT decomposes the signal into detailed and approximation coefficients by analysing the signal into different frequency bands.…”

Section: B Feature Extractionmentioning

confidence: 99%

“…However, in this work we have different numbers of resulting wavelet coefficients because the participants can perform the imagination in different time lengths. To make the number of features identical for all trials, in [4], [19] it has been proposed to calculate the Relative Wavelet Energy (RWE) for all the detailed coefficients and the approximation coefficient to equalize the number of features. However, the calculation of energy includes summation of DWT coefficients which reduces the effectiveness of DWT because it removes the temporal information included in the coefficients [16].…”

Section: B Feature Extractionmentioning

confidence: 99%

See 1 more Smart Citation

Examining Temporal Variations in Recognizing Unspoken Words Using EEG Signals

Al-Saleh

Moore

Christensen

et al. 2018

2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC)

View full text Add to dashboard Cite

Studies on recognising unspoken speech with the use of electroencephalographic (EEG) signals vary in their designs. The participants are either asked to imagine unspoken speech within a specific time frame, or alternatively indicate the start and end of the imagined speech. Optimizing the length and training size of imagined speech is important to improve the rate and speed of recognizing unspoken speech in on-line applications. In this study, we recorded EEG data when the participants performed unspoken speech of five words using two technologies: (1) marking the start and end of the trial by using mouse clicks and (2) performing the imagination in a four-second fixed time window. Four classifiers were trained in all experiment parts: support vector machine, naive bayes, random forest, and linear discriminate analysis. The results show that the best time frame is 3.5-4 seconds length. Moreover, the increase in training size improve the average classification accuracy. However, this improvement becomes slight between 125-175 total training trials. The training data can be recorded in parts, however, the required training size should be increased to have better classification accuracy. In all analysis parts, random forest classifier shows better results among the other classifiers.

show abstract

“…Importantly, these weaknesses are even more pronounced when using non-gel wireless EEG devices. Studies in the area of imagined speech using EEG technologies, can be divided into three types, based on the type of imagined speech used, namely word imagination [3], [4], [5], [6], [7], syllable imagination [8], [9] and vowel imagination [10], [11].…”

Section: Introductionmentioning

confidence: 99%