Decoding Imagined and Spoken Phrases From Non-invasive Neural (MEG) Signals

Abstract

Speech production is a hierarchical mechanism involving the synchronization of the brain and the oral articulators, where the intention of linguistic concepts is transformed into meaningful sounds. Individuals with locked-in syndrome (fully paralyzed but aware) lose their motor ability completely including articulation and even eyeball movement. The neural pathway may be the only option to resume a certain level of communication for these patients. Current brain-computer interfaces (BCIs) use patients' visual and attentional correlates to build communication, resulting in a slow communication rate (a few words per minute). Direct decoding of imagined speech from the neural signals (and then driving a speech synthesizer) has the potential for a higher communication rate. In this study, we investigated the decoding of five imagined and spoken phrases from single-trial, non-invasive magnetoencephalography (MEG) signals collected from eight adult subjects. Two machine learning algorithms were used. One was an artificial neural network (ANN) with statistical features as the baseline approach. The other was convolutional neural networks (CNNs) applied on the spatial, spectral and temporal features extracted from the MEG signals. Experimental results indicated the possibility to decode imagined and spoken phrases directly from neuromagnetic signals. CNNs were found to be highly effective with an average decoding accuracy of up to 93% for the imagined and 96% for the spoken phrases.

Evaluation of Hyperparameter Optimization in Machine and Deep Learning Methods for Decoding Imagined Speech EEG

“…Fourth-order Daubechies were used to achieve the results presented in [56] and were therefore used here. A feature vector was constructed using the RWE of decomposition levels D2 (16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30)(31)(32), D3 (8-16 Hz), D4 (4-8 Hz), D5 (2-4 Hz), and A5 (<2 Hz), for each channel. This resulted in a 30-element feature vector for each trial.…”

“…One recent study has demonstrated the potential for spoken sentences to be synthesized from neural activity [ 14 ] and another has shown speech reconstruction directly from the auditory cortex while subjects listened to overt speech [ 15 ]. Non-invasive studies using magnetoencephalogram [ 16 ] and EEG [ 17 ] have demonstrated potential for decoding speech using these technologies. Several studies have used the advantage of non-invasive recording through EEG to investigate imagined speech as a communicative paradigm for BCIs (e.g., [ 17 , 18 , 19 , 20 , 21 , 22 , 23 ]).…”

MEG Sensor Selection for Neural Speech Decoding

“…The signals were of 1 kHz sampling frequency, from which the first two detail coefficients (d respectively. Considering the effectiveness of db-4 wavelet in increased SNR and decoding performance in our prior works [12], [47], [66] and its use in other decoding studies [26], we implemented this denoising and decomposing step prior to decoding. The use of wavelets to generate distinct MEG brainwaves has been shown previously [67]- [69].…”

Silent Speech Interfaces for Speech Restoration: A Review

“…Unfortunately, MEG requires expensive superconducting quantum interference devices operating in appropriate magnetically shielded rooms. Thus, MEG has been mostly used in neuroimaging and studies, although recently some attempts have been done in using this technique for silent speech recognition [198].…”