Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus

Abstract

Objective : To evaluate the potential of intracortical electrode array signals for brain-computer interfaces (BCIs) to restore lost speech, we measured the performance of classifiers trained to discriminate a comprehensive basis set for speech: 39 English phonemes. We classified neural correlates of spoken-out-loud words in the "hand knob" area of precentral gyrus, which we view as a step towards the eventual goal of decoding attempted speech from ventral speech areas in patients who are unable to speak. Approach : Neural and audio data were recorded while two BrainGate2 pilot clinical trial participants, each with two chronically-implanted 96-electrode arrays, spoke 420 different words that broadly sampled English phonemes. Phoneme onsets were identified from audio recordings, and their identities were then classified from neural features consisting of each electrode's binned action potential counts or high-frequency local field potential power. We also examined two potential confounds specific to decoding overt speech: acoustic contamination of neural signals and systematic differences in labeling different phonemes' onset times..

Imagined speech can be decoded from low- and cross-frequency intracranial EEG features

“…Off-line analyses such as those we present here are the first and necessary step to guide us once we will be able to use those electrodes in humans along with on-line systems. Unlike the robotic arms that are currently being developed for motor restoration, which are optimally controlled by the dense sampling of a spatially restricted cortical area (typically a Utah array) 1 , 7 , a language BCI system for severe aphasia will require broader coverage of the cortical surface, including the frontal and the temporal lobes, to not only cope with the high physiological intersubject variability of inner speech production but also with the variable structural damage (cortical, subcortical) that patients may have suffered from. In post-stroke Broca-type aphasia, the efforts to overcome the overt speech planning deficit during imagined speech are expected to implicate a large range of regions of the language network, which will all have to be sampled.…”

“…Although potentially interesting, this hypothesis is limited in scope as it can only apply to cases where language and cortical motor commands are preserved (such as in motor neuron disease), i.e. a minority of the patients with severe speech production deficits 6 , 7 . If, as in most post-stroke aphasia cases, the cortical language network is injured, other decoding strategies must be envisaged, for instance using neural signals from the remaining intact brain regions that encode speech, e.g.…”

A Spatial Transcriptomics Study of the Brain-Electrode Interface in Rat Motor Cortex

“…Clinical applications of these devices have progressed rapidly over the past decade and have recently gained newfound interest due to the potential for use in brain-computer interfaces (BCIs). Recent advances in recording-based implants have restored quadriplegic 1,2 or quadriparesis 3 patients' ability to communicate. BCI research has shown the capability of recording implants in motor cortex to drive movement of a robotic limb or computer cursor 4,5 .…”

Decoding Covert Speech From EEG-A Comprehensive Review

“…The distribution of the modalities used for decoding imagined speech in these papers is given in Figure 1 . These modalities include EEG, ECoG (Herff et al, 2015 , 2016 ), fMRI (Yoo et al, 2004 ; Abe et al, 2011 ), fNIRS (Herff et al, 2012 ; Kamavuako et al, 2018 ; Sereshkeh et al, 2018 ), MEG (Destoky et al, 2019 ; Dash et al, 2020 ), ICE (Brumberg et al, 2011 ; Kennedy et al, 2017 ; Wilson et al, 2020 ) etc. Clearly, EEG is the most popular modality used for decoding imagined speech with 18 articles using it for capturing the neural changes during imagined speech.…”