Imagined speech can be decoded from low- and cross-frequency intracranial EEG features

Abstract

Reconstructing intended speech from neural activity using brain-computer interfaces holds great promises for people with severe speech production deficits. While decoding overt speech has progressed, decoding imagined speech has met limited success, mainly because the associated neural signals are weak and variable compared to overt speech, hence difficult to decode by learning algorithms. We obtained three electrocorticography datasets from 13 patients, with electrodes implanted for epilepsy evaluation, who performed overt and imagined speech production tasks. Based on recent theories of speech neural processing, we extracted consistent and specific neural features usable for future brain computer interfaces, and assessed their performance to discriminate speech items in articulatory, phonetic, and vocalic representation spaces. While high-frequency activity provided the best signal for overt speech, both low- and higher-frequency power and local cross-frequency contributed to imagined speech decoding, in particular in phonetic and vocalic, i.e. perceptual, spaces. These findings show that low-frequency power and cross-frequency dynamics contain key information for imagined speech decoding.

Event-related causality in Stereo-EEG discriminates syntactic processing of noun phrases and verb phrases

“…In recent years, there have been important technological and methodological advancements in perceived and imagined speech decoding (Martin et al, 2018;Panachakel & Ramakrishnan, 2021). Recent works focus on the classification of vowels (M. S. Mahmud et al, 2020; N. T. Duc & B. Lee, 2020), syllables (Archila-Meléndez et al, 2018;Brandmeyer et al, 2013;Correia et al, 2015), words (Ossmy et al, 2015;Proix et al, 2022;Vorontsova et al, 2021) and complete sentences (Chakrabarti et al, 2015;Zhang et al, 2012), distinguishing stimuli mainly at the semantic level. The most advanced online decoding techniques rely heavily on the articulatory representation of syllables and words in the motor and supplementary motor cortices (Anumanchipalli et al, 2019).…”

“…However, this approach can only be applied to patients with intact motor commands, which represent a minority of the patients with speech impairment (Guenther et al, 2009;Wilson et al, 2020). Thus, other decoding strategies that rely on the brain regions that encode speech are needed (Proix et al, 2022). Here, we decoded the acoustic stimuli exploiting 29 different speech-encoding cortical areas spanning the entire brain.…”

“…Here, we decoded the acoustic stimuli exploiting 29 different speech-encoding cortical areas spanning the entire brain. Only recently such strategy has been used in the decoding of groups of syllables and words (Proix et al, 2022). However, our approach relies on the time evolution of the connectivity values between recording contacts.…”

Identifying stimulus-driven neural activity patterns in multi-patient intracranial recordings

“…Multi-electrode LFP recordings from one or more brain areas can be especially informative. For example, multi-channel LFPs may be used to decode visual stimuli [12,13], auditory stimuli [13,143], speech production [52,112,122], acute pain onset and intensity [165], and even semantic representations [39,97].…”

“…For example, in the domains of speech comprehension and speech production, temporal information is a primary indicator of meaning. Studying neural responses to speech therefore requires considering how neural correlates of speech unfold over time [52,122].…”