Dataset of Speech Production in intracranial Electroencephalography

Verwoert, Maxime; Ottenhoff, Maarten C; Colon, Albert; Wagner, Louis; Tousseyn, Simon; Dijk, J.P. van; Kubben, Pieter L.

doi:10.1038/s41597-022-01542-9

Cited by 18 publications

(30 citation statements)

References 84 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cometa et al discovered involvement from both cortical and subcortical in syntactic processing, including from the non-dominant hemisphere 15 . Verwoert et al published an open access sEEG dataset of 10 participants reading Dutch words 16 .…”

Section: Discussionmentioning

confidence: 99%

Investigation of contributions from cortical and subcortical brain structures for speech decoding

Wu,

Cai,

Ming

et al. 2023

Preprint

View full text Add to dashboard Cite

Language impairments often arise from severe neurological disorders, prompting the development of neural prosthetics based on electrophysiological signals for the restoration of comprehensible language information. Previous decoding efforts have focused mainly on signals from the cerebral cortex, neglecting the potential contributions of subcortical brain structures to speech decoding in brain-computer interfaces (BCIs). This study aims to explore the role of subcortical structures for speech decoding by utilizing stereotactic electroencephalography (sEEG). Two native Mandarin Chinese speakers, who underwent sEEG implantation for pharmaco-resistant epilepsy, participated in this study. sEEG contacts were primarily located in the superior temporal gyrus, middle temporal gyrus, inferior temporal gyrus, thalamus, hippocampus, insular gyrus, amygdala, and parahippocampal gyrus. The participants were asked to read Chinese text, which included 407 Chinese characters (covering all Chinese syllables), displayed on a screen after receiving prompts. 1-30, 30-70 and 70-150 Hz frequency band powers of sEEG signals were used as key features. A deep learning model based on long short-term memory (LSTM) was developed to evaluate the contribution of different brain structures during encoding of speech. Prediction of speech characteristics of consonants (articulatory place and manner) and tone within single words based on the selected features and electrode contact locations was made. Cortical signals were generally better at articulatory place prediction (86.5% accuracy, chance level = 12.5%), while cortical and subcortical signals predicted articulatory manner at similar level (51.5% vs 51.7% accuracy, respectively, chance level = 14.3%). Subcortical signals generated better prediction for tone (around 58.3% accuracy, chance level = 25%). Superior temporal gyrus remains highly relevant during speech decoding for both consonants and tone. Prediction reached the highest level when cortical and subcortical inputs were combined, especially for tone prediction. Our findings indicate that both cortical and subcortical structures can play crucial roles for speech decoding, each contributing to different aspects of speech.

show abstract

Section: Discussionmentioning

confidence: 99%

Investigation of contributions from cortical and subcortical brain structures for speech decoding

Wu,

Cai,

Ming

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In the open SEEG dataset [23], ten participants with intractable epilepsy were implanted with SEEG depth electrodes (Dixi Medical, Besancon, France) at the Epilepsy Center Kempenhaeghe in the Netherlands. Intracranial recordings were acquired at 1024 Hz or 2048 Hz using SD LTM amplifiers (Micromed, Treviso, Italy).…”

Section: Open Seeg Datasetmentioning

confidence: 99%

“…Our second aim was to provide appropriate physiological interpretation of our results based on individual contributions of neural features across all available recording sites. Our third aim was to compare our original ECoG dataset against an open SEEG dataset for speech production [23] to elaborate on the overall coverage of brain structures. It must be clearly noted that the effect of feedback on decoding performance was beyond the scope of our analyses.…”

Section: Introductionmentioning

confidence: 99%

Continuous synthesis of artificial speech sounds from human cortical surface recordings during silent speech production

et al. 2023

View full text Add to dashboard Cite

Objective: Brain-computer interfaces can restore various forms of communication in paralyzed patients who have lost their ability to articulate intelligible speech. This study aimed to demonstrate the feasibility of closed-loop synthesis of artificial speech sounds from human cortical surface recordings during silent speech production. Approach: Ten participants with intractable epilepsy were temporarily implanted with intracranial electrode arrays over cortical surfaces. A decoding model that predicted audible outputs directly from patient-specific neural feature inputs was trained during overt word reading and immediately tested with overt, mimed and imagined word reading. Predicted outputs were later assessed objectively against corresponding voice recordings and subjectively through human perceptual judgments. Main results: Artificial speech sounds were successfully synthesized during overt and mimed utterances by two participants with some coverage of the precentral gyrus. About a third of these sounds were correctly identified by naïve listeners in two-alternative forced-choice tasks. A similar outcome could not be achieved during imagined utterances by any of the participants. However, neural feature contribution analyses suggested the presence of exploitable activation patterns during imagined speech in the postcentral gyrus and the superior temporal gyrus. In future work, a more comprehensive coverage of cortical surfaces, including posterior parts of the middle frontal gyrus and the inferior frontal gyrus, could improve synthesis performance during imagined speech. Significance: As the field of speech neuroprostheses is rapidly moving toward clinical trials, this study addressed important considerations about task instructions and brain coverage when conducting research on silent speech with non-target participants.

show abstract

“…While such attempts have shown promise, no existing research has reported investigation on logosyllabic languages, such as Chinese and Thai language, for which the speaking population is more than 1.3 billion 15 . In contrast to the alphabetic languages, where words are constructed from combinations of limited size alphabets (e.g., 26 for English), logosyllabic languages employ logographic characters to represent either entire words or single morphemes, which cannot be typed or spelled out directly. Moreover, logosyllabic languages such as Mandarin Chinese have an extensive inventory of over 50,000 logographic characters 16,17 , each with its unique, complex glyph.…”

Section: Mainmentioning

confidence: 99%

“…Most previous studies on language prosthesis use Electrocorticography (ECoG) electrodes 6,11,13,[22][23][24] or Utah array electrodes 4,10,25 to acquire cortical signals. However, some studies show that the subcortical signals have potential to enhance the effectiveness of decoder 26,27 . The current study utilized stereo-EEG to capture both cortical and subcortical neural signals simultaneously from four participants (two males and two females, all young native Mandarin speakers) undergoing intracranial monitoring for epilepsy (Fig.…”

Section: Mainmentioning

confidence: 99%

Acoustic inspired brain-to-sentence decoder for logosyllabic language

Feng,

Cao,

et al. 2023

Preprint

View full text Add to dashboard Cite

Many severe neurological diseases, such as stroke and amyotrophic lateral sclerosis, can lead to patients completely losing their ability to communicate1,2. Several language brain-computer interface systems have been shown to have the potential to help these patients regain their communication abilities by decoding speech or movement-related neural signals3. However, the aforementioned language brain-computer interfaces (BCIs) were all developed based on alphabetic linguistic systems without one specially designed for logosyllabic languages like Mandarin Chinese. Here, we established the first language BCI specifically designed for Chinese, decoding speech-related stereoelectroencephalography (sEEG) signals into sentences. Firstly, based on the acoustic features of full-spectrum Chinese syllable pronunciation, we constructed prediction models for three syllable elements (initials, tones, and finals). Subsequently, a language model transformed all the predicted syllable elements, integrating them with semantic information, to generate the most probable sentence. Ultimately, we achieved a high-performance decoder with a median character error rate of 29%, demonstrating preliminary potential for clinical application. Our research fills the gap in language BCI for logosyllabic languages and leverages a powerful language model to enhance the performance of the language BCI, offering new insights for future logosyllabic language neuroprosthesis studies.

show abstract

Dataset of Speech Production in intracranial Electroencephalography

Cited by 18 publications

References 84 publications

Investigation of contributions from cortical and subcortical brain structures for speech decoding

Investigation of contributions from cortical and subcortical brain structures for speech decoding

Continuous synthesis of artificial speech sounds from human cortical surface recordings during silent speech production

Acoustic inspired brain-to-sentence decoder for logosyllabic language

Contact Info

Product

Resources

About