Multivariate analysis of speech envelope tracking reveals coupling beyond auditory cortex

Chalas, Nikos; Daube, Christoph; Kluger, Daniel S.; Abbasi, Omid; Nitsch, Robert; Groß, Joachim

doi:10.1016/j.neuroimage.2022.119395

Cited by 23 publications

(18 citation statements)

References 93 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our modeling approach was targeting the temporal dynamics of the speech and music stimuli. Beyond confirming that acoustic dynamics are strongly tracked by auditory neural dynamics, it revealed, investigating the entire cortex, that such neural tracking also occurs well outside of auditory regions -up to motor and inferior frontal areas (Figure 3B, see also Chalas et al, 2022;Zion Golumbic et al, 2013). Of note, this spatial map of speech dynamics encoding is very similar to former reports of the brain regions belonging to the language system (Diachek et al, 2020).…”

Section: P R E P R I Nsupporting

confidence: 85%

Speech and music recruit frequency-specific distributed and overlapping cortical networks

Rietmolen¹,

Mercier²,

Trébuchon

et al. 2022

Preprint

View full text Add to dashboard Cite

To what extent do speech and music processing rely on domain-specific and domain-general neural networks? Adopting a dynamical system framework, we investigate the presence of frequency-specific and network-level selectivity and combine it with a statistical approach in which a clear distinction is made between shared, preferred, and category-selective neural responses. Using intracranial EEG recordings in 18 epilepsy patients listening to natural and continuous speech and music, we show that the majority of focal and network-level neural activity is shared between speech and music processing. Our data also reveal an absence of regional selectivity. Instead, neural selectivity is restricted to dis- tributed and frequency-specific coherent oscillations, typical of spectral fingerprints. Our work addresses a longstanding debate and redefines an epistemological posture on how to map cognitive and brain functions.

show abstract

Section: P R E P R I Nsupporting

confidence: 85%

Speech and music recruit frequency-specific distributed and overlapping cortical networks

Rietmolen¹,

Mercier²,

Trébuchon

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…When tracking story speech, the distribution of brain sources included not only auditory regions but also frontoparietal cognitive regions. This is consistent with recent findings based on MEG and functional MRI [14,48], which have higher spatial resolution. The bilateral temporal lobe plays a vital role in the perception of acoustic signals.…”

Section: Neural Entrainment Patterns In the Source Spacesupporting

confidence: 93%

Study on neural entrainment to continuous speech using dynamic source connectivity analysis

Yang

Zhou

et al. 2023

J. Neural Eng.

View full text Add to dashboard Cite

Objective. Many recent studies investigating the processing of continuous natural speech have employed electroencephalography (EEG) due to its high temporal resolution. However, most of these studies explored the response mechanism limited to the electrode space. In this study, we intend to explore the underlying neural processing in the source space, particularly the dynamic functional interactions among different regions during neural entrainment to speech. Approach. We collected 128-channel EEG data while 22 participants listened to story speech and time-reversed speech using a naturalistic paradigm. We compared three different strategies to determine the best method to estimate the neural tracking responses from the sensor space to the brain source space. After that, we used dynamic graph theory to investigate the source connectivity dynamics among regions that were involved in speech tracking. Main result. By comparing the correlations between the predicted neural response and the original common neural response under the two experimental conditions, we found that estimating the common neural response of participants in the electrode space followed by source localization of neural responses achieved the best performance. Analysis of the distribution of brain sources entrained to story speech envelopes showed that not only auditory regions but also frontoparietal cognitive regions were recruited, indicating a hierarchical processing mechanism of speech. Further analysis of inter-region interactions based on dynamic graph theory found that neural entrainment to speech operates across multiple brain regions along the hierarchical structure, among which the bilateral insula, temporal lobe, and inferior frontal gyrus are key brain regions that control information transmission. All of these information flows result in dynamic fluctuations in functional connection strength and network topology over time, reflecting both bottom-up and top-down processing while orchestrating computations toward understanding. Significance. Our findings have important implications for understanding the neural mechanisms of the brain during processing natural speech stimuli.

show abstract

“…Neural speech tracking is widely used to study the neural processing of continuous speech, though primarily with audio-only stimuli (Brodbeck, Hong, et al, 2018;Chalas et al, 2022;Di Liberto et al, 2015;Keitel et al, 2018). Recent studies have used audiovisual speech settings, but without directly modeling the visual speech features Golumbic et al, 2013) or not incorporating their temporal dynamics due to the use of frequency-based methods (Aller et al, 2022;Bröhl et al, 2022;Park et al, 2016).…”

Section: Discussionmentioning

confidence: 99%

Neural speech tracking benefit of lip movements predicts behavioral deterioration when the speaker’s mouth is occluded

Reisinger

Gillis

Suess

et al. 2023

Preprint

View full text Add to dashboard Cite

Observing lip movements of a speaker is known to facilitate speech understanding, especially in challenging listening situations. Converging evidence from neuroscientific studies shows enhanced processing of audiovisual stimuli. However, the interindividual variability of this visual benefit and its consequences on behavior are unknown. Here, we analyzed source-localized magnetoencephalographic (MEG) responses from normal-hearing participants listening to audiovisual speech with or without an additional distractor speaker. Using temporal response functions (TRFs), we show that neural responses to lip movements are, in general, enhanced when speech is challenging. After conducting a crucial control for speech acoustics, we show that lip movements effectively contribute to higher neural speech tracking, particularly when a distractor speaker is present. However, the extent of this visual benefit varied greatly among participants. Probing the behavioral relevance, we show that individuals who benefit more from lip movement information in terms of neural speech tracking, show a stronger drop in performance and an increase in perceived difficulty when the mouth is occluded by a surgical face mask. By contrast, no effect was found when the mouth was not occluded. We provide novel insights on how the benefit of lip movements in terms of neural speech tracking varies among individuals. Furthermore, we reveal its behavioral relevance by demonstrating negative consequences for behavior when visual speech is absent. Our results also offer potential implications for future objective assessments of audiovisual speech perception.

show abstract

Multivariate analysis of speech envelope tracking reveals coupling beyond auditory cortex

Cited by 23 publications

References 93 publications

Speech and music recruit frequency-specific distributed and overlapping cortical networks

Speech and music recruit frequency-specific distributed and overlapping cortical networks

Study on neural entrainment to continuous speech using dynamic source connectivity analysis

Neural speech tracking benefit of lip movements predicts behavioral deterioration when the speaker’s mouth is occluded

Contact Info

Product

Resources

About