Timing of brain entrainment to the speech envelope during speaking, listening and self-listening

Pérez, Alejandro; Davis, Matthew H.; Ince, Robin A. A.; Zhang, Hanna; Fu, Zhanao; Lamarca, Melanie; Ralph, Matthew A. Lambon; Monahan, Philip J.

doi:10.1016/j.cognition.2022.105051

Cited by 13 publications

(10 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As the frontally and centrally distributed channels (corresponding to the primary auditory cortex, superior temporal gyrus, premotor cortex, etc.) have been frequently reported to be related to the processing of speech acoustics (Bidelman & Howell, 2016;Broderick et al, 2019;Hickok & Poeppel, 2007;Pérez et al, 2022;Zou et al, 2019) , the present TRF results would imply similar recruitment of these brain regions for acoustic-level processing for naturalistic speech under various noise levels. However, the post-onset 400-ms latency was prolonged as compared to the commonly reported latency of < 300 ms in previous studies (e.g., Broderick et al, 2019;Di Liberto et al, 2015).…”

Section: Discussionsupporting

confidence: 73%

Leading and Following: Noise Differently Affects Semantic and Acoustic Processing during Naturalistic Speech Comprehension

Zhang

et al. 2023

Preprint

View full text Add to dashboard Cite

While our daily verbal communication is challenging with unavoidable environmental noise, the neural mechanisms underlying speech-in-noise comprehension remain to be elucidated. The present study investigated the neural tracking of acoustic and semantic speech information during noisy naturalistic speech comprehension. Participants listened to narrative audios mixed with spectrally matched stationary noise at three signal-to-ratio (SNR) levels and 60-channel electroencephalography signals were recorded. A temporal response function (TRF) method was employed to derive event-related-like responses to the continuous speech stream at both the acoustic and the semantic levels. Whereas the amplitude envelope of the naturalistic speech was taken as the acoustic feature, word entropy and word surprisal were extracted via the natural language processing method as two semantic features. Theta-band frontocentral TRF responses to the acoustic feature were observed at around 400 ms following speech fluctuation onset over all three SNR levels, and the response latency was more delayed with increasing SNR. Delta-band frontal TRF responses to the semantic feature of word entropy were observed at around 200 to 600 ms leading to speech fluctuation onset over all three SNR levels, and the response latency became more leading with increasing SNR. While the following responses to speech acoustics were consistent with previous studies, the leading responses to speech semantics suggest our brain could be taking an active approach to deal with noisy speech comprehension.

show abstract

Section: Discussionsupporting

confidence: 73%

Leading and Following: Noise Differently Affects Semantic and Acoustic Processing during Naturalistic Speech Comprehension

Zhang

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…For a more in-depth explanation of Gaussian copula MI, we refer to Ince et al (2017). Over the past recent years, the Gaussian copula MI has been applied in numerous neural envelope tracking papers (Coopmans et al, 2022; Daube et al, 2019; Giordano et al, 2017; Perez et al, 2022).…”

Section: Calculating Mutual Informationmentioning

confidence: 99%

“…In sum, the advantage of MI is that it can find statistical effects beyond linear relationships while retaining spatial and temporal interpretations. For these reasons, MI analyses are gradually becoming a more popular approach to studying neural envelope tracking (Chalas et al, 2022; Coopmans et al, 2022; Daube et al, 2019; Giordano et al, 2017; Kaufeld et al, 2020; Keitel et al, 2018; Keshavarzi et al, 2021; Perez et al, 2022; Pfeffer et al, 2022). However, the field currently lacks a proper comparison between the different methodologies that exist to derive MI.…”

Section: Introductionmentioning

confidence: 99%

Beyond Linear Neural Envelope Tracking: A Mutual Information Approach

Clercq

Vanthornhout

Vandermosten

et al. 2022

Preprint

View full text Add to dashboard Cite

The human brain tracks the temporal envelope of speech, which contains essential cues for speech understanding. Linear models are the most common tool to study neural envelope tracking. However, information on how speech is processed can be lost since nonlinear relations are precluded. As an alternative, mutual information (MI) analysis can detect both linear and nonlinear relations. Yet, several different approaches to calculating MI are applied without consensus on which approach to use. Furthermore, the added value of nonlinear techniques remains a subject of debate in the field. To resolve this, we applied linear and MI analyses to electroencephalography (EEG) data of participants listening to continuous speech. Comparing the different MI approaches, we conclude that results are most reliable and robust using the Gaussian copula approach, which first transforms the data to standard Gaussians. With this approach, the MI analysis is a valid technique for studying neural envelope tracking. Like linear models, it allows spatial and temporal interpretations of speech processing, peak latency analyses, and applications to multiple EEG channels combined. Finally, we demonstrate that the MI analysis can detect nonlinear components on the single-subject level, beyond the limits of linear models. We conclude that the MI analysis is a more informative tool for studying neural envelope tracking.

show abstract

“…First, we provided evidence of zero-lagged enhanced inter-brain similarities during a turn-taking conversation (Pérez, Carreiras, & Duñabeitia, 2017). Second, we provided evidence of differential timing for the engagement to the speech audio (namely, speech tracking) depending on the conversational role (Pérez et al, 2022). Specifically, we described a timeline for speech tracking in speakers and listeners with maximal speech tracking after the auditory presentation during perception (approximately 110 ms) and before vocalisation during speech production (approximately 25 ms).…”

Section: Instantaneous or Lagged Inter-brain Relationships?mentioning

confidence: 79%

“…Moreover, single subject level 'inter-brain' results can help to define which individual within a pair or group is driving an atypical inter-brain pattern. Perhaps most importantly, we can compare speaking and listening without stimulus differences and set up an 'inter-brain' situation in which accurate predictions for upcoming speech signals are possible and complete comprehension is expected (Pérez et al, 2022). This 'perfect prediction' of upcoming speech constitutes an interesting feature for experiments testing the mutual prediction theory.…”

Section: Emergent Property Versus Shared Stimulationmentioning

confidence: 99%

Speaking and listening to inter-brain relationships

Pérez¹,

Davis²

2022

Preprint

View full text Add to dashboard Cite

Studies of the inter-brain relationship thrive, and yet many reservations regarding their scope and interpretation of these phenomena have been raised by the scientific community. It is thus essential to establish common ground on methodological and conceptual definitions related to this topic and to open debate about any remaining points of uncertainty. We here offer insights to improve the conceptual clarity and empirical standards offered by social neuroscience studies of inter-personal interaction using hyperscanning with a particular focus on verbal communication.

show abstract

Timing of brain entrainment to the speech envelope during speaking, listening and self-listening

Cited by 13 publications

References 60 publications

Leading and Following: Noise Differently Affects Semantic and Acoustic Processing during Naturalistic Speech Comprehension

Leading and Following: Noise Differently Affects Semantic and Acoustic Processing during Naturalistic Speech Comprehension

Beyond Linear Neural Envelope Tracking: A Mutual Information Approach

Speaking and listening to inter-brain relationships

Contact Info

Product

Resources

About