Neural mechanisms for selectively tuning in to the target speaker in a naturalistic noisy situation

Abstract

The neural mechanism for selectively tuning in to a target speaker while tuning out the others in a multi-speaker situation (i.e., the cocktail-party effect) remains elusive. Here we addressed this issue by measuring brain activity simultaneously from a listener and from multiple speakers while they were involved in naturalistic conversations. Results consistently show selectively enhanced interpersonal neural synchronization (INS) between the listener and the attended speaker at left temporal–parietal junction, compared with that between the listener and the unattended speaker across different multi-speaker situations. Moreover, INS increases significantly prior to the occurrence of verbal responses, and even when the listener’s brain activity precedes that of the speaker. The INS increase is independent of brain-to-speech synchronization in both the anatomical location and frequency range. These findings suggest that INS underlies the selective process in a multi-speaker situation through neural predictions at the content level but not the sensory level of speech.

Enhanced amplitude modulations contribute to the Lombard intelligibility benefit: Evidence from the Nijmegen Corpus of Lombard Speech

“…Instead, we interpret the outcomes of the present study in light of neurobiological models of speech perception (Ghitza, 2011;Giraud & Poeppel, 2012;Peelle & Davis, 2012) that posit a central role for endogenous theta oscillations closely following the syllabic rhythm of speech (Arnal et al, 2015;Bosker & Ghitza, 2018;Bosker & Kösem, 2017;Kösem et al, 2018). Applying these models to speech-in-noise and speech-in-speech comprehension, a range of electrophysiological studies have provided evidence that listeners' envelope-tracking response to an attended speaker is amplified compared to an ignored speaker (Dai et al, 2018;Ding & Simon, 2012;Golumbic, Cogan, et al, 2013;Lakatos et al, 2008;Mesgarani & Chang, 2012). This dynamic neural representation of the temporal structure of the attended speech stream (e.g., in a noisy environment, or with a competing speech signal) is thought to function as an amplifier and a temporal filter, aiding speech comprehension in challenging listening conditions.…”

What Can Computational Models Learn From Human Selective Attention? A Review From an Audiovisual Unimodal and Crossmodal Perspective

“…Besides, the INS increased only for the noisy naturalistic conversations with competing speech but not for the two-person conversation and was only associated with the speech content. Their findings implied that the prediction of the speaker's speech content might play an important role in the Cocktail Party Effect (Dai et al, 2018). In summary, the human brain's auditory processing during the Cocktail Party problem is not hierarchical but heterarchical, which is mainly a bottom-up process aided by top-down modulation (Bregman, 1994).…”

Instructor-learner brain coupling discriminates between instructional approaches and predicts learning

“…725 4.7.3. Analysis step C: Brain-to-brain coupling prediction 726 Finally, we explored whether brain-to-brain coupling allowed us to predict if an 727 instructor employed the scaffolding or explanation strategy, using a decoding analysis 728 (Dai et al, 2018;Jiang et al, 2015). The analysis details and strategies can be 729 described as follows.…”