Linguistic processing of task-irrelevant speech at a Cocktail Party

Abstract

Paying attention to one speaker in noisy environments can be extremely difficult. This is because task-irrelevant speech competes for processing resources with attended speech. However, whether this competition is restricted to acoustic-phonetic interference, or if it extends to competition for linguistic processing as well, remains highly debated. To address this debate, here we test whether task-irrelevant speech sounds are integrated over time to form hierarchical representations of lexical and syntactic structures.Neural activity was recorded using Magnetoencephalography (MEG) during a dichotic listening task, where human participants attended to natural speech presented to one ear, and task-irrelevant stimuli were presented to the other. Task-irrelevant stimuli consisted either of random sequences of syllables (Non-Structured), or syllables ordered to form coherent sentences (Structured). Using hierarchical frequency-tagging, the neural signature of different linguistic-hierarchies within the Structured stimuli – namely words, phrases and sentences – can be uniquely discerned from the neural response.We find that, indeed, the phrasal structure of task-irrelevant stimuli was represented in the neural response, primarily in left inferior frontal and posterior parietal regions. Moreover, neural tracking of attended speech in left inferior frontal regions was enhanced when task-irrelevant stimuli were linguistically structured. This pattern suggests that syntactic structurebuilding processes are applied to task-irrelevant speech, at least under these circumstances, and that selective attention does not fully eliminate linguistic processing of task-irrelevant speech. Rather, the inherent competition for linguistic processing resources between the two streams likely results in the increased listening effort experienced when trying to focus selective attention in multi-speaker contexts.Significance statementThis study addresses the fundamental question of how the brain deals with competing speech in noisy environments. Specifically, we ask: when one attempts to focus their attention on a particular speaker, what level of linguistic processing is applied to other, task-irrelevant speech? By measuring neural activity, we find evidence that the phrasal structure of task-irrelevant speech is indeed discerned, indicating that linguistic information is integrated over time and undergoes some syntactic analysis. Moreover, neural responses to attended speech were also enhanced in speech-processing regions, when presented together with comprehensible yet task-irrelevant speech. These results nicely demonstrate the inherent competition for linguistic processing resources among concurrent speech, providing evidence that selective attention does not fully eliminate linguistic processing of task-irrelevant speech.

Invariant neural subspaces maintained by feedback modulation

“…However, separating more complex sounds requires additional active top-down processes (Parthasarathy et al, 2020; Oberfeld and Kloeckner-Nowotny, 2016). In our model top-down feedback guides the source separation itself, while the selection of a source would occur at a later processing stage – consistent with recent evidence for “late selection”(Brodbeck et al, 2020; Yahav and Golumbic, 2021).…”

An ecological investigation of the capacity to follow simultaneous speech and preferential detection of ones’ own name

“…Moreover, adequate performance on this dual-task was not correlated with individual working-memory capacity (WMC), further supporting its relative low-cognitive-demand nature (Conway et al, 2001; Colflesh and Conway, 2007; Gygi and Shafiro, 2012; Naveh-Benjamin et al, 2014). At the same time, the neural speech tracking analysis shows that the Narrative Stream was represented more robustly than the Barista Stream, a pattern reminiscent of the enhanced speech-tracking of task-relevant speech in selective attention studies (Kerlin et al, 2010; Ding and Simon, 2012b, 2012a; Mesgarani and Chang, 2012; Power et al, 2012; Zion Golumbic et al, 2013; O’Sullivan et al, 2015; Fuglsang et al, 2017; Fiedler et al, 2019; Har-shai Yahav and Zion Golumbic, 2021; Kaufman and Golumbic, 2022). In discussing this result, we acknowledge the possibility that the lack of a reliable speech-tracking response to the Barista Stream may be due, at least in part, to the highly structured nature of this stimulus which contains substantial autocorrelation.…”

“…Research into this question has largely focused on selective attention paradigms - where participants are instructed to listen to one ‘main’ stimulus and perform a task, while disregarding other task-irrelevant stimuli (Cherry, 1953; Broadbent, 1958; Lane and Pearson, 1982; Driver, 2001; Ding et al, 2018). However, these selective attention paradigms have yielded mixed results regarding the depth of processing that is applied to task-irrelevant speech, with some studies suggesting that task-irrelevant speech is only represented at an acoustic level but not at a semantic/linguistic level, while others do find evidence for some linguistic processing of task-irrelevant speech (Dupoux et al, 2003; Brodbeck et al, 2020; Dai et al, 2021; Har-shai Yahav and Zion Golumbic, 2021), particularly if it contains salient content words (Moray, 1959; Treisman, 1960; Wood and Cowan, 1995; Rivenez et al, 2008) Perhaps one of the most well-known demonstrations of this phenomena is the conscious detection of one’s own name in supposedly “unattended” speech (Moray, 1959; Wood and Cowan, 1995; Conway et al, 2001; Tamura et al, 2012; Tateuchi et al, 2012; Röer et al, 2013; Naveh-Benjamin et al, 2014; Holtze et al, 2021). However, one major tension in interpreting these results is the ambiguity regarding what participants actually do in selective attention tasks.…”

Effect of Audiovisual Cross-Modal Conflict during Working Memory Tasks: A Near-Infrared Spectroscopy Study

“…Additionally, the inferior parietal cortex (IPC) plays an important role in filtering visual distractors [13]. In unimodal auditory studies, inhibition of auditory distraction, which disrupts WM performance, requires the activation of the lateral prefrontal cortex, especially the VLPFC [14,15]. Meanwhile, few neuroimaging studies have focused on cross-modal conflicts; in particular, only a few studies have focused on auditory targets with visual distractors [1].…”