Selective cortical representation of attended speaker in multi-talker speech perception

Abstract

Humans possess a remarkable ability to attend to a single speaker’s voice in a multi-talker background1–3. How the auditory system manages to extract intelligible speech under such acoustically complex and adverse listening conditions is not known, and, indeed, it is not clear how attended speech is internally represented4,5. Here, using multi-electrode surface recordings from the cortex of subjects engaged in a listening task with two simultaneous speakers, we demonstrate that population responses in non-primary human auditory cortex encode critical features of attended speech: speech spectrograms reconstructed based on cortical responses to the mixture of speakers reveal the salient spectral and temporal features of the attended speaker, as if subjects were listening to that speaker alone. A simple classifier trained solely on examples of single speakers can decode both attended words and speaker identity. We find that task performance is well predicted by a rapid increase in attention-modulated neural selectivity across both single-electrode and population-level cortical responses. These findings demonstrate that the cortical representation of speech does not merely reflect the external acoustic environment, but instead gives rise to the perceptual aspects relevant for the listener’s intended goal.

Mechanisms of noise robust representation of speech in primary auditory cortex

“…The functional properties observed for speech and conspecific vocalizations in distorted conditions may reflect computational strategies used in these behaviors, and future studies that combine our unique computational approach and behavior paradigms promise valuable new insight into the general problem of identifying signals in the background of interfering sources (5). Although our focus in this study was to determine how different mechanisms contribute to the formation of the cortical representation under convolutive and additive distortions, a wider range of conditions is necessary to expand the findings and to determine how these mechanisms contribute to signal enhancement in various stationary, nonstationary, and, ultimately, competing vocalizations (35).…”

Spatiotemporal dynamics of auditory attention synchronize with speech

“…The ipsi-and contralateral sensors were used to calculate the second attention index, that is, the alpha lateralization index [ALI α = (α att_ipsi -α att_contra )/(α att_ipsi + α att_contra )]. The ALI was calculated over the time course of the trial using a 250-ms sliding window (54) and revealed a temporally resolved measure of attention effects on alpha power (Fig. 5A).…”

Effects of Spectral Degradation on Attentional Modulation of Cortical Auditory Responses to Continuous Speech

“…Recent electrophysiological studies on normalhearing listeners provided further evidence for object-based processing during speech stream segregation (e.g., Ding and Simon 2012a; Kerlin et al 2010;Mesgarani and Chang 2012), in which attention enhances cortical responses to the attended speech stream and reduces the responses to the ignored speech stream (e.g., Horton et al 2013Horton et al , 2014Kong et al 2014;O'Sullivan et al 2014). Ding and Simon (2012a) varied the attended and background speakers separately over a wide range of intensity and found that the neural representation of the attended speech did not adapt to the intensity of the background speaker, which strongly suggests an object-level intensity gain control.…”