Dimension-selective attention as a possible driver of dynamic, context-dependent re-weighting in speech processing

Abstract

The contribution of acoustic dimensions to an auditory percept is dynamically adjusted and reweighted based on prior experience about how informative these dimensions are across the long-term and short-term environment. This is especially evident in speech perception, where listeners differentially weight information across multiple acoustic dimensions, and use this information selectively to update expectations about future sounds. The dynamic and selective adjustment of how acoustic input dimensions contribute to perception has made it tempting to conceive of this as a form of non-spatial auditory selective attention. Here, we review several human speech perception phenomena that might be consistent with auditory selective attention although, as of yet, the literature does not definitively support a mechanistic tie. We relate these human perceptual phenomena to illustrative nonhuman animal neurobiological findings that offer informative guideposts in how to test mechanistic connections. We next present a novel empirical approach that can serve as a methodological bridge from human research to animal neurobiological studies. Finally, we describe four preliminary results that demonstrate its utility in advancing understanding of human non-spatial dimension-based auditory selective attention.

Effects of language experience on domain-general perceptual strategies

“…Another class of models which aim to explain the effects of language experience on dimensional weighting during categorization are dimension-selectiveattention accounts, which suggest that up-weighted dimensions are more salient, tending to capture selective attention and therefore influence categorization to a greater extent (Gordon et al 1993, Francis and Nusbaum 2002, Holt et al 2018. That language experience can have an effect on dimensional salience in speech is supported by the results of Experiment 3, which showed that Mandarin speakers continue to be greatly influenced by pitch information even when explicitly asked to direct their attention away from pitch and towards another dimension.…”

“…For example, according to these models, when native Mandarin speakers categorize English syllables as stressed or unstressed, they refer to tone categories in Mandarin, and so place more weight on pitch information (and less on other cues) compared to native English speakers, given that pitch is by far the most important dimension for distinguishing between lexical tones (although other cues such as duration can be used if pitch is neutralized, such as in whispered speech; Lin and Repp 1989, Blicher, Diehl & Cohen 1990, Whalen and Xu 1992, Fu and Zeng 2000, Liu and Samuel 2004. Attentional theories of how information from different acoustic dimensions is combined during speech perception suggest that dimensions that have a greater tendency to capture attention are more strongly weighted (Gordon, Eberhardt & Rueckl 1993, Francis and Nusbaum 2002, Holt et al 2018. According to these models, repeated task-relevance of a particular dimension leads to an increase in perceptual salience of that dimension, thereby increasing the weight that is assigned to that dimension as a source of evidence during perception.…”

“…One possibility is that repeated task-relevance of a particular dimension leads it to become more salient, i.e. have a greater tendency to capture an individual's attention, and that this increased salience leads the dimension to have greater influence during perceptual categorization (Gordon et al 1993, Francis and Nusbaum 2002, Holt et al 2018. In experiment 3 we tested one prediction of this account, which is that Mandarin speakers would have difficulty ignoring pitch and focusing on other information during a simple perceptual comparison task.…”

Tailored perception: listeners’ strategies for perceiving speech fit their individual perceptual abilities

“…However, perceptual weights are not fixed: dimensional weights shift when the fidelity of the signal is degraded (for review see Holt, Tierney, Guerra, Laffere, & Dick, 2018). For example, when speech is presented in masking noise, VOT is harder to detect, making F0 more effective at signaling voicing categories.…”

Attentional modulation of neural phase is enhanced by short-term training and linked to musical experience

“…Despite the importance of robust auditory selective attention for everyday listening, there are large individual differences in the ability to listen and respond to single auditory streams in the presence of distractors (47). The neural foundations of individual differences in selective attention are however poorly understood, as previous studies of the neural correlates of selective attention have used small numbers of participants and focused their analyses on main effects of attention.…”