Perceptual Plasticity for Auditory Object Recognition

Abstract

In our auditory environment, we rarely experience the exact acoustic waveform twice. This is especially true for communicative signals that have meaning for listeners. In speech and music, the acoustic signal changes as a function of the talker (or instrument), speaking (or playing) rate, and room acoustics, to name a few factors. Yet, despite this acoustic variability, we are able to recognize a sentence or melody as the same across various kinds of acoustic inputs and determine meaning based on listening goals, expectations, context, and experience. The recognition process relates acoustic signals to prior experience despite variability in signal-relevant and signal-irrelevant acoustic properties, some of which could be considered as “noise” in service of a recognition goal. However, some acoustic variability, if systematic, is lawful and can be exploited by listeners to aid in recognition. Perceivable changes in systematic variability can herald a need for listeners to reorganize perception and reorient their attention to more immediately signal-relevant cues. This view is not incorporated currently in many extant theories of auditory perception, which traditionally reduce psychological or neural representations of perceptual objects and the processes that act on them to static entities. While this reduction is likely done for the sake of empirical tractability, such a reduction may seriously distort the perceptual process to be modeled. We argue that perceptual representations, as well as the processes underlying perception, are dynamically determined by an interaction between the uncertainty of the auditory signal and constraints of context. This suggests that the process of auditory recognition is highly context-dependent in that the identity of a given auditory object may be intrinsically tied to its preceding context. To argue for the flexible neural and psychological updating of sound-to-meaning mappings across speech and music, we draw upon examples of perceptual categories that are thought to be highly stable. This framework suggests that the process of auditory recognition cannot be divorced from the short-term context in which an auditory object is presented. Implications for auditory category acquisition and extant models of auditory perception, both cognitive and neural, are discussed.

Rear negativity: Verbal messages coming from behind are perceived as more negative

“…In the social domain, the spoken word is a major tool of communication (Chomsky, /2009; Scott‐Phillips, ). Therefore, interpreting spoken messages is an essential part of interactions with other people (Heald, Hedger, & Nusbaum, ). The content of the message is the most important part of its interpretation, but it is not the only one.…”

Integrating prediction errors at two time scales permits rapid recalibration of speech sound categories

“…The change in /ada/ effectively leads to category boundary shifts in acoustic continua, boundary shifts that are transient and will only add-up after consistent repetitions. Importantly, these effects are distinct from perceptual bias and specific adaptation, and from other documented short-term effects due to changes in the relative informational content of partially redundant acoustic features (Heald, Van Hedger, and Nusbaum 2017;Gabay and Holt 2018).…”

Integrating prediction errors at two time scales permits rapid recalibration of speech sound categories

“…Speech sound categories are constantly revised as a function of the most recently presented stimuli (Samuel and Kraljic 2009;Kleinschmidt and Jaeger 2015;Heald, Van Hedger, and Nusbaum 2017). The proposed model provides a possible account for fast and transient changes in speech sound categories when confronted to a volatile sensory environment, involving the constant recalibration of internal models of speech.…”

“…The model was motivated by experimental results showing that /aba/ sounds were more frequently mis-categorized as 'ada' when they were preceded by a fused McGurk (Lüttke et al 2016). Since the reported effect was distinct from other well-documented across-trial dependency effects, such as perceptual priming or selective adaptation (Heald, Van Hedger, and Nusbaum 2017;Gabay and Holt 2018), we sought to model it considering only changes in perceived speech categories without external feedback. Like previous approaches, ours builds on the idea that the brain achieves perception by inverting a generative model (e.g., Rao and Ballard 1999;Knill and Pouget 2004;Karl J. Friston 2005) and by continuously monitoring its performance to adapt it to changing stimulus landscapes.…”