Perception in a Variable but Structured World: The Case of Speech Perception

Abstract

Perceptual systems have to make sense out of a world that is not only noisy and ambiguous, but that also varies from situation to situation. Human speech perception is a perceptual domain where this problem has long been acknowledged: individual talkers vary substantially in how they produce linguistic units using acoustic cues. Yet, how the speech system solves this problem of talker variability remains poorly understood. This thesis presents a computational framework---the ideal adapter---for understanding this problem and how the speech perception system solves it. The basic insight of this framework is that variability in speech is not arbitrary but rather structured: talkers are reasonably consistent in the way they produce cues, and individual talkers tend to cluster into groups by gender, regional background, etc. This structure means that listeners can use their previous experience with other talkers to guide perception of unfamiliar talkers, as well as familiar talkers that they encounter again. This framework unifies a large and messy literature on how listeners cope with talker variability, leads to quantitative models that provide good fits to human behavior in a variety of situations, and makes specific, testable predictions that open up new frontiers in understanding speech perception. This framework also applies to perception in general, and highlights how speech perception can serve as a model organism for understanding how perceptual systems cope with a variable but structured world.

“Big” Sounds Bigger in More Widely Spoken Languages

“…One of the properties in which languages of larger and smaller communities differ is the greater input variability in larger communities. Phonetically, individuals vary more in their production of vowels than of consonants (Kleinschmidt, 2016). Therefore, vowels would be a less reliable cue in larger communities than smaller communities.…”

Phonetic cue weighting in perception and production

“…Recently, the development of frameworks to model an "ideal listener" (e.g., Kleinschmidt, 2016) have provided an exciting opportunity for computationally implementable models that can be informed by what we know about speech perception and tested through comparison with empirical findings. Since listeners' judgments are influenced by category overlap, variance, and other factors like perceptual salience of a dimension, we think that an accurate computationally implementable model will likely need to include this information.…”

The influence of social network size on speech perception

“…Success at identifying vowels in noise is one measure that reflects the robustness of one’s vowel category representations. The decision to focus on vowel perception was due to the fact that even though variation exists at all levels, research shows that variation is much greater across vowels than it is across consonants (Kleinschmidt, 2016), and even more importantly, that variation in vowel production is structured by indexical factors, whereas other types of variation, such as for VOTs in stops, is not (Allen, Miller, & DeSteno, 2003; Kleinschmidt, 2016). Correspondingly, while past research on vowel production showed its dependence on indexical properties, past research on variability in consonant production has mostly shown its dependence on phonetic context and speech style.…”

“…For example, vowel production has been shown to be influenced by sex, vocal tract size and shape, and dialect (e.g., Bachorowski & Owren, 1999; Peterson & Barney, 1952). In contrast, Allen et al (2003) discovered that sex differences in VOTs are eliminated once speech rate is controlled for, and Kleinschmidt (2016) similarly found that indexical properties did not predict VOT production yet did account for variation in vowel production. Exposure to multiple speakers might therefore increase input variability for vowels more than for consonants, and importantly, it will allow listeners to learn the conditioning of this variability, and thus assist in perception of vowels by new speakers.…”