What information is necessary for speech categorization? Harnessing variability in the speech signal by integrating cues computed relative to expectations.

Abstract

Most theories of categorization emphasize how continuous perceptual information is mapped to categories. However, equally important is the informational assumptions of a model, the type of information subserving this mapping. This is crucial in speech perception where the signal is variable and context-dependent. This study assessed the informational assumptions of several models of speech categorization, in particular, the number of cues that are the basis of categorization and whether these cues represent the input veridically or have undergone compensation. We collected a corpus of 2880 fricative productions (Jongman, Wayland & Wong, 2000) spanning many talker- and vowel-contexts and measured 24 cues for each. A subset was also presented to listeners in an 8AFC phoneme categorization task. We then trained a common classification model based on logistic regression to categorize the fricative from the cue values, and manipulated the information in the training set to contrast 1) models based on a small number of invariant cues; 2) models using all cues without compensation, and 3) models in which cues underwent compensation for contextual factors. Compensation was modeled by Computing Cues Relative to Expectations (C-CuRE), a new approach to compensation that preserves fine-grained detail in the signal. Only the compensation model achieved a similar accuracy to listeners, and showed the same effects of context. Thus, even simple categorization metrics can overcome the variability in speech when sufficient information is available and compensation schemes like C-CuRE are employed.

A unified account of categorical effects in phonetic perception

“…Finally we consider the fricatives lying between / / and /s/. In terms of cues to identification, these fricatives are identified largely by the static spectral peak locations, though there are many other cues that are also relevant to fricative identification (see (McMurray and Jongman, 2011) for a review). Below we go through more details on the three classes of phonemes more broadly, considering behavioral findings, explanations for categorical effects, and models meant to capture the source of these effects.…”

“…We give further consideration to the difference between isolated and coarticulated vowels as well as consonants below. Fricatives have the widest array of critical cues (see (McMurray and Jongman, 2011) for an overview), including dynamic spectral cues in the form of locus equations (Jongman et al 2000) and changes in moments (Forrest et al 1988), static temporal noise duration (Strevens, 1960;Jassem, 1962), and static spectral cues including F2 onset frequency, spectral moments, and spectral peak location (Jongman et al 2000). Mirman et al (2004) showed that the type of acoustic cue used to distinguish a contrast affects listeners' discrimination behavior, using evidence from listeners' identification and discrimination of non-speech stimuli.…”

Perception in a Variable but Structured World: The Case of Speech Perception

“…This means that speech perception can be understood and modeled as problem of inference under uncertainty (Clayards, Tanenhaus, Aslin, & Jacobs, 2008;Norris & McQueen, 2008). Second, the statistical properties of speech are non-stationary, and the distributions of acoustic cues change from situation to situation (Peterson & Barney, 1952;Allen, J. L. Miller, & DeSteno, 2003;Jongman, Wayland, & Wong, 2000;McMurray & Jongman, 2011;Newman et al, 2001;Hillenbrand, Getty, Clark, & Wheeler, 1995). This has long been recognized as a central challenge to successful speech perception (cf.…”

“…This also imp some phonetic categories may naturally be more (e.g., Allen, J. L. Miller, & DeSteno, 2003;McMurray & Jongman, 2011;Newman et al, 2001). These differences might arise from fixed, physical differences in, for instance, vocal tract size, but they also arise from variable or stylistic factors like language, dialect, or sociolect (e.g., Babel & Benjamin Munson, 2014;Johnson, 2006;Labov, 1972;Pierrehumbert, 2003).…”

“…But as discussed above, this is not always the case: talkers can differ dramatically in the acoustic cues they use to realize phonetic categories, and thus the true likelihood function differs across situa- Miller, & DeSteno, 2003;Labov, 1972;Hillenbrand et al, 1995;McMurray & Jongman, 2011;Pierrehumbert, 2003).…”