Is desynchrony tolerance adaptable in the perceptual organization of speech?

Abstract

Speech signal components that are desynchronized from the veridical temporal pattern lose intelligibility. In contrast, audiovisual presentations with large desynchrony in visible and audible speech streams are perceived without loss of integration. Under such conditions, the limit of desynchrony that permits audiovisual integration is also adaptable. A new project directly investigated the potential for adaptation to consistent desynchrony with unimodal auditory sine-wave speech. Listeners transcribed sentences that are highly intelligible, with veridical temporal properties. Desynchronized variants were created by leading or lagging the tone analog of the second formant relative to the rest of the tones composing the sentences, in 50-msec steps, ranging from 250-msec lead to 250-msec lag. In blocked trials, listeners only tolerated desynchronies <50 msec, and exhibited no gain in intelligibility to consistent desynchrony. Unimodal auditory and bimodal audiovisual forms of perceptual integration evidently exhibit different temporal characteristics, an indication of distinct perceptual functions.

Constraints on Sensitivity to Auditory Modulation in the Perceptual Organization of Speech

“…Error bars represent the 95% confidence interval for differences in desynchrony collapsing over synthesis rate. For comparison, a solid line with round bullets shows the performance reported by Remez et al (2010) on tests with the same desynchronies of the tone analog of the second formant of temporally veridical sine-wave sentences.…”

“…To meet this empirical aim, the items used in these tests were derived from materials originally created for Remez et al (2008; 2010) which sought to sharpen the indices of Greenberg and Arai (1998) and Silipo, Greenberg and Arai (1999). Fifteen sentences spoken by an adult male provided the natural samples from which the test items were produced.…”

“…The abscissa indicates the temporal offset of the tone analog of the second formant relative to the other components of the sine-wave sentence. Intelligibility measures of the effects of a desynchronized tone component in temporally veridical sentences (from Remez et al, 2010) are shown in the round bullets.…”

“…But, at the brief end of this linguistic gradient, estimates of sensory integration have differed, ranging from segmental grain to suprasyllabic grain. In addition, some reports describe a sensory integration window of fixed temporal span (Remez, Ferro, Dubowski, Meer, Broder & Davids, 2010). Yet, because the rate of speech is variable, the modulation characteristic of the auditory samples of speech also varies.…”

Neural constraints and flexibility in language processing

“…Speech sounds, like other auditory signals, are short-lived. Classic speech perception studies have shown that very little of the auditory trace remains after 100 ms (Elliott 1962), with more recent studies indicating that much acoustic information already is lost after just 50 ms (Remez et al 2010). Similarly, and of relevance for the perception of sign language, studies of visual change detection suggest that the ability to maintain visual information beyond 60-70 ms is very limited (Pashler 1988).…”