Audio/visual compensation for coarticulation

Abstract

This study investigates how visual phonetic information affects compensation for coarticulation in speech perception. A series of CV syllables with fricative continuum from [s] to [sh] before [a],[u] and [y] was overlaid with a video of a face saying [s]V, [∫]V, or a visual blend of the two fricatives. We made separate movies for each vowel environment. We collected [s]/[∫] boundary locations from 24 native English speakers. In a test of audio-visual integration, [∫] videos showed significantly lower boundary locations (more [sh] responses) than [s] videos (t[23]=2.9, p<0.01) in the [a] vowel environment. Regardless of visual fricative condition, the participants showed a compensation effect with [u] (t[23] > 3, p<0.01), but not with the unfamiliar vowel [y]. This pattern of results was similar to our findings from an audio-only version of the experiment, implying that the compensation effect was not strengthened by seeing the lip rounding of [y].