Audio-visual identification of place of articulation and voicing in white and babble noise

Abstract

Research shows that noise and phonetic attributes influence the degree to which auditory and visual modalities are used in audio-visual speech perception (AVSP). Research has, however, mainly focused on white noise and single phonetic attributes, thus neglecting the more common babble noise and possible interactions between phonetic attributes. This study explores whether white and babble noise differentially influence AVSP and whether these differences depend on phonetic attributes. White and babble noise of 0 and -12 dB signal-to-noise ratio were added to congruent and incongruent audio-visual stop consonant-vowel stimuli. The audio (A) and video (V) of incongruent stimuli differed either in place of articulation (POA) or voicing. Responses from 15 young adults show that, compared to white noise, babble resulted in more audio responses for POA stimuli, and fewer for voicing stimuli. Voiced syllables received more audio responses than voiceless syllables. Results can be attributed to discrepancies in the acoustic spectra of both the noise and speech target. Voiced consonants may be more auditorily salient than voiceless consonants which are more spectrally similar to white noise. Visual cues contribute to identification of voicing, but only if the POA is visually salient and auditorily susceptible to the noise type.

Audio-visual speech experience with age influences perceived audio-visual asynchrony in speech

“…In AV speech research white noise is frequently applied as a masker (e.g., Dodd, 1977;Easton and Basala, 1982;Fixmer and Hawkins, 1998), whereas the more naturally occurring babble noise is seldom used (cf. Alm et al, 2009). The masking effect of babble and white noise may vary for different phonetic attributes, for instance, place of articulation and voicing (Alm et al, 2009).…”

“…Alm et al, 2009). The masking effect of babble and white noise may vary for different phonetic attributes, for instance, place of articulation and voicing (Alm et al, 2009).…”

Individual Differences in L2 Speech Perception: The Role of Phonological Memory and Lipreading Ability

“…Speech perception is not an exclusively auditory process, and access to visual information about a speaker (i.e., gesture, facial movements, mouth shape) often provides an important source of information by supplementing the auditory signal (Sumby & Pollack, ). It is not surprising that numerous studies have shown that adding visual information to the auditory signal can greatly enhance perception when speech is degraded by noise (e.g., Alm et al., ; Rosenblum, Johnson, & Saldaña, ; Sommers, Tye–Murray, & Spehar, ). The benefit of seeing a speaker's face has also been shown to facilitate comprehension of sung lyrics (Jesse & Massaro, ), understanding of speech that is heavily accented or with a particularly dense content (Arnold & Hill, ; Reisberg, McLean, & Goldfield, ), L2 listening comprehension (Sueyoshi & Hardison, ), and even L1 acquisition, as evidenced by the delay in the acquisition of the visually salient /m/–/n/ contrast by blind children (Mills, ).…”

Confusões perceptivas entre consoantes do Português Brasileiro em função do ruído

“…In the present study, the confusions [p] -[f] and [ʃ] -[k] also revealed a difference in articulation point. The literature (14) shows that the perception of the articulation point is more impaired under white noise.…”