Perceptual dominance during lipreading

Abstract

Two experiments were performed under visual-only and visual-auditory discrepancy conditions (dubs) to assess observers' abilities to read speech information on a face. In the first experiment, identification and multiple choice testing were used. In addition, the relation between visual and auditory phonetic information was manipulated and related to perceptual bias. In the eecond experiment, the "eompellingn8ls" of the visual·auditory discrepancy as a single speech event was manipulated. Subjects alao rated the confidence they had that their perception of the lipped word was accurate. Results indicated that competing visual information exerted little effect on auditory speech recognition, but visual speech recognition was substantially interfered with when discrepant auditory information was present. The extent of auditory bias was found to be related to the abilities of observers to read speech under nondiscrepancy conditions, the magnitude of the visual-auditory discrepancy, and the compellingness of the visual-auditory discrepancy as a single event. Auditory bias during speech was found to be a moderately compelling conscious experience, and not simply a case of confused responding or guessing. Results were discussed in terms of current models of perceptual dominance and related to results from modality discordance during space perception.

Audiovisual integration in perception of real words

“…In one, the visual discrepancy was marked (as inface-fame and teeth-rrwuth). In the other, it was "obscure" (as in buzz-bunch and chime-time).l A major fmding of Easton and Basala's (1982) Experiment 1 was that discrepant visual speech information specifying complete words appeared to exert little or no effect on auditory speech identification. Conversely, discrepant acoustic information exerted a strong effect on lipreading.…”

“…Our purpose in the present series of experiments is not to distinguish among the theories (but see Fowler & Dekle, 1991); rather, it is to follow up on a set of findings reported by Easton and Basala (1982) that are unexpected from all three theoretical perspectives. In a comparison of visual influences on reports of heard words and of acoustic influences on reports of mouthed words, Easton and Basala found no visual influence-that is, no McGurk effect-but a strong reverse influence of acoustic signals for words on judgments of mouthed words.…”

“…Our speculation that Easton and Basala (1982) may have underestimated an influence of visual information on judgments of heard words derives from our examination of their stimulus materials. They did not provide a list of all 30 pairs of words in their tests, but their Table 1 provides 9 of the 15 monosyllabic word pairs.…”

“…

Three experiments follow up on Easton and Basala's (1982) report that the "McGurk effect" (an influence of a visibly mouthed utterance on a dubbed acoustic one) does not occur when utterances are real words rather than nonsense syllables. In contrast, with real·word stimuli, Easton and Basala report a strong reverse effect whereby a dubbed soundtrack strongly affects identification of lipread words.

…”

“…Our examination did suggest a reason why Easton and Basala might have underestimated the role of visual information in spoken-word perception, and our experiments attempt to test our hypothesis. In Easton and Basala's (1982) Experiment 1, standard lipreading (SLR) and dubbed lipreading (DLR) tests were performed with two groups of subjects. In the SLR, subjects identified mouthed words with the soundtrack turned off.…”