Hemispheric contributions to the integration of visual and auditory information in speech perception

Abstract

Differential hemispheric contributions to the perceptual phenomenon known as the McGurk effect were examined in normal subjects, 1 callosotomy patient, and 4 patients with intractable epilepsy. Twenty-five right-handed subjects were more likely to demonstrate an influence of a mouthed word on identification of a dubbed acoustic word when the speaker's face was lateralized to the LVF as compared with the RVF. In contrast, display of printed response alternatives in the RVF elicited a greater percentage of McGur)t.responses than display in the LVF. Visual field differences were absent in a group of 15 left-handed subjects. These results suggest that in right-handers, the two hemispheres may make distinct contributions to the McGurk effect. The callosotomy patient demonstrated reliable McGurk effects, but at a lower rate than the normal subjects and the epileptic control subjects. These data support the view that both the right and left hemisphere can make significant contributions to the McGurk effect.

Gaze behavior in audiovisual speech perception: The influence of ocular fixations on the McGurk effect

“…Although both left-field (R. Campbell, 1986) and right-field (R. Campbell, de Gelder, & de Haan, 1996) advantages have been observed for static face tasks, a right visual field advantage was reported for silent speechreading (Smeele, Massaro, Cohen, & Sittig, 1998). Two other studies (Baynes, Funnell, & Fowler, 1994;Diesch, 1995) also showed a right visual field advantage for the audiovisual fusion effect provided by incongruent McGurk stimuli such as ours. Most of these studies used small gaze displacements (,5º from central fixation) and did not directly manipulate stimulus eccentricities, with the exception of Smeele and colleagues (Smeele et al, 1998), who reported a reliable decrease in speechreading performance when the subject's gaze was displaced eccentrically (but within 10º) from the talking face.…”

A sex difference in visual influence on heard speech

“…A number of functional neuroimaging studies have shown that females engage the right perisylvian cortex more than do males for phonologically based language tasks (Jaeger et al, 1998;Pugh, Shaywitz, Shaywitz, Constable, et al, 1996;Rossell et al, 2002;Shaywitz et al, 1995). In addition, previous research has indicated that there is a right-hemisphere (RH) processing advantage for audiovisually presented speech (Baynes, Funnell, & Fowler, 1994;Diesch, 1995). Bilaterality in female perceivers' neural organization has been proposed to lead to faster or more efficient processing of incongruent auditory and visual speech (Coney, 2002).…”

The lateralization of lip-reading: A second look

“…In our earlier study [3], tachistoscopic displays of low illumination and low contrast were used; more different mouth shapes all produced by a single speaker--were seen. In Baynes et al's study [1], moving facial stimuli were seen on a video display. All these conditions may foster a reduced RVF advantage.…”

“…Patient D, with dense face-processing problems consequent to a right-sided lesion in an analogous site, showed no impairment in any lip-reading task. By contrast, in normal subjects, it has been reported (a) that matching a still photograph of a lip-shape to an aftercoming speechsound was faster in the LVF/right hemisphere than the RVF/left hemisphere [3], and (b) that very slight asymmetries in favour of the right hemisphere can sometimes be found for audiovisual dubbed speech sounds [1,7]. Furthermore, two commisurotomized patients, NG and JW, do not show a clear left hemisphere advantage in the few speech-reading tasks which have been explored [1,4].…”