Seeing speech in space and time: psychological and neurological findings

Campbell, Ruth

doi:10.1109/icslp.1996.607899

Cited by 6 publications

(8 citation statements)

References 18 publications

(16 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This notion is supported by recent evidence (Campbell, 1996;Campbell, de Haan, & Brooks, 1997) showing that reversing the brightness variations of a talking face by presenting the image in photonegative severely disrupts performance with visual speech and fails to generate McGurk responses. In a more recent investigation (Kanzaki & Campbell, 1999), using a different speaker, the effects of brightness reversal were less disruptive, although still evident.…”

Section: Luminance Variations and Their Spatial Distribution Across Tsupporting

confidence: 70%

Visual and audiovisual speech perception with color and gray-scale facial images

Jordan

McCotter

Thomas

2000

Perception & Psychophysics

View full text Add to dashboard Cite

Research has shown that auditory speech recognition is influenced by the appearance of a talker's face, but the actual nature of this visual information has yet to be established, Here, we report three experiments that investigated visual and audiovisual speech recognition using color, gray-scale, and point-light talking faces (which allowed comparison with the influence of isolated kinematic information). Auditory and visual forms of the syllables Ibal, Ibil, Igal, Igil, Ivai, and Ivil were used to produce auditory, visual, congruent, and incongruent audiovisual speech stimuli. Visual speech identification and visual influences on identifyingthe auditory components of congruent and incongruent audiovisual speech were identical for color and gray-scale faces and were much greater than for point-light faces, These results indicate that luminance, rather than color, underlies visual and audiovisual speech perception and that this information is more than the kinematic information provided by point-light faces. Implications for processing visual and audiovisual speech are discussed.

show abstract

Section: Luminance Variations and Their Spatial Distribution Across Tsupporting

confidence: 70%

Visual and audiovisual speech perception with color and gray-scale facial images

Jordan

McCotter

Thomas

2000

Perception & Psychophysics

View full text Add to dashboard Cite

show abstract

“…Speech perception is often multimodal (Campbell, 1996(Campbell, , 2008 visual attention] and the suggestion that multimodal information processing is atypical in FXS). Additionally, auditory memory is worse than visual memory in DS (e.g., Marcell & Armstrong, 1982), which indicates that, in at least some tasks, individuals rely more on one modality than another.…”

Section: Speech Perception In Typically Developing Childrenmentioning

confidence: 99%

Audio-visual speech perception in infants and toddlers with Down syndrome, fragile X syndrome, and Williams syndrome

D’Souza

Johnson

et al. 2016

Infant Behavior and Development

View full text Add to dashboard Cite

“…Dissimilarities between syllables could be computed in terms of differences in velocities and/or accelerations. The extent to which kinematic information alone is sufficient for visual speech perception is an ongoing topic (Campbell, 1996;Rosenblum & Saldaña, 1998). The methods of the present study could be extended to determine the extent to which perceptual dissimilarity is accounted for by kinematic dissimilarity.…”

Section: Limitations Of the Present Studymentioning

confidence: 99%

“…; see Munhall & VatikiotisBateson, 1998). Some effects of global physical stimulus factors have been examined, such as overall spatial resolution (Erber, 1979;Munhall, Kroos, Jozan, & VatikiotisBateson, 2004), viewing angle (Jordan & Thomas, 2001), presence or absence of dynamic information (Campbell, 1996;Rosenblum & Saldaña, 1998), and color and luminance (McCotter & Jordan, 2003).…”

mentioning

confidence: 99%

Similarity structure in visual speech perception and optical phonetic signals

Jiang

Auer

Alwan

et al. 2007

Perception & Psychophysics

View full text Add to dashboard Cite

A complete understanding of visual phonetic perception (lipreading) requires linking perceptual effects to physical stimulus properties. However, the talking face is a highly complex stimulus, affording innumerable possible physical measurements. In the search for isomorphism between stimulus properties and phonetic effects, second-order isomorphism was examined between the perceptual similarities of video-recorded perceptually identified speech syllables and the physical similarities among the stimuli. Four talkers produced the stimulus syllables comprising 23 initial consonants followed by one of three vowels. Six normal-hearing participants identified the syllables in a visual-only condition. Perceptual stimulus dissimilarity was quantified using the Euclidean distances between stimuli in perceptual spaces obtained via multidimensional scaling. Physical stimulus dissimilarity was quantified using face points recorded in three dimensions by an optical motion capture system. The variance accounted for in the relationship between the perceptual and the physical dissimilarities was evaluated using both the raw dissimilarities and the weighted dissimilarities. With weighting and the full set of 3-D optical data, the variance accounted for ranged between 46% and 66% across talkers and between 49% and 64% across vowels. The robust second-order relationship between the sparse 3-D point representation of visible speech and the perceptual effects suggests that the 3-D point representation is a viable basis for controlled studies of first-order relationships between visual phonetic perception and physical stimulus attributes.

show abstract

Seeing speech in space and time: psychological and neurological findings

Cited by 6 publications

References 18 publications

Visual and audiovisual speech perception with color and gray-scale facial images

Visual and audiovisual speech perception with color and gray-scale facial images

Audio-visual speech perception in infants and toddlers with Down syndrome, fragile X syndrome, and Williams syndrome

Similarity structure in visual speech perception and optical phonetic signals

Contact Info

Product

Resources

About