Auditory and visual speech perception: Confirmation of a modality-independent source of individual differences in speech recognition

Abstract

Two experiments were run to determine whether individual differences in auditory speech-recognition abilities are significantly correlated with those for speech reading (lipreading), employing a total sample of 90 normal-hearing college students. Tests include single words and sentences, recorded on a videodisc by a male speaker [Bernstein and Eberhardt, Johns Hopkins Lipreading Corpus, The Johns Hopkins University, Baltimore, MD, 1986]. The auditory speech was presented with a white noise masker, at -7 dB Sp/N. The correlations between overall auditory and visual performance were 0.52 and 0.43 in the two experiments, consistent with the existence of a modality-independent ability to perceive linguistic "wholes" on the basis of linguistic fragments. Subjects in the second experiment also identified printed sentences, with 40%-60% portions of the printed characters deleted. Performance on this graphical "fragmented-sentences test" also correlated significantly with auditory speech recognition, providing a possible clue to the cognitive basis for the look-versus-listen correlation. The existence of a modality-independent source of variance in speech-recognition abilities may be a partial explanation of the difficulty in demonstrating strong associations between psychoacoustic measures of spectral or temporal acuity, and speech discrimination or identification. Female subjects in both experiments were significantly better lipreaders than their male counterparts.

Effects of speechreading and signal-to-noise ratio on understanding mainstream American English by American and Indian adults.

“…Johnson et al (1988) also reported that females performed better on identification of phonemes and sentences than males; however, males did better in identifying visemes (e.g., /b, m, p/). Results of studies by Dancer et al in 1994 andWatson, Qiu, Chamberlain, andLi in 1996 were consistent with previous studies indicating females performed better on identifying sentences than males. A recent study by Tye-Murray et al (2007b) reported, however, that differences in speechreading performances between females and males in identifying consonants, words, and sentences were not significant.…”

A sex difference in visual influence on heard speech

“…Previous research has shown that women are better at identifying lipread speech than are men for sentences (e.g., Johnson, Hicks, Goldberg, & Myslobodsky, 1988;Watson, Qiu, Chamberlain, & Li, 1996). However, shorter speech stimuli, such as the syllables used in the present study, have not been examined previously.…”

Analysis and analogy in the perception of vowels

“…Independently of pitch matching studies, though, the premise that vowel contrasts are equivalent to simple contrasts of stationary pitch is not well supported (see Nearey & Assmann, 1986), nor have campaigns to describe multimodal speech perception settled on an auditory pitch space as the common metric (see Summerfield, 1987;Watson, Qiu, Chamberlain, & Li, 1996). In this light, the present results offer a way to explain the ability of subjects to form pitch analogies to vowels without invoking a perceptual pitch representation as an intermediate form.…”