The role of visual spatial attention in audiovisual speech perception

Andersen, T; Tiippana, Kaisa; Laarni, Jari; Kojo, Ilpo; Sams, Mikko

doi:10.1016/j.specom.2008.07.004

Cited by 51 publications

(56 citation statements)

References 48 publications

(53 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Instead, there was a qualitative difference in audiovisual speech perception. When an acoustic /p/ was presented with a visual /k/, the control participants heard this McGurk stimulus predominantly in accordance with the visual component (as in our previous studies using the same stimuli; Andersen et al 2009;Tiippana et al 2010).…”

Section: Discussionsupporting

confidence: 60%

“…We have used these stimuli in previous studies, and they elicit a McGurk effect of hearing the stimulus in accordance with the visual component in typically developed populations (Andersen et al 2009;Tiippana et al 2010). Therefore, we expected the control participants to hear 'k' for the former stimulus (ApVk), and 't' for the latter stimulus (ApVt).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Audiovisual Speech Perception and Eye Gaze Behavior of Adults with Asperger Syndrome

Saalasti

Kätsyri

Tiippana

et al. 2011

J Autism Dev Disord

Self Cite

View full text Add to dashboard Cite

Audiovisual speech perception was studied in adults with Asperger syndrome (AS), by utilizing the McGurk effect, in which conflicting visual articulation alters the perception of heard speech. The AS group perceived the audiovisual stimuli differently from age, sex and IQ matched controls. When a voice saying /p/ was presented with a face articulating /k/, the controls predominantly heard /k/. Instead, the AS group heard /k/ and /t/ with almost equal frequency, but with large differences between individuals. There were no differences in gaze direction or unisensory perception between the AS and control participants that could have contributed to the audiovisual differences. We suggest an explanation in terms of weak support from the motor system for audiovisual speech perception in AS.

show abstract

Section: Discussionsupporting

confidence: 60%

Section: Introductionmentioning

confidence: 99%

Audiovisual Speech Perception and Eye Gaze Behavior of Adults with Asperger Syndrome

Saalasti

Kätsyri

Tiippana

et al. 2011

J Autism Dev Disord

Self Cite

View full text Add to dashboard Cite

show abstract

“…In a separate pilot study (N = 14) we obtained similar results when we asked typical readers to report what they heard (instead of perceived). The rate of visual-based responses also exceeded the rate of fusion responses possibly because the visually presented /aka/ was clearly recognizable and thus did not support the /ata/ interpretation (see Tiippana, 2014, for a similar argument and, e.g., Andersen, Tiippana, Laarni, Kojo, & Sams, 2009;Saalasti et al, 2012, for similar patterns).…”

Section: Discussionmentioning

confidence: 99%

A General Audiovisual Temporal Processing Deficit in Adult Readers With Dyslexia

Francisco

Jesse

Groen

et al. 2017

J Speech Lang Hear Res

View full text Add to dashboard Cite

Purpose Because reading is an audiovisual process, reading impairment may reflect an audiovisual processing deficit. The aim of the present study was to test the existence and scope of such a deficit in adult readers with dyslexia. Method We tested 39 typical readers and 51 adult readers with dyslexia on their sensitivity to the simultaneity of audiovisual speech and nonspeech stimuli, their time window of audiovisual integration for speech (using incongruent /aCa/ syllables), and their audiovisual perception of phonetic categories. Results Adult readers with dyslexia showed less sensitivity to audiovisual simultaneity than typical readers for both speech and nonspeech events. We found no differences between readers with dyslexia and typical readers in the temporal window of integration for audiovisual speech or in the audiovisual perception of phonetic categories. Conclusions The results suggest an audiovisual temporal deficit in dyslexia that is not specific to speech-related events. But the differences found for audiovisual temporal sensitivity did not translate into a deficit in audiovisual speech perception. Hence, there seems to be a hiatus between simultaneity judgment and perception, suggesting a multisensory system that uses different mechanisms across tasks. Alternatively, it is possible that the audiovisual deficit in dyslexia is only observable when explicit judgments about audiovisual simultaneity are required.

show abstract

“…It is well known that 4-mo-old infants are already able to correctly match a sound with a face Meltzoff, 1982, 1984). When subjects are presented with an auditory source dubbed on a screen containing two faces, visual spatial attention is also able to choose between the faces when lipreading (Andersen et al, 2009). Selective attention to the appropriate face has been correlated with steady-state visual evoked potentials on the visual scalp (Senkowski et al, 2008a).…”

Section: Discussionmentioning

confidence: 99%

Binding and unbinding the auditory and visual streams in the McGurk effect

Nahorna¹,

Berthommier²,

Schwartz³

2012

The Journal of the Acoustical Society of America

View full text Add to dashboard Cite

Subjects presented with coherent auditory and visual streams generally fuse them into a single percept. This results in enhanced intelligibility in noise, or in visual modification of the auditory percept in the McGurk effect. It is classically considered that processing is done independently in the auditory and visual systems before interaction occurs at a certain representational stage, resulting in an integrated percept. However, some behavioral and neurophysiological data suggest the existence of a two-stage process. A first stage would involve binding together the appropriate pieces of audio and video information before fusion per se in a second stage. Then it should be possible to design experiments leading to unbinding. It is shown here that if a given McGurk stimulus is preceded by an incoherent audiovisual context, the amount of McGurk effect is largely reduced. Various kinds of incoherent contexts (acoustic syllables dubbed on video sentences or phonetic or temporal modifications of the acoustic content of a regular sequence of audiovisual syllables) can significantly reduce the McGurk effect even when they are short (less than 4 s). The data are interpreted in the framework of a two-stage "binding and fusion" model for audiovisual speech perception.

show abstract

The role of visual spatial attention in audiovisual speech perception

Cited by 51 publications

References 48 publications

Audiovisual Speech Perception and Eye Gaze Behavior of Adults with Asperger Syndrome

Audiovisual Speech Perception and Eye Gaze Behavior of Adults with Asperger Syndrome

A General Audiovisual Temporal Processing Deficit in Adult Readers With Dyslexia

Binding and unbinding the auditory and visual streams in the McGurk effect

Contact Info

Product

Resources

About