Reducing Playback Rate of Audiovisual Speech Leads to a Surprising Decrease in the McGurk Effect

Magnotti, John F.; Mallick, Debshila Basu; Beauchamp, Michael S.

doi:10.1163/22134808-00002586

Cited by 6 publications

(3 citation statements)

References 32 publications

(44 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many factors, including co-articulation (number of repetitions of the syllable, i.e., ba vs. baba ) and speech rate (fast vs. slow) may contribute to the efficacy of a given McGurk exemplar (Magnotti et al, 2018a, b). In the absence of techniques for psychometrically manipulating McGurk stimuli as there are for simpler sensory stimuli (such as parametrically changing the volume of a tone to measure auditory sensitivity) participants in the present study viewed a battery of nine different ba/ga and pa/ka McGurk exemplars, with the average score across stimuli measuring the overall susceptibility of a given participant to the illusion.…”

Section: Discussionmentioning

confidence: 99%

A Laboratory Study of the McGurk Effect in 324 Monozygotic and Dizygotic Twins

Feng

Zhou

et al. 2019

Front. Neurosci.

Self Cite

View full text Add to dashboard Cite

Multisensory integration of information from the talker’s voice and the talker’s mouth facilitates human speech perception. A popular assay of audiovisual integration is the McGurk effect, an illusion in which incongruent visual speech information categorically changes the percept of auditory speech. There is substantial interindividual variability in susceptibility to the McGurk effect. To better understand possible sources of this variability, we examined the McGurk effect in 324 native Mandarin speakers, consisting of 73 monozygotic (MZ) and 89 dizygotic (DZ) twin pairs. When tested with 9 different McGurk stimuli, some participants never perceived the illusion and others always perceived it. Within participants, perception was similar across time (r = 0.55 at a 2-year retest in 150 participants) suggesting that McGurk susceptibility reflects a stable trait rather than short-term perceptual fluctuations. To examine the effects of shared genetics and prenatal environment, we compared McGurk susceptibility between MZ and DZ twins. Both twin types had significantly greater correlation than unrelated pairs (r = 0.28 for MZ twins and r = 0.21 for DZ twins) suggesting that the genes and environmental factors shared by twins contribute to individual differences in multisensory speech perception. Conversely, the existence of substantial differences within twin pairs (even MZ co-twins) and the overall low percentage of explained variance (5.5%) argues against a deterministic view of individual differences in multisensory integration.

show abstract

Section: Discussionmentioning

confidence: 99%

A Laboratory Study of the McGurk Effect in 324 Monozygotic and Dizygotic Twins

Feng

Zhou

et al. 2019

Front. Neurosci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…One concern about online data collection that is particularly relevant to audiovisual speech experiments is that poor auditory or visual quality may cloud effects that would be observable in a laboratory setting (but see [ 14 , 101 ] for other examples of online studies on the McGurk effect). To address this issue, we ensured that participants had sufficiently good auditory equipment by employing a recently introduced headphone screening [ 69 ].…”

Section: Discussionmentioning

confidence: 99%

What accounts for individual differences in susceptibility to the McGurk effect?

et al. 2018

View full text Add to dashboard Cite

The McGurk effect is a classic audiovisual speech illusion in which discrepant auditory and visual syllables can lead to a fused percept (e.g., an auditory /bɑ/ paired with a visual /gɑ/ often leads to the perception of /dɑ/). The McGurk effect is robust and easily replicated in pooled group data, but there is tremendous variability in the extent to which individual participants are susceptible to it. In some studies, the rate at which individuals report fusion responses ranges from 0% to 100%. Despite its widespread use in the audiovisual speech perception literature, the roots of the wide variability in McGurk susceptibility are largely unknown. This study evaluated whether several perceptual and cognitive traits are related to McGurk susceptibility through correlational analyses and mixed effects modeling. We found that an individual’s susceptibility to the McGurk effect was related to their ability to extract place of articulation information from the visual signal (i.e., a more fine-grained analysis of lipreading ability), but not to scores on tasks measuring attentional control, processing speed, working memory capacity, or auditory perceptual gradiency. These results provide support for the claim that a small amount of the variability in susceptibility to the McGurk effect is attributable to lipreading skill. In contrast, cognitive and perceptual abilities that are commonly used predictors in individual differences studies do not appear to underlie susceptibility to the McGurk effect.

show abstract

“…The causal inference of multisensory speech (CIMS) model incorporates these processes into a principled framework that predicts perception of arbitrary combinations of auditory and visual speech (Magnotti and Beauchamp, 2017). The CIMS model has been used to explain a number of puzzling audiovisual speech phenomena, such as the increase in the McGurk effect observed with co-articulation (Magnotti et al, 2018b); the decrease in the McGurk effect observed with slower playback rates (Magnotti et al, 2018a); and why the McGurk effects is produced by some incongruent syllables but not others (Magnotti and Beauchamp, 2017 (Alsius et al, 2018;Rosenblum, 2019).…”

Section: Introductionmentioning

confidence: 99%

Causal inference explains the stimulus-level relationship between the McGurk Effect and auditory speech perception

Wegner-Clemens

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Acknowledgments: This research was supported by NIH R01NS065395. Abstract The McGurk effect is widely used as a measure of multisensory integration during speech perception. Two observations have raised questions about the relationship between the effect and everyday speech perception. First, there is high variability in the strength of the McGurk effect across different stimuli and observers. Second, there is low correlation across observers between perception of the McGurk effect and measures of everyday speech perception, such as the ability to understand noisy audiovisual speech. Using the framework of the causal inference of multisensory speech (CIMS) model, we explored the relationship between the McGurk effect, syllable perception, and sentence perception in seven experiments with a total of 296 different participants. Perceptual reports revealed a relationship between the efficacy of different McGurk stimuli created from the same talker and perception of the auditory component of the McGurk stimuli presented in isolation, either with or without added noise. The CIMS model explained this high stimulus-level correlation using the principles of noisy sensory encoding followed by optimal cue combination within a representational space that was identical for McGurk and everyday speech. In other experiments, CIMS successfully modeled low observer-level correlation between McGurk and everyday speech. Variability in noisy speech perception was modeled using individual differences in noisy sensory encoding, while variability in McGurk perception involved additional differences in causal inference. Participants with all combinations of high and low sensory encoding noise and high and low causal inference disparity thresholds were identified. Perception of the McGurk effect and everyday speech can be explained by a common theoretical framework that includes causal inference.

show abstract

Reducing Playback Rate of Audiovisual Speech Leads to a Surprising Decrease in the McGurk Effect

Cited by 6 publications

References 32 publications

A Laboratory Study of the McGurk Effect in 324 Monozygotic and Dizygotic Twins

A Laboratory Study of the McGurk Effect in 324 Monozygotic and Dizygotic Twins

What accounts for individual differences in susceptibility to the McGurk effect?

Causal inference explains the stimulus-level relationship between the McGurk Effect and auditory speech perception

Contact Info

Product

Resources

About