Perceived Congruency in Audiovisual Stimuli Consisting of Gabor Patches and AM and FM Tones

Abstract

Experiments that focus on how humans perceive temporal, spatial or synaesthetic congruency in audiovisual sensory information have often employed stimuli consisting of a Gabor patch and an amplitude (AM) or frequency (FM)-modulated sound. Introducing similarity between the static and dynamic features of the Gabor patch and the (carrier) frequency or modulation frequency of the sound is often assumed to be effective enough to induce congruency. However, comparative empirical data on perceived congruency of various stimulus parameters are not readily available, and in particular with respect to sound modulation, it is still not clear which type (AM or FM) induces perceived congruency best in tandem with various patch parameters. In two experiments, we examined Gabor patches of various spatial frequencies with flickering (2, 3 and 4 flickers/s) or drifting (0.5, 1.0 and 1.5 degrees/s) gratings in combinations with AM or FM tones of 2-, 3- and 4-Hz modulation and 500-, 1000- and 2000-Hz carrier frequencies. Perceived congruency ratings were obtained by asking participants to rate stimulus (in)congruency from 1 (incongruent) to 7 (congruent). The data showed that varying the spatial frequency of the Gabor patch and the carrier frequency of the modulated tone had comparatively little impact on perceived congruency. Similar to previous findings, similarity between the temporal frequency of the Gabor patch and the modulated tone effectively promoted perceived congruency. Furthermore, direct comparisons convincingly showed that AM tones in combination with flickering Gabor patches received significantly higher audiovisual congruency ratings compared to FM tones.