Audiovisual phenomenal causality

Abstract

We report three experiments in which visual or audiovisual displays depicted a surface (target) set into motion shortly after one or more events occurred. A visual motion was used as an initial event, followed directly either by the target motion or by one of three marker events: a collision sound, a blink of the target stimulus, or the blink together with the sound. The delay between the initial event and the onset of the target motion was varied systematically. The subjects had to rate the degree of perceived causality between these events. The results of the first experiment showed a systematic decline of causality judgments with an increasing time delay. Causality judgments increased when additional auditory or visual information marked the onset of the target motion. Visual blinks of the target and auditory clacks produced similar causality judgments. The second experiment tested several models of audiovisual causal processing by varying the position of the sound within the visual delay period. No systematic effect of the sound position occurred. The third experiment showed a subjective shortening of delays filled by a clack sound, as compared with unfilled delays. However, this shortening cannot fully explain the increased tolerance for delays containing the clack sound. Taken together, the results are consistent with the interpretation that the main source of the causality judgments in our experiments is the impression of a plausible unitary event and that perfect synchrony is not necessary in this case.

Audio-visual simultaneity judgments

“…In particular, discrete auditory and visual signals are more likely to be perceived as referring to a single multisensory event rather than to two separate events when they are presented in spatiotemporal proximity, thus inducing a sensation of cross-modal "phenomenal causality" (Guski & Troje, 2003). It could be argued that the participants in Experiments 1 and 2 may have found it more natural to respond "simultaneous" when the stimuli were presented from the same location than when they were presented from different locations.…”

High-intensity sound increases the size of visually perceived objects

“…We consider an appropriate temporal window of audiovisual integration to be the range within which multisensory events are regarded as a single event. According to Guski and Troje (2003), this range can be estimated as −130 to +250 ms. Thus, the temporal window of the present effect was narrower than that described above for an audiovisual interaction.…”

Effects of grouping and attention on the perception of causality

“…This project also began with Michotte (1946Michotte ( /1963, who-in the course of over 100 experiments in the English translation of his landmark bookworked out the details of how perceptual causality is impacted by the absolute and relative speeds of the objects, the distances and directions in which they travel (both before and after "impact"), various types of spatial and temporal gaps, and many other variables. More recent investigations have continued this project, exploring how the perception of causality is affected by different types of motion (e.g., apparent motion; Gordon, Day, & Stecher, 1990), the use of more than two interacting objects (e.g., the tool effect; Michotte & Thinès, 1963/1991, other types of spatiotemporal gaps (e.g., Schlottmann & Anderson, 1993), other modalities (e.g., Guski & Troje, 2003), and many other factors (e.g., Boyle, 1960;Gemelli & Cappellini, 1958;Hubbard, Blessum, & Ruppel, 2001;Kruschke & Fragassi, 1996;Natsoulas, 1961;Schlottmann, Allen, Linderoth, & Hesket, 2002;Schlottmann & Shanks, 1992;Weir, 1978;White, in press;White & Milne, 1997, 1999Yela, 1952).…”