Auditory motion affects visual biological motion processing

Abstract

The processing of biological motion is a critical, everyday task performed with remarkable efficiency by human sensory systems. Interest in this ability has focused to a large extent on biological motion processing in the visual modality (see, for example, Cutting, J. E., Moore, C., & Morrison, R. (1988). Masking the motions of human gait. Perception and Psychophysics, 44(4),[339][340][341][342][343][344][345][346][347]. In naturalistic settings, however, it is often the case that biological motion is defined by input to more than one sensory modality. For this reason, here in a series of experiments we investigate behavioural correlates of multisensory, in particular audiovisual, integration in the processing of biological motion cues. More specifically, using a new psychophysical paradigm we investigate the effect of suprathreshold auditory motion on perceptions of visually defined biological motion. Unlike data from previous studies investigating audiovisual integration in linear motion processing [Meyer, G. Research, 19,[185][186][187][188][189][190][191][192][193][194], we report the existence of direction-selective effects: relative to control (stationary) auditory conditions, auditory motion in the same direction as the visually defined biological motion target increased its detectability, whereas auditory motion in the opposite direction had the inverse effect. Our data suggest these effects do not arise through general shifts in visuo-spatial attention, but instead are a consequence of motion-sensitive, direction-tuned integration mechanisms that are, if not unique to biological visual motion, at least not common to all types of visual motion. Based on these data and evidence from neurophysiological and neuroimaging studies we discuss the neural mechanisms likely to underlie this effect.

Gender bending: auditory cues affect visual judgements of gender in biological motion displays

“…Previous investigations have demonstrated audio-visual facilitation in lower-order tasks; detection of biological motion (Burr et al 2007), and also in discriminating the direction in which a point-light Wgure moves (Brooks et al 2007;Meyer and Wuerger 2007). The Wndings we report here are some evidence that the same types of interactions now seem to aVect also representational qualities of Wgures in motion.…”

“…With that in mind, it is perhaps surprising that only recently the multisensory interactions have become a focus within the biological motion literature. For example, we recently reported that auditory information predictably inXuences vision-based biological motion sensitivity (Brooks et al 2007). More speciWcally, auditory motion cues seem to aVect observers' abilities to detect PLWs in noisy visual environments.…”

Multidimensional processing of dynamic sounds: more than meets the ear

“…Recent studies of audiovisual perception of biological motion similarly have proposed that naturally relevant (and/or familiar) dynamics facilitate perceptual integration and detection (Brooks et al, 2007;Saygin et al, 2008;Schouten et al, 2011). These studies have shown that adding auditory information that is consistent (congruent) with a possible interpretation of biological motion within patterns of moving lights increases the likelihood that a person will detect biological motion.…”

“…Strong cross-modal interaction exists between the auditory and visual modalities (Alais et al, 2010;Brooks, van der Zwan, Billard, Petreska, Clarke, & Blanke, 2007;SotoFaraco, & Kingstone, 2004;Wuerger, Hofbauer, & Meyer, 2003). For example, when a sound is played in the dark, perception of the origin of the sound can be biased toward the location of a concurrent flash-the spatial ventriloquist illusion (Alais & Burr, 2004b;Bertelson & Aschersleben, 1998;Bertelson & Radeau, 1976).…”

“…For instance, when a single flash is presented with multiple beeps, participants falsely perceive that there is more than one flash (Shams, Kamitani, & Shimojo, 2000). Other recent work has shown that the availability of auditory cues impacts the detectability and discriminability of biological motion in point light displays (Brooks et al, 2007;Saygin, Driver, & de Sa, 2008;Schouten, Troje, Vroomen, & Verfaillie, 2011).…”