Music can elicit a visual motion aftereffect

Abstract

Motion aftereffects (MAEs) are thought to result from the adaptation of both subcortical and cortical systems involved in the processing of visual motion. Recently, it has been reported that the implied motion of static images in combination with linguistic descriptions of motion is sufficient to elicit an MAE, although neither factor alone is thought to directly activate visual motion areas in the brain. Given that the monotonic change of musical pitch is widely recognized in music as a metaphor for vertical motion, we investigated whether prolonged exposure to ascending or descending musical scales can also produce a visual motion aftereffect. After listening to ascending or descending musical scales, participants made decisions about the direction of visual motion in random-dot kinematogram stimuli. Metaphoric motion in the musical stimuli did affect the visual direction judgments, in that repeated exposure to rising or falling musical scales shifted participants' sensitivity to visual motion in the opposite direction. The finding that music can induce an MAE suggests that the subjective interpretation of monotonic pitch change as motion may have a perceptual foundation.

Auditory Motion Elicits a Visual Motion Aftereffect

“…Kitagawa and Ichihara (2002) concluded that this finding supports the notion of visual dominance over auditory perception for spatial events. This finding is in contrast, however, with those of Maeda et al (2004) who found that auditory stimuli consisting of sounds increasing or decreasing in pitch was sufficient to induce cross-modal changes in visual motion perception in the vertical plane (Maeda et al, 2004), as well as findings by Hedger et al (2013) who found that music containing ascending or descending scales can induce visual MAEs in the vertical plane (Hedger et al, 2013). However, these studies rely on the metaphorical relationship between a sound and an associated visual representation rather than on physical motion cues of the sounds themselves.…”

“…However, these studies rely on the metaphorical relationship between a sound and an associated visual representation rather than on physical motion cues of the sounds themselves. In this way, the findings of Maeda et al (2004) and Hedger et al (2013) are much like the positive aftereffects (i.e., changes in visual motion perception in the same direction as the adapting auditory stimulus) found in the sound contingent visual motion aftereffect (Hidaka et al, 2011a), and the negative aftereffects (i.e., changes in visual motion perception in the opposite direction as the adapting auditory stimulus) found for verbal language descriptions of visual events (Dils and Boroditsky, 2010), respectively. Furthermore, the looming/receding auditory motion stimuli used in Kitagawa and Ichihara's (2002) study were simply tones increasing or decreasing in volume and provided weak motion cues compared to those from veridical horizontal auditory motion (Carlile and Leung, 2016).…”

“…Previous studies investigating whether auditory stimuli can elicit visual MAEs have either made use of auditory stimuli containing weak spatial cues (i.e., increasing or decreasing loudness; Kitagawa and Ichihara, 2002) or have relied on sounds that elicit strong visual motion metaphors (Maeda et al, 2004; Hedger et al, 2013) or were strongly associated with visual-motion stimuli (Dils and Boroditsky, 2010; Hidaka et al, 2011a). Thus, the results from previous studies have led to somewhat mixed conclusions about whether a cross-modal MAE from auditory stimuli is possible.…”

“…Each auditory motion adaption block was followed by the converse auditory motion adaption block (i.e., a rightward sound adaption block always came after a leftward sound adaptation block and vice versa), and the order was counterbalanced across participants. In each block, the first auditory motion stimulus was presented for 60 s, with each subsequent auditory motion stimulus within a block presented for 10 s. This “top-up” method of adaption is commonly employed in MAE studies (Dils and Boroditsky, 2010; Winawer et al, 2010; Hedger et al, 2013) and was used here to ensure that any established adaptation would not diminish before the end of the block. Stimulus presentation and data acquisition were controlled using PsychoPy (Peirce, 2007) software (version 1.82.01) on a 21.5-inch (54.61 cm) iMac computer, and the experiments took place in a soundproof testing room (40-decibel noise reduction).…”

Models and processes of multisensory cue combination

“…While the Poisson-like PPC framework does account for several important aspects of multisensory cue combination, some of its underlying assumptions may not always be valid. For example, neural weights appear to depend on stimulus reliability in MSTd [4**, 24], and although the theory assumes unisensory representations are independent, sensory interactions may begin before multisensory integration occurs [58*, 59]. Furthermore, computations like causal inference cannot be performed explicitly using Poisson-like PPCs [60].…”