Simple and complex crossmodal correspondences involving audition

Abstract

The last few years have seen an explosion of interest from researchers in the crossmodal correspondences, defined as the surprising connections that the majority of people share between seemingly-unrelated stimuli presented in different sensory modalities. Intriguingly, many of the crossmodal correspondences that have been documented/studied to date have involved audition as one of the corresponding modalities. In fact, auditory pitch may well be the single most commonly studied dimension in correspondences research thus far. That said, relatively separate literatures have focused on the crossmodal correspondences involving simple versus more complex auditory stimuli. In this review, I summarize the evidence in this area and consider the relative explanatory power of the various different accounts (statistical, structural, semantic, and emotional) that have been put forward to explain the correspondences. The suggestion is made that the relative contributions of the different accounts likely differs in the case of correspondences involving simple versus more complex stimuli (i.e., pure tones vs. short musical excerpts). Furthermore, the consequences of presenting corresponding versus non-corresponding stimuli likely also differ in the two cases. In particular, while crossmodal correspondences may facilitate binding (i.e., multisensory integration) in the case of simple stimuli, the combination of more complex stimuli (such as, for example, musical excerpts and paintings) may instead be processed more fluently when the component stimuli correspond. Finally, attention is drawn to the fact that the existence of a crossmodal correspondence does not in-and-ofitself necessarily imply that a crossmodal influence of one modality on the perception of stimuli in the other will also be observed.

Bodies moving with sound: Effects of pitch and musical sounds on body-representations

“…We attribute this feeling to the perception of being "pushed up" by the ascending sound, which is compatible with the facilitation of the upwards movement. We also found an effect on emotional state, in line with the affective correspondence between raising pitch and happiness [20,21].…”

“…Nevertheless, as musical sounds with richer spectrum are also more pleasant to listen to than pure tones, as said above, the interaction effect may derive from the fact that crossmodal correspondences between pitch and upward/downward space are emotionally mediated, as previously suggested ( [20]; see also [21,22] for reviews). Our current data does not allow us to make any conclusive remarks on the origin of the interaction effects between the effect of pitch and the musicality of the sound, and this should be explored in future studies.…”

“…Because harmonics are shown to be overall more pleasant to listen to, we expected an increase in the bodily feelings, in the positive direction, but not in the other effects. However, if crossmodal correspondences between pitch and upward/downward space are emotionally mediated ( [20]; see also [21,22] for reviews) then we could also see a more general increase in all three effects. In addition, because the dynamic changes in loudness often present in music interact with the perception of pitch [23][24][25][26], and can also elicit impressions of changes in spatial distance [27,28] a richer musical timbre may modulate (either maximize or diminish) the effects observed for pure tones.…”

“…In Experiment 2, we found an interaction effect between the harmonic content (i.e., timbre) and the overall effect of pitch in several bodily feelings. Since the aesthetic or affective aspects of sounds may change people's overall emotional states and/or change the multisensory integration (e.g., [104]), we cannot exclude that this interaction may derive from the fact that crossmodal correspondences between pitch and upward/downward space are emotionally mediated [20][21][22]. However, since Experiment 2 did not include a measure of emotional state (beyond motivation), our current data does not allow us to make any conclusive remarks in this respect.…”

The Effects of Ethnically Congruent Music on Eye Movements and Food Choice—A Cross-Cultural Comparison between Danish and Chinese Consumers

“…In the context of food, researchers and practitioners have for decades been using this idea to study and create optimal dining and retail atmospherics that facilitate consumption and sales [ 2 , 3 ]. Auditory contributions to these sensory nudges are becoming an increasingly popular topic of research [ 4 , 5 ]. In particular, literature on ambient, atmospheric, or in-store music has been robustly shown to influence a myriad of consumer behavior and perception [ 6 , 7 , 8 , 9 ].…”