Cross-modal relationships between music, emotion, and visual imagery: A comparative study of Iran, Canada, and Japan [Stage 1 Registered Report Snapshot]

Abstract

Many people experience emotions and visual imagery while listening to music. Previous research has identified cross-modal associations between musical features such as pitch, tempo, and visual features including height, and colour saturation/brightness. But while researchers have explored cross-cultural links between music and emotion as well as music and visual imagery, few studies have simultaneously investigated cross-cultural links between music, visual imagery, and emotion in order to distinguish the role of cultural experiences in contrast to innate perceptual capabilities. We hypothesize that cross-cultural emotion and cross-modal associations encompass both universality and diversity determined by innate perceptual capabilities and cultural experiences respectively and represented by: 1) cross-culturally consistent correlations between high level auditory features and low level visual features and dimensional emotions, 2) cross-cultural diversity in categorical emotional appraisal. In this study, we investigate the relationship between emotion and visual imagery induced by 6 musical excerpts in participants in Japan, Iran, and Canada through forced choice options such as matching excerpts with visual textures that vary in density, as well as dimensional and categorical emotion ratings for each excerpt manipulated in pitch and tempo. In our full manuscript, we will provide our power analysis and provide the details of our planned sample size.We will test the following four hypotheses: 1) tempo correlates with arousal levels cross-culturally, 2) tempo correlates with visual density cross-culturally, 3) emotion category ratings tend to vary between cultures and often even within cultures in response to slow tempo musical excerpts, and 4) emotion ratings show more consistency cross-culturally in response to faster tempo pieces.Each of the hypotheses helps us conclude which cross-modal and emotion associations are 1) cross-culturally consistent, or 2) culturally-dependent. If we find cross-cultural consistency across all three groups, it would suggest physiological link/embodied cognition represented in cross-modal associations. If we find consistency in two cultures only or none at all, it would suggest that cross-modal associations are culturally dependent. We also plan to conduct exploratory analyses of data on colour associations using the Berkeley Colour project stimuli.