Cross-modal associations between paintings and sounds: Effects of embodiment

Iosifyan, Marina; Sidoroff-Dorso, Anton V.; Wolfe, Judith

doi:10.1177/03010066221126452

Cited by 3 publications

(1 citation statement)

References 82 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In other words, artist-intended AV congruence was perceivable but did not affect subjective ratings. These results are in line with previous studies showing perceivable correspondences between art and music 13,25 , and contribute to the debate around whether such correspondences affect aesthetic experience 11,[17][18][19] .…”

Section: Discussionsupporting

confidence: 91%

The role of audiovisual congruence in perception and aesthetic appreciation of contemporary music and visual art

Fink,

Fiehn,

Wald-Fuhrmann

2023

Preprint

View full text Add to dashboard Cite

While low-level, cross-modal correspondences between vision and hearing are well-documented (e.g., Kiki-Bouba effect), it is unclear whether cross-modal correspondences are perceivable between complex, multi-dimensional stimuli, like contemporary music and art. Further, previous studies show conflicting results regarding whether audiovisual correspondence affects subjective aesthetic experience. Here, in collaboration with the Kentler International Drawing Space (NYC, USA), we use material from the Music as Image and Metaphor exhibition, consisting of music composed for each work of visual art. Our pre-registered online experiment consisted of 4 conditions: Audio, Visual, Audio-Visual-Intended (artist-intended pairing of art/music), and Audio-Visual-Random (random shuffling). Participants (N=201) were presented with 16 pieces and could click to proceed to the next piece whenever they liked. After each piece, they were asked about their subjective experience. Analyzing results by condition, we found that participants spent significantly more time with Audio, followed by Audiovisual, followed by Visual pieces; however, they felt most moved in the Audiovisual (bi-modal) conditions. The Audiovisual-Intended pieces were perceived to have greater correspondence than those in the Audiovisual-Random condition. Interestingly, though, there were no significant differences for these two conditions on any other subjective rating scale or for time spent. Collectively, these results extend our understanding of cross-modal correspondence to complex, professional, real-world abstract art and contemporary music, and call into question the use of time spent as an implicit measure of aesthetic appreciation in multi-modal conditions.

show abstract

Section: Discussionsupporting

confidence: 91%

The role of audiovisual congruence in perception and aesthetic appreciation of contemporary music and visual art

Fink,

Fiehn,

Wald-Fuhrmann

2023

Preprint

View full text Add to dashboard Cite

show abstract

Sensory Profiles and Technological Deficits: Classification, Relevance, and Gaps

Hutson,

Harper-Nichols

2024

Sensemaking and Neuroaesthetics

View full text Add to dashboard Cite

Prokofiev was (almost) right: A cross-cultural investigation of auditory-conceptual associations in Peter and the Wolf

Di Stefano,

Ansani,

Schiavio

et al. 2024

Psychon Bull Rev

View full text Add to dashboard Cite

Over recent decades, studies investigating cross-modal correspondences have documented the existence of a wide range of consistent cross-modal associations between simple auditory and visual stimuli or dimensions (e.g., pitch-lightness). Far fewer studies have investigated the association between complex and realistic auditory stimuli and visually presented concepts (e.g., musical excerpts-animals). Surprisingly, however, there is little evidence concerning the extent to which these associations are shared across cultures. To address this gap in the literature, two experiments using a set of stimuli based on Prokofiev’s symphonic fairy tale Peter and the Wolf are reported. In Experiment 1, 293 participants from several countries and with very different language backgrounds rated the association between the musical excerpts, images and words representing the story’s characters (namely, bird, duck, wolf, cat, and grandfather). The results revealed that participants tended to consistently associate the wolf and the bird with the corresponding musical excerpt, while the stimuli of other characters were not consistently matched across participants. Remarkably, neither the participants’ cultural background, nor their musical expertise affected the ratings. In Experiment 2, 104 participants were invited to rate each stimulus on eight emotional features. The results revealed that the emotional profiles associated with the music and with the concept of the wolf and the bird were perceived as more consistent between observers than the emotional profiles associated with the music and the concept of the duck, the cat, and the grandpa. Taken together, these findings therefore suggest that certain auditory-conceptual associations are perceived consistently across cultures and may be mediated by emotional associations.

show abstract

Cross-modal associations between paintings and sounds: Effects of embodiment

Cited by 3 publications

References 82 publications

The role of audiovisual congruence in perception and aesthetic appreciation of contemporary music and visual art

The role of audiovisual congruence in perception and aesthetic appreciation of contemporary music and visual art

Sensory Profiles and Technological Deficits: Classification, Relevance, and Gaps

Prokofiev was (almost) right: A cross-cultural investigation of auditory-conceptual associations in Peter and the Wolf

Contact Info

Product

Resources

About