When the Eye Listens: A Meta-analysis of How Audio-visual Presentation Enhances the Appreciation of Music Performance

Platz, Friedrich; Kopiez, Reinhard

doi:10.1525/mp.2012.30.1.71

Cited by 123 publications

(117 citation statements)

References 34 publications

Supporting

Mentioning

101

Contrasting

Unclassified

Order By: Relevance

“…On the basis that music is a multisensory phenomenon [34], visual kinematic information from an event (e.g. performer actions) has been documented to be a crucial factor in the communication of meaning [28] and in the emotional reactions evoked [34] within the audience. Nonetheless, electronic music performances are often sensor or laptop-based, which are not always visible to the public and whose usage does not require big gestures and actions from the performer.…”

Section: Audience Researchmentioning

confidence: 99%

The Role of Live Visuals in Audience Understanding of Electronic Music Performances

Correia

Castro

Tanaka

2017

Proceedings of the 12th International Audio Mostly Conference on Augmented and Participatory Sound and Music Experiences

View full text Add to dashboard Cite

There is an identified lack of visual feedback in electronic music performances. Live visuals have been used to fill in this gap. However, there is a scarcity of studies that analyze the effectiveness of live visuals in conveying feedback. In this paper, we aim to study the contribution of live visuals to the understanding of electronic music performances, from the perspective of the audience. We present related work in the fields of audience studies in performing arts, electronic music and audiovisuals. For this purpose, we organized two live events, where 10 audiovisual performances took place. We used questionnaires to conduct an audience study in these events. Results point to a better audience understanding in two of the four design patterns we used as analytical framework. In our discussion, we suggest best practices for the design of audiovisual performance systems that can lead to improved audience understanding.

show abstract

Section: Audience Researchmentioning

confidence: 99%

The Role of Live Visuals in Audience Understanding of Electronic Music Performances

Correia

Castro

Tanaka

2017

Proceedings of the 12th International Audio Mostly Conference on Augmented and Participatory Sound and Music Experiences

View full text Add to dashboard Cite

show abstract

“…Various studies from this first line of research have demonstrated the fundamental principle that visual information accompanying the act of listening to music can influence what we hear and, at times, heighten our enjoyment of it (Behne & Wollner, 2011;for reviews, see Platz &Kopiez, 2012 andSchutz, 2008). Schutz and Lipscomb (2007), for example, report an illusion in which the duration of short-length tones played by a marimba player was perceived by participants as longer when accompanied by a video of the performer making a longer, larger performance gesture.…”

Section: Empirical Research On Audiovisual Music Perceptionmentioning

confidence: 99%

Gesture-sound causality from the audience’s perspective: Investigating the aesthetic experience of performances with digital musical instruments.

Emerson¹,

Egermann²

2018

Psychology of Aesthetics, Creativity, and the Arts

View full text Add to dashboard Cite

In contrast to their traditional, acoustic counterparts, digital musical instruments (DMIs) rarely feature a clear, causal relationship between the performer's actions and the sounds produced. They often function simply as systems for controlling digital sound synthesis, triggering computer-generated audio. This study aims to shed light on how the level of perceived causality of DMI designs impacts audience members' aesthetic responses to new DMIs. In a preliminary survey, 49 concert attendees listed adjectives that described their experience of a number of DMI performances. In a subsequent experiment, 31 participants rated video clips of performances with DMIs with causal and acausal mapping designs using the 8 most popular adjectives from the preliminary survey. The experimental stimuli were presented in their original version and in a manipulated version with a reduced level of gesture-sound causality. The manipulated version was created by placing the audio track of one section of the recording over the video track of a different section. It was predicted that the causal DMIs would be rated more positively, with the manipulation having a stronger effect on the ratings for the causal DMIs. Our results confirmed these hypotheses, and indicate that a lack of perceptible causality does have a negative impact on ratings of DMI performances. The acausal group received no significant difference in ratings between original and manipulated clips. We posit that this result arises from the greater understanding that clearer gesture-sound causality offers spectators. The implications of this result for DMI design and practice are discussed.

show abstract

“…Average ratings of congruence were found to be significantly higher for the matched stimuli than the mismatched stimuli. Platz and Kopiez [25] conducted a meta-analysis of studies into the influence of the visual modality on evaluations of music performance. Aggregating the results from 15 previous studies revealed that the influence of the visual component has a medium effect size (quantified using Cohen's d).…”

Section: Introductionmentioning

confidence: 99%

Influence of Visual Stimuli on Perceptual Attributes of Spatial Audio

Woodcock¹,

Davies²,

Cox³

2019

J. Audio Eng. Soc.

View full text Add to dashboard Cite

Reproduced audio is often accompanied with visuals (i.e., television, virtual reality, gaming, and cinema). However, the audio technology for these systems is often researched and evaluated in isolation from the visual component. Previous research indicates that the auditory and visual modalities are not processed separately. For example, visual stimuli can influence ratings of audio quality and vice versa. This paper presents an experiment to investigate the influence of visual stimuli on a set of attributes relevant to the perception of spatial audio. Eighteen participants took part in a paired comparison listening test where they were asked to judge pairs of stimuli rendered to 14-, 5-, and 2-channel systems using 10 perceptual attributes. The stimuli were presented in audio only and audiovisual conditions. The results show a significant and large main effect of the loudspeaker configuration for all the tested attributes other than overall spectral balance and depth of field. The effect of visual stimuli was found to be small and significant for the attributes realism, sense of space, and spatial clarity. These results suggest that evaluations of audiovisual technologies aiming to evoke a sense of realism or presence should consider the influence of both the audio and visual modalities.

show abstract

When the Eye Listens: A Meta-analysis of How Audio-visual Presentation Enhances the Appreciation of Music Performance

Cited by 123 publications

References 34 publications

The Role of Live Visuals in Audience Understanding of Electronic Music Performances

The Role of Live Visuals in Audience Understanding of Electronic Music Performances

Gesture-sound causality from the audience’s perspective: Investigating the aesthetic experience of performances with digital musical instruments.

Influence of Visual Stimuli on Perceptual Attributes of Spatial Audio

Contact Info

Product

Resources

About