Analyzing correspondence between sound objects and body motion

Nymoen, Kristian; Godøy, Rolf Inge; Jensenius, Alexander Refsum; Tørresen, Jim

doi:10.1145/2465780.2465783

Cited by 37 publications

(41 citation statements)

References 35 publications

(46 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Second, pitch was correlated with the hand's vertical position [Nymoen et al 2011]. A recent study also showed how analysis methods can complement themselves in such experimental contexts [Nymoen et al 2013].…”

Section: Introductionmentioning

confidence: 99%

The Role of Sound Source Perception in Gestural Sound Description

Caramiaux

Bevilacqua

Bianco

et al. 2014

ACM Trans. Appl. Percept.

View full text Add to dashboard Cite

We investigated gesture description of sound stimuli performed during a listening task. Our hypothesis is that the strategies in gestural responses depend on the level of identification of the sound source, and specifically on the identification of the action causing the sound. To validate our hypothesis, we conducted two experiments. In the first experiment, we built two corpora of sounds. The first corpus contains sounds with identifiable causal actions. The second contains sounds where no causal actions could be identified. These corpora properties were validated through a listening test. In the second experiment, participants performed arm and hand gestures synchronously while listening to sounds taken from these corpora. Afterwards, we conducted interviews asking participants to verbalize their experience, watching their own video recordings. They were questioned on their perception of the listened sounds and on their gestural strategies. We showed that for the sounds where causal action can be identified, participants mainly mimic the action that has produced the sound. In the other case, when no action can be associated to the sound, participants trace contours related to sound acoustic features. We also found that the inter-participants gesture variability is higher for causal sounds compared to non-causal sounds. Variability demonstrates that in the first case, participants have several ways of producing the same action whereas in the second case, the sound features tend to make the gesture responses consistent.

show abstract

Section: Introductionmentioning

confidence: 99%

The Role of Sound Source Perception in Gestural Sound Description

Caramiaux

Bevilacqua

Bianco

et al. 2014

ACM Trans. Appl. Percept.

View full text Add to dashboard Cite

show abstract

“…This may be seen as a variation of 'air performance' studies, in which participants try to imitate the sound-producing actions of the music to which they listen [2]. Nymoen et al carried out a series of sound-tracing studies focusing on movements of the hands [60,61], elaborating on several feature extraction methods to be used for sound-tracing as a methodology. The current paper is inspired by these studies, but extending the methodology to full-body motion capture.…”

Section: Sound-tracingsmentioning

confidence: 99%

Analyzing Free-Hand Sound-Tracings of Melodic Phrases

Kelkar

Jensenius

2018

Applied Sciences

View full text Add to dashboard Cite

Abstract:In this paper, we report on a free-hand motion capture study in which 32 participants 'traced' 16 melodic vocal phrases with their hands in the air in two experimental conditions. Melodic contours are often thought of as correlated with vertical movement (up and down) in time, and this was also our initial expectation. We did find an arch shape for most of the tracings, although this did not correspond directly to the melodic contours. Furthermore, representation of pitch in the vertical dimension was but one of a diverse range of movement strategies used to trace the melodies. Six different mapping strategies were observed, and these strategies have been quantified and statistically tested. The conclusion is that metaphorical representation is much more common than a 'graph-like' rendering for such a melodic sound-tracing task. Other findings include a clear gender difference for some of the tracing strategies and an unexpected representation of melodies in terms of a small object for some of the Hindustani music examples. The data also show a tendency of participants moving within a shared 'social box'.

show abstract

“…These include so-called "air instrument" studies, where we wanted to find out what knowledge of sound-producing body motion people with different levels of musical training possessed (Godøy, Jensenius, & Haga, 2006). Related to this, we also conducted a number of studies of so-called "sound-tracing," in which listeners were asked to make spontaneous tracings of the salient features of sound excerpts, either on a digital tablet or in threedimensional space with their hands (Nymoen et al, 2013). In addition, we studied what we called "free dance"-spontaneous whole-body motion to a given sound excerpt (Haga, 2008).…”

Section: Motor Theorymentioning

confidence: 99%

Ubiquitous Motor Cognition in Musical Experience: Open Peer Review of Jacques Launay’s “Musical Sounds, Motor Resonance, and Detectable Agency”

Godøy

2015

EMR

View full text Add to dashboard Cite

Motor cognition, defined as the capacity to conceive, plan, control, perceive, and imagine body motion, is here seen as a ubiquitous element in music: music is produced by body motion, people often move in various ways when listening to music, and images of body motion seem to be integral to mental images of musical sound. Given this ubiquity of motor cognition in musical experience, it could be argued that motor cognition is a fundamental element in music, and thus could be hypothesized to also have been an essential element in the evolution of music, regardless of whether music is seen as primarily a social or as a more solitary phenomenon. It could furthermore be argued that music in all cases has intersubjective significance because of shared motor cognition among people, and also that this motor cognition may be applied to most perceptually salient features of music.

show abstract

Analyzing correspondence between sound objects and body motion

Cited by 37 publications

References 35 publications

The Role of Sound Source Perception in Gestural Sound Description

The Role of Sound Source Perception in Gestural Sound Description

Analyzing Free-Hand Sound-Tracings of Melodic Phrases

Ubiquitous Motor Cognition in Musical Experience: Open Peer Review of Jacques Launay’s “Musical Sounds, Motor Resonance, and Detectable Agency”

Contact Info

Product

Resources

About