Learning Movement Kinematics with a Targeted Sound

Boyer, Eric; Pyanet, Quentin; Hanneton, Sylvain; Bevilacqua, Frédéric

doi:10.1007/978-3-319-12976-1_14

Cited by 10 publications

(12 citation statements)

References 19 publications

(19 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since both feedback types can occur simultaneously during real-time movement sonification (Boyer et al, 2014), specific formalization of auditory feedback types should be developed.…”

Section: Discussion and Future Research Challengesmentioning

confidence: 99%

Sensori-Motor Learning with Movement Sonification: Perspectives from Recent Interdisciplinary Studies

et al. 2016

Self Cite

View full text Add to dashboard Cite

This article reports on an interdisciplinary research project on movement sonification for sensori-motor learning. First, we describe different research fields which have contributed to movement sonification, from music technology including gesture-controlled sound synthesis, sonic interaction design, to research on sensori-motor learning with auditory-feedback. In particular, we propose to distinguish between sound-oriented tasks and movement-oriented tasks in experiments involving interactive sound feedback. We describe several research questions and recently published results on movement control, learning and perception. In particular, we studied the effect of the auditory feedback on movements considering several cases: from experiments on pointing and visuo-motor tracking to more complex tasks where interactive sound feedback can guide movements, or cases of sensory substitution where the auditory feedback can inform on object shapes. We also developed specific methodologies and technologies for designing the sonic feedback and movement sonification. We conclude with a discussion on key future research challenges in sensori-motor learning with movement sonification. We also point out toward promising applications such as rehabilitation, sport training or product design.

show abstract

“…Since both feedback types can occur simultaneously during real-time movement sonification (Boyer et al, 2014), specific formalization of auditory feedback types should be developed.…”

Section: Discussion and Future Research Challengesmentioning

confidence: 99%

Sensori-Motor Learning with Movement Sonification: Perspectives from Recent Interdisciplinary Studies

et al. 2016

Self Cite

View full text Add to dashboard Cite

show abstract

“…While these different interactions are performed in the air without any tangible interface, the very familiar types of sound that responds to the movements (both the clear splashing at the surface and the muffled underwater sound) can create the illusion of the water volume (Boyer 2014).…”

Section: Virtual Water Tankmentioning

confidence: 99%

Designing Action–Sound Metaphors Using Motion Sensing and Descriptor-Based Synthesis of Recorded Sound Materials

Bevilacqua¹,

Schnell²,

Françoise³

et al. 2017

The Routledge Companion to Embodied Music Interaction

Self Cite

View full text Add to dashboard Cite

In this chapter, we present a general methodology for designing interactive musical systems, using movement sensing and descriptor-based synthesis of recorded sound materials. Importantly, the design principles focus on action-sound metaphors that can be built upon audio features of recorded sound materials and their possible relationships to human movement. We describe the critical design choices, including sensing technologies, movement analysis, and action-sound models, along with application examples.

show abstract

“…DANCE shares similarities with the previous Embodied Generative Music project (EGM). 2 The purpose of the DANCE project is to investigate if it is possible to perceive expressive movement qualities in dance solely through the auditory channel, i.e. to capture expressive qualities of dance movements and convey them through sounds.…”

Section: Introductionmentioning

confidence: 99%

“…For example, in a study on the sonification of handwriting it was found that sound models characterized by low frequency components were more suitable for both aiding and communicating fluency of movements, while sound models characterized by high frequency crackling sounds (impact sounds) were suitable for portraying jerky hand movements lacking fluency [7]. In another study focusing on the use of sounds in learning of movement kinematics, researchers found that sounds that were noisier or had loud high-frequency components could help users identify motion behavior [2]. Moreover, in our previous study [11], a smooth sound model with a continuous sound was rated as significantly more fluid than a sound model characterized by a high level of amplitude modulation and sudden amplitude irregularities.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Interactive sonification of a fluid dance movement: an exploratory study

Frid

Elblaus

Bresin

2018

J Multimodal User Interfaces

View full text Add to dashboard Cite

In this paper we present three different experiments designed to explore sound properties associated with fluid movement: (1) an experiment in which participants adjusted parameters of a sonification model developed for a fluid dance movement, (2) a vocal sketching experiment in which participants sketched sounds portraying fluid versus nonfluid movements, and (3) a workshop in which participants discussed and selected fluid versus nonfluid sounds. Consistent findings from the three experiments indicated that sounds expressing fluidity generally occupy a lower register and has less high frequency content, as well as a lower bandwidth, than sounds expressing nonfluidity. The ideal sound to express fluidity is continuous, calm, slow, pitched, reminiscent of wind, water or an acoustic musical instrument. The ideal sound to express nonfluidity is harsh, non-continuous, abrupt, dissonant, conceptually associated with metal or wood, unhuman and robotic. Findings presented in this paper can be used as design guidelines for future applications in which the movement property fluidity is to be conveyed through sonification.

show abstract

Learning Movement Kinematics with a Targeted Sound

Cited by 10 publications

References 19 publications

Sensori-Motor Learning with Movement Sonification: Perspectives from Recent Interdisciplinary Studies

Sensori-Motor Learning with Movement Sonification: Perspectives from Recent Interdisciplinary Studies

Designing Action–Sound Metaphors Using Motion Sensing and Descriptor-Based Synthesis of Recorded Sound Materials

Interactive sonification of a fluid dance movement: an exploratory study

Contact Info

Product

Resources

About