Gesture-Timbre Space: Multidimensional Feature Mapping Using Machine Learning and Concatenative Synthesis

Abstract

This paper presents a method for mapping embodied gesture, acquired with electromyography and motion sensing, to a corpus of small sound units, organised by derived timbral features using concatenative synthesis. Gestures and sounds can be associated directly using individual units and static poses, or by using a sound tracing method that leverages our intuitive associations between sound and embodied movement. We propose a method for augmenting corporal density to enable expressive variation on the original gesture-timbre space.

From a PhD to assisting biomusic research

“…It provides guidelines to make the product and its features visible and understandable, to foresee errors and to provide ways for fixing potential mistakes (Norman, 1988). The UCD method was adapted here to be applied in a musical context, and it centred around participatory workshops, rehearsals and performances (Di Donato et al, 2020;Tanaka et al, 2019;Zbyszyński et al, 2020). These were designed to leverage practitioner knowledge in performance Di Donato (2022) From a PhD To Assisting BioMusic Research.…”