Assessing Sound Perception Through Vocal Imitations of Sounds that Evoke Movements and Materials

Abstract

In this paper we studied a new approach to investigate sound perception. Assuming that a sound contains specific morphologies that convey perceptually relevant information responsible for its recognition, called invariants, we explored the possibility of a new method to determine such invariants, using vocal imitation. We conducted an experiment, asking participants to imitate sounds evoking movements and materials generated through a sound synthesizer. Given that that the sounds produced by the synthesizer were based on invariant structures, we aimed at retrieving this information from the imitations. Results showed that the participants were able to correctly imitate the dynamics of the sounds, i.e. the action-related information evoked by the sound, whereas texture-related information evoking the material of the sound source was less easily imitated.

Timbre from Sound Synthesis and High-Level Control Perspectives

“…Such approaches have for instance been used in the synthesis of guitar sounds using vocal imitation (Traube and Depalle 2004), to extract relevant features of kitchen sounds (Lemaitre et al 2011, Lemaitre and Susini, Chap. 9) and are currently used to identify perceptually relevant sound structures of evoked movements and materials (Bordonné et al 2017). The vocal imitations provided a simplified description of the dynamic evolution that enabled to link the perceived sportiness to the "ON/AN" transitions.…”

Animation notation, score distribution and AR-VR environments for spectral mimetic transfer in music composition

“…According to the author, Thomas Bordonné, this study "aimed at determining the main characteristics of sounds used by participants during vocal imitations". Bordonné concludes: "Vocal imitations seem to be a good tool to access perception and determine which aspects of the sounds are relevant" [16]. In this setup, therefore, the spontaneous vocal response of participants could be interpreted in similar ways to the SmartVox-led reading sessions in which the singers are asked to imitate what they hear.…”

“…However, important questions then arise regarding what would be pertinent to achieve with such technology: in a VR context for instance, with a player on stage, how can the limitation due to the destabilization of the performer's proprioception be overcome? Also, now that any form of immersive score can be prototyped, 16 how would this be of musical interest, and how could this have a convincing musical impact on the compositional/performative outcome? SmartVox, for instance, delivers and synchronizes mp4 files, and should soon support 360-degree videos 17 , but how can this benefit the performer in a musical sense?…”

How Would You Reproduce a Synthetic Sound of an Ellipse in Water? A Phenomenological Investigation of Pre-reflexive Contents of Consciousness

“…Other approaches such as sensory analysis during which a group of subjects identify sensory descriptors such as onomatopoeias have been used, for instance to characterize the formantic transition from"ON" (pronounced [Õ]) to "AN" (pronounced [Ã]) that characterizes sounds from car engines [22,25]. Other approaches, such as vocal imitations, that do not specifically focus on everyday or analytical listening have been used to extract relevant features of kitchen sounds [16], and more recently to reveal invariant structures responsible for the evocation of movements and materials [4,5]. Psycholinguistic analyses have been used to characterize sounds from musical instruments through spontaneous verbalizations.…”