Social Resonance: Acoustic Information about Upper Limb Movement in Voicing

Abstract

We show that upper limb movement affects voice acoustics in a way that allows listeners to synchronize to those movements. In this pre-registered motion-tracking study (within-subjects, N = 30), participants listened to vocalizers producing a steady-state phonation while moving the upper limb with a wrist versus an arm motion at different tempos. Listeners were asked to synchronize their own wrist or arm movement with what they perceived in the voice acoustics to be the movement of the vocalizer. Previous research has shown that higher physical impetus gestures can directly affect lower vocal tract activity, leading to peaks in the fundamental frequency (F0; perceived as pitch) and intensity. Here we show that listeners can attune to acoustic information so as to synchronize both in frequency and in phasing with the vocalizer, even with very subtle movements. We interpret the current results in support of a possible ecological psychological approach to speech prosody.

Exploring the effects of restraining the use of gestures on narrative speech

“…Also, the speech data was tested perceptually and showed that listeners were not able to tell, by hearing the speakers' voice only, whether someone was gesturing or not while speaking. However, Pouw et al (2019) showed that gestures can actually be "heard" in the moment they occur during vocalizations, as forces reverberate to the lower vocal tract, affecting aspects of prosodic control (specifically, listeners succeeded in synchronizing their own wrist or arm movement with what they perceived in the voice acoustics to be the movement of the speaker).…”