The timing of head movements: The role of prosodic heads and edges

Abstract

This study examines the influence of the position of prosodic heads (accented syllables) and prosodic edges (prosodic word and intonational phrase boundaries) on the timing of head movements. Gesture movements and prosodic events tend to be temporally aligned in the discourse, the most prominent part of gestures typically being aligned with prosodically prominent syllables in speech. However, little is known about the impact of the position of intonational phrase boundaries on gesture-speech alignment patterns. Twenty-four Catalan speakers produced spontaneous (experiment 1) and semi-spontaneous head gestures with a confirmatory function (experiment 2), along with phrasefinal focused words in different prosodic conditions (stress-initial, stress-medial, and stress-final). Results showed (a) that the scope of head movements is the associated focused prosodic word, (b) that the left edge of the focused prosodic word determines where the interval of gesture prominence starts, and (c) that the speech-anchoring site for the gesture peak (or apex) depends both on the location of the accented syllable and the distance to the upcoming intonational phrase boundary. These results demonstrate that prosodic heads and edges have an impact on the timing of head movements, and therefore that prosodic structure plays a central role in the timing of co-speech gestures.

The Prosodic Characteristics of Non-referential Co-speech Gestures in a Sample of Academic-Lecture-Style Speech

“…In this paper, we focus on the manual co-speech gestures in the sample, i.e., those that involve the hand(s) and arm(s) of the speaker [The co-speech movements of other articulators, such as the head, eyebrows, direction of gaze and upper torso, are also of interest for their alignment with spoken prosody (e.g., McClave, 2000 ; Keating et al, 2003 ; Shattuck-Hufnagel et al, 2010 ; Swerts and Krahmer, 2010 ; Esteve-Gibert et al, 2017 ) but will not be discussed here]. For most aspects of the labeling, the gestures were annotated without listening to the speech, and the speech without viewing the video, to avoid any possibility of the labeler’s judgment about events in one channel being influenced by events in the other.…”

“…Over the past few decades, the incorporation of phrase-level prosodic constituency and prominence patterns into linguistic grammars (e.g., Liberman and Prince, 1977 ; Selkirk, 1984 ; Beckman and Pierrehumbert, 1986 ; Nespor and Vogel, 1986 ), along with the development of an extensive system for capturing significant systematic aspects of gestural movements and their communicative function ( Kendon, 1972 , 1980 , 2004 ; McNeill, 1992 , 2005 ) has opened the door to a range of studies asking how these two streams of behavior interact. This is an important question, because to the extent that both sets of actions contribute to the communication of a message during the act of speaking, it is a reasonable presumption that they are planned together ( Esteve-Gibert and Prieto, 2013 ; Krivokapic, 2014 ; Wagner et al, 2014 ; Krivokapic et al, 2015 ; Esteve-Gibert et al, 2017 ). Such a view has critical implications for the development of a comprehensive model of the speech production planning process.…”

How Referential Gestures Align With Speech: Evidence From Monolingual and Bilingual Speakers

“…A growing body of research has targeted temporal alignment between speech and movements of other articulators, such as head, eyebrows, torso (e.g., Ambrazaitis & House, ; Esteve‐Gibert, Borras‐Comes, Asor, Swerts, & Prieto, ; Swerts & Krahmer, ; Wagner, ). We have not discussed this literature here because the present analysis focused solely on manual gesture.…”

Coverbal speech gestures signal phrase boundaries: A production study of Japanese and English infant- and adult-directed speech

“…Phrasing also impacts the use of coverbal gestures. Catalan talkers spontaneously produce nods in focused words, and importantly, the alignment of their peaks is influenced by whether a phrase boundary precedes or follows the word (Esteve-Gibert et al 2017). Further, Japanese talkers produce head nods in about 30%-40% of phrases containing strong boundaries, but only in 10%-15% of phrases with weak boundaries (Ishi, Ishiguro & Hagita 2014).…”