Multimodal Development in Children's Narrative Speech: Evidence for Tight Gesture–Speech Temporal Alignment Patterns as Early as 5 Years Old

Abstract

Purpose: This study aims to analyze the development of gesture–speech temporal alignment patterns in children's narrative speech from a longitudinal perspective and, specifically, the potential differences between different gesture types, namely, gestures that imagistically portray or refer to semantic content in speech (i.e., referential gestures) and those that lack semantic content (i.e., non-referential gestures). Method: This study uses an audiovisual corpus of narrative productions ( n = 332) from 83 children (43 girls, 40 boys) who participated in a narrative retelling task at two time points in development (at 5–6 and 7–9 years of age). The 332 narratives were coded for both manual co-speech gesture types and prosody. Gestural annotations included gesture phasing (i.e., preparation, stroke, hold, and recovery) and gesture types (in terms of referentiality, i.e., referential and non-referential), whereas prosodic annotations included pitch-accented syllables. Results: Results revealed that by ages 5–6 years, children already temporally aligned the stroke of both referential and non-referential gestures with pitch-accented syllables, showing no significant differences between these two gesture types. Conclusions: The results of the present study contribute to the view that both referential and non-referential gestures are aligned with pitch accentuation, and therefore, this is not only a characteristic of non-referential gestures. Our results also add support to McNeill's phonological synchronization rule from a developmental perspective and indirectly back up recent theories about the biomechanics of gesture–speech alignment, suggesting that this is an inherent ability of oral communication.

Multimodal language in child-directed versus adult-directed speech

“…This finding corroborates earlier evidence suggesting that beat gestures are used extensively during narrative discourse (Vilà-Giménez et al, 2019, 2021). Moreover, using and observing beat gestures enhance narrative performance, such as producing and recalling narratives (Florit-Pons et al, 2020; Llanes-Coromina et al, 2018; Vilà-Giménez et al, 2019, 2021; Vilà-Giménez & Prieto, 2020). Thus, the narrative context in the storytelling task compared with the other two tasks that did not involve narration might yield present results on beat gesture use.…”

Référence multimodale dans les narrations d’enfants : les gestes servent-ils à clarifier les expressions référentielles ambiguës ?

“…D'une part, de nombreuses études se sont intéressées à la construction des chaînes de référence dans les récits d'enfants, montrant des difficultés persistant bien au-delà de 6 ans dans le choix 1 de formes linguistiques adaptées à la présentation et à la réintroduction de référents (Bamberg, 1986 ;Dupret, 2022 ;Hickmann, 2002 ;Jisa, 2000 ;Karmiloff-Smith, 1985 ;Kern, 1997). D'autre part, plusieurs études ont analysé l'utilisation des gestes co-verbaux accompagnant le discours narratif, mettant en évidence la complexité de la structuration d'un récit à différents niveaux et sous différentes dimensions (McNeill et Levy, 1982 ;Alibali et al, 2009 ;Alamillo et al, 2013 ;Demir et al, 2015 ;Vilà-Giménez, 2020 ;Bressem et al, 2018 ;Cravotta et al, 2019 ;Florit-Pons et al, 2020). Cependant, les études qui ont pour but d'analyser l'articulation entre emploi de gestes et usage de la référence en récit sont plus rares, bien que des recherches aient été menées auprès d'adultes (So et al, 2013 ;Perniss et Özyürek, 2015).…”