Sign advantage: Both children and adults’ spatial expressions in sign are more informative than those in speech and gestures combined

Abstract

Expressing Left-Right relations is challenging for speaking-children. Yet, this challenge was absent for signing-children, possibly due to iconicity in the visual-spatial modality of expression. We investigate whether there is also a modality advantage when speaking-children’s co-speech gestures are considered. Eight-year-old child and adult hearing monolingual Turkish speakers and deaf signers of Turkish-Sign-Language described pictures of objects in various spatial relations. Descriptions were coded for informativeness in speech, sign, and speech-gesture combinations for encoding Left-Right relations. The use of co-speech gestures increased the informativeness of speakers’ spatial expressions compared to speech-only. This pattern was more prominent for children than adults. However, signing-adults and children were more informative than child and adult speakers even when co-speech gestures were considered. Thus, both speaking- and signing-children benefit from iconic expressions in visual modality. Finally, in each modality, children were less informative than adults, pointing to the challenge of this spatial domain in development.

Understanding Children's Multimodal Conversational Development: Challenges and Opportunities

“…What about the development of these multimodal signaling strategies in childhood? While there is a wealth of studies about young children's exposure or use of multimodal signals [39,20,36,13], there is a scarcity of research investigating how these abilities develop into adult-like skills in face-to-face conversation. The few existing studies generally point towards a protracted learning timeline [e.g., 17].…”

Gestures cued by demonstratives in speech guide listeners’ visual attention during spatial language comprehension.

“…Gestures help both speakers and listeners when expressing, communicating, and thinking about spatial information (e.g., Allen, 2003; Chu & Kita, 2008, 2011; Emmorey et al, 2000; Göksun, Goldin-Meadow, et al, 2013; Hostetter et al, 2011; Özer et al, 2017; So et al, 2015). One spatial context in which speakers use an abundant number of gestures is the descriptions of relative spatial relations (e.g., left–right, on–under, front–behind, next to) between a figure object (i.e., the object whose relative position to be located) and a ground object (i.e., the reference object; Driskell & Radtke, 2003; Göksun, Lehet, et al, 2013; Holler et al, 2009; Karadöller et al, 2019; Karadöller, Sümer, Ünal, & Özyürek, 2021; Karadöller, Sümer, & Özyürek, 2021; McNeil et al, 2000).…”

“…Given gesture's expressive potential, speakers often convey visual-spatial information such as object size, object location, manner of movement, and spatial location in gestures rather than in speech (Beattie & Shovelton, 2006; Emmorey & Casey, 2001; Holler & Stevens, 2007; Karadöller et al, 2019, Karadöller, Sümer, & Özyürek, 2021; Kita & Özyürek, 2003). Speakers tend to encode spatial information such as object's size only in their gestures rather than in speech, especially when the size information is crucial (Beattie & Shovelton, 2006) or when they talk to interlocutors with whom the size information is new (Holler & Stevens, 2007).…”

“…The accompanying demonstratives in speech refer to speakers' gestures and signals to listeners that an essential part of the message is conveyed through the gesture channel. For example, a speaker who says, "The candle is here," can produce a gesture that shows the right side of the speaker to express that the candle is on the speaker's right side (Karadöller et al, 2019(Karadöller et al, , 2022. In such a multimodal utterance, the gesture expresses the critical complementary information to the accompanying speech and is communicatively cued by speech for the successful comprehension.…”

“…Individuals use their hands to think and communicate (McNeill, 1992), particularly for spatial information, such as relative spatial relations between objects (e.g., left-right or on-under; Alibali, 2005;Beattie & Shovelton, 1999a, 2002aDargue et al, 2019;Hostetter, 2011;Karadöller et al, 2019;Karadöller, Sümer, Ünal, & Özyürek, 2021;Lavergne & Kimura, 1987). Speakers tend to use gestures to complement spoken deictic constructions, such as place-referring demonstratives (e.g., "here") when describing spatial relations (e.g., Beattie & Shovelton, 2006;Holler & Stevens, 2007).…”