Hearing non-signers use their gestures to predict iconic form-meaning mappings at first exposure to signs

Abstract

The sign languages of deaf communities and the gestures produced by hearing people are communicative systems that exploit the manual-visual modality as means of expression. Despite their striking differences they share the property of iconicity, understood as the direct relationship between a symbol and its referent. Here we investigate whether non-signing hearing adults exploit their implicit knowledge of gestures to bootstrap accurate understanding of the meaning of iconic signs they have never seen before. In Study 1 we show that for some concepts gestures exhibit systematic forms across participants, and share different degrees of form overlap with the signs for the same concepts (full, partial, and no overlap). In Study 2 we found that signs with stronger resemblance with signs are more accurately guessed and are assigned higher iconicity ratings by non-signers than signs with low overlap. In addition, when more people produced a systematic gesture resembling a sign, they assigned higher iconicity ratings to that sign. Furthermore, participants had a bias to assume that signs represent actions and not objects. The similarities between some signs and gestures could be explained by deaf signers and hearing gesturers sharing a conceptual substrate that is rooted in our embodied experiences with the world. The finding that gestural knowledge can ease the interpretation of the meaning of novel signs and predicts iconicity ratings is in line with embodied accounts of cognition and the influence of prior knowledge to acquire new schemas. Through these mechanisms we propose that iconic gestures that overlap in form with signs may serve as some type of 'manual cognates' that help non-signing adults to break into a new language at first exposure.

Use and acquisition of mouth actions in L2 sign language learners

“…Hearing L2 learners might transfer forms of the same gestures, such as thumbs up and no (palms forward) to their signing for signs with equivalent meaning (Chen Pichler et al 2016). Gestural knowledge is seen to facilitate L2 signers' comprehension but can contribute to L2 phonological variations in signs, i.e., errors (Ortega & Morgan 2015b;Ortega, Schiefner & Özyürek 2019). Ortega, Schiefner & Özyürek (2019) furthermore suggest that the gestural repertoire serves as manual cognates for learning the sign language lexicon.…”

“…Gestural knowledge is seen to facilitate L2 signers' comprehension but can contribute to L2 phonological variations in signs, i.e., errors (Ortega & Morgan 2015b;Ortega, Schiefner & Özyürek 2019). Ortega, Schiefner & Özyürek (2019) furthermore suggest that the gestural repertoire serves as manual cognates for learning the sign language lexicon. Studies by Gullberg and colleagues (Gullberg 1998;Gullberg, Bot & Volterra 2008) have also pointed out the important role of gestures in spoken Second Language Acquisition, among other things, finding a link between language fluency and gesture frequency: that is, gestures are (among other things) used as communica-tion strategies to compensate for less language fluency and to overcome obstacles (Gullberg 1998).…”

Multimodality and the origin of a novel communication system in face-to-face interaction

“…Many of these concepts are already associated with conventionalized gestures. For example, Ortega et al [109,110] asked hearing participants to produce gestures for a range of meanings. For many meanings, the gestures produced were highly consistent across participants.…”

Innovation and enculturation in child communication: a cross-sectional study

“…In other words, the ways in which people conceptualize real-world objects or concepts derives from the ways in which they physically interact with those objects or concepts (Hostetter & Alibali, 2008). Grounding real-world concepts in terms of their associated motor schemas makes it likely that people who interact physically with the world in similar ways will tend to align on the same gestures when communicating the same concepts (Ortega et al, 2019). This may explain why we observed more similarity among participants' gestures than their vocalizations: participants of all ages tend to interact with their environment in similar physical ways, so when asked to represent the world through gestures, they tended to align upon similar (and mutually salient) solutions.…”