Language comprehenders represent object distance both visually and auditorily

Abstract

Abstract. While the arbitrariness of the sign has occupied a central space in linguistic theory for a century, counter-evidence to this basic tenet has been mounting. Recent findings from cross-linguistic studies on spoken languages have suggested that, contrary to purely arbitrary distributions of phonological content, languages often exhibit systematic and regular phonological and sub-phonological patterns of form-meaning mappings. To date, studies of distributional tendencies of this kind have not been conducted for signed languages.In an investigation of phoneme distribution in American Sign Language (ASL) and Língua Brasileira de Sinais (Libras), tokens of the claw-5 handshape were extracted and analyzed for whether the handshape contributed to the overall meaning of the sign. The data suggests that distribution of the claw-5 handshape is not randomly distributed across the lexicon, but clusters around six formmeaning patterns: convex-concave, Unitary-elements, non-compact matter, hand-as-hand, touch, and interlocking. Interestingly, feature-level motivations were uncovered as the source of the mappings These findings are considered within a new cognitive framework to better understand how and why sub-morphemic units develop and maintain motivated form-meaning mappings. The model proposed here, Embodied Cognitive Phonology, builds on cognitive and usage-based approaches but incorporates theories of embodiment to address the source of the claw-5 mappings. Embodied Cognitive Phonology provides a unifying framework for understanding the perceived differences in phonological patterning and organization across the modalities. Both language-internal and languageexternal sources of motivation contribute to the emergence of form-meaning mappings. Arbitrariness is argued to be but one possible outcome from the process of emergence and schematization of phonological content, and exists alongside motivation as a legitimate state of linguistic units of all sizes of complexity. Importantly, because language is dynamic, these states are not fixed, but are in continuous flux, as language users reinvent and reinterpret form and meaning over time.

Embodied cognition and the magical future of interaction design

“…Even just listening to language can trigger these activations in the associative cortex. The sentence "the alarm sounded and John jumped out of bed" will activate areas in the auditory and motor cortex related to alarms and jumping out of bed [Kaschak et al 2006;Winter and Bergen 2012]. This is the received embodiment view.…”

Dutch modality exclusivity norms: Simulating perceptual modality in space

“…Further, cross-linguistic research shows many languages make a distinction between near and far space in demonstratives (e.g., this and that) (Diessel, 1999;Levinson et al, in press), suggesting the psychological salience of this spatial distinction (see also Kemmerer, 1999). Winter and Bergen (2012) have shown spatial distance is actually simulated during sentence comprehension. They found participants were faster to respond to large pictures and loud sounds after reading sentences describing objects near in space rather than far in space (e.g., You are looking at the milk bottle in the fridge/across the supermarket), and faster to respond to small pictures and quiet sounds after sentences describing objects that are far rather than near.…”

“…Previous work shows objects can be simulated in near and far space based on their described location (Winter & Bergen, 2012), but can the simulation of near and far space also be affected by the postulated differences in perceptual modalities described above? If so, then objects strongly experienced in the visual modality (e.g., traffic light) should be perceptually simulated in the distance, as should objects strongly experienced in the auditory modality (e.g., thunder).…”

“…In order to investigate this idea, we tested for spatial simulation using a lexical decision task with words presented as Bnear to the body^or Bfar from the body.^We used presentation size to imply distance (as in Winter & Bergen, 2012): words in the near condition had a larger font than words in the far condition. We also presented words as high or low on screen to enhance this distance through perspective.…”

An Introduction to Embodied Cognitive Phonology: Claw-5 Hand-shape Distribution in ASL and Libras

“…Mental simulation studies reveal that spoken language users engage areas of the brain responsible for physical motor routines (without producing the physical action) (Glenberg & Kaschak 2002), and vision (Kosslyn, Ganis & Thompson 2001) when engaging in linguistic tasks. Winter and Bergen (2012) have shown that linguistic processing which references what something sounds like engages perceptual representations of what those relevant objects/events sound like. Work on simulation clearly shows that vision, motor routines, audition, and proprioception are all at work in dynamic construction of meaning.…”