Multimodality and construction grammar

Abstract

The meaning-making process in face-to-face interaction relies on the integration of meaningful information being conveyed by speech as well as the tone of voice, facial expressions, hand and head gestures, body postures and movements (McNeill 1992;Kendon 2004). Hence, it is inherently multimodal. Usage-based linguistics attributes language use a fundamental role in linguistic theorizing by positing that the language system is grounded in and abstracted from (multimodal) language use. However, despite this inherent epistemological link, usage-based linguists have hitherto conceptualized language as a system of interconnected verbal, i. e. monomodal units, leaving nonverbal usage aspects and the question of their potential entrenchment as part of language largely out of the picture. This is -at least at first sight -surprising because the usage-based model of Construction Grammar (C × G) seems particularly well-equipped to unite the natural interest of linguists in the units that define language systems with the multimodality of language use. Constructions are conceptualized as holistic "conventionalized clusters of features (syntactic, prosodic, pragmatic, semantic, textual, etc.) that recur as further indivisible associations between form and meaning" (Fried 2015: 974). Given its conceptual openess to all levels of usage features, several studies have recently argued for the need to open up the current focus of C × G towards kinesic recurrences (Günthner &

Constructing a protolanguage: reconstructing prehistoric languages in a usage-based construction grammar framework

“…In addition, instead of being only gestural or only vocal, PL1 was very likely multimodal in nature [106–109]. This assumption is consistent with research on multimodal processing in human communication [110,111], and can also be captured in a CxG framework [112,113]. Current approaches in a CxG framework see language as a multimodal phenomenon and have also started to describe constructions in multimodal terms.…”

When Gesture “Takes Over”: Speech-Embedded Nonverbal Depictions in Multimodal Interaction

“…Finally, recent years have seen attempts at incorporating gesture into the theoretical framework of linguistic analysis, coming from various theoretical orientations and with different approaches [e.g., “integrated message model” ( Bavelas and Chovil, 2000 ); “composite signal” ( Clark, 1996 ); “composite utterance” ( Enfield, 2009 ); “multimodal grammar” ( Fricke, 2012 ); multimodal negation ( Harrison, 2018 ); incorporation of gesture into Cognitive Grammar ( Kok and Cienki, 2016 ); “mixed syntax” ( Slama-Cazacu, 1976 )]. Construction Grammar, in particular, sees a recent debate on Multimodal Construction Grammar (e.g., Steen and Turner, 2013 ; Schoonjans et al, 2015 ; Cienki, 2017 ; Hoffmann, 2017 ; Schoonjans, 2017 ; Ziem, 2017 ; Zima and Bergs, 2017 ). Arguing for nonverbal signals being as integral to language as canonical speech, these studies touch upon cases of gestures without simultaneous speech, acknowledging their crucial role in language use, but the primary focus remains on gesture-speech co-occurrence.…”

Construction grammar for monkeys?

“…Lepic, 2019) and co-speech gesture (e.g. Zima & Bergs, 2017) represent a burgeoning field with a wealth of theoretical and empirical developments that promise to be highly relevant for studies of ape gestures. This is especially so given that many models both in animal communication and linguistics treat vocal and gestural signals very differently (Liebal & Oña, 2018), a problem that might be overcome with the help of multimodal construction grammar.…”