Evaluating Models of Gesture and Speech Production for People With Aphasia

Abstract

People with aphasia use gestures not only to communicate relevant content but also to compensate for their verbal limitations. The Sketch Model (De Ruiter, 2000) assumes a flexible relationship between gesture and speech with the possibility of a compensatory use of the two modalities. In the successor of the Sketch Model, the AR‐Sketch Model (De Ruiter, 2017), the relationship between iconic gestures and speech is no longer assumed to be flexible and compensatory, but instead iconic gestures are assumed to express information that is redundant to speech. In this study, we evaluated the contradictory predictions of the Sketch Model and the AR‐Sketch Model using data collected from people with aphasia as well as a group of people without language impairment. We only found compensatory use of gesture in the people with aphasia, whereas the people without language impairments made very little compensatory use of gestures. Hence, the people with aphasia gestured according to the prediction of the Sketch Model, whereas the people without language impairment did not. We conclude that aphasia fundamentally changes the relationship of gesture and speech.

Verdeckung neurogener Sprachstörungen in der sozialen Interaktion

“…Levinson 57 befassen sich genauer mit den Phänomenen, die den Austausch von Informationen, die Regulierung des Turn Taking und bei auftretenden Problemen dann Repair-Strategien betrachtet. Levinson bezieht hier neben der verbalen auch die nonverbalen Ebenen, speziell die Gesten, mit ein, wie dies auch im Modell von de Ruiter 58 formalisiert und von de Beer et al 59 auf die kommunikativen Strategien bei Menschen mit Aphasie bezogen wird.…”

What is Functional Communication? A Theoretical Framework for Real-World Communication Applied to Aphasia Rehabilitation

“…Alongside the emergence of single-word processing models, researchers such as Kendon (1980), McNeill (1992, and Goodwin (1995) emphasised the multimodal nature of information exchange, in which non-verbal channels such as gesture, facial expression, and body movement complement and supplement verbal language. Gesture has since become a fruitful area of research in aphasia (e.g., de Beer et al, 2017de Beer et al, , 2020Roper et al, 2016;van Nispen et al, 2017; see section Language use is multimodal below).…”

A kinematic-acoustic analysis of gesture-speech coupling in persons with aphasia

“…The increased gesture use has been compared to communicative impairments in typical aging (>60 years of age) versus aphasic populations (Kong, Law, Kwan, et al, 2015;Kong, Law, Wat, et al, 2015). As both aging neurotypicals and aphasic populations are affected in linguistic performance tasks of fluency, word selection, and comprehension, and given that linguistic performance inclination related to gesture increases, it may be concluded that there is compensatory support by gesture in communication (Beer et al, 2020;de Ruiter, 2000). Research on the nature of the compensation suggests that it might lie in the gestural clarification or disambiguation of difficult-to-interpret speech, as well as resolving word retrieval issues (Akhavan et al, 2018;Beer et al, 2020;Kistner et al, 2019;Rose et al, 2017;van Nispen et al, 2017).…”

“…We keep our overview of mainstream gesture research and the implications for possible "box-and-arrow" theories of gesture production in aphasia deliberately brief (see however de Beer et al, 2020), as they simply are not directly engaging with the phenomenon of multimodal prosody, i.e., the way that gesture modulations (in kinematics) relate to (acoustic markers of) speech prosody (Wagner et al, 2014). This is a pity as the prosodic modulations together with gestures may be an important semiotic means to communicate in aphasia despite impaired speech processes on other levels (syntax, lexical).…”

“…What mechanisms do we need to invoke to model such a behavioral feat? In De Ruiter's extended Levelt model, it is assumed that at later formulation stages gesture and speech become "executed separately by the two underlying processing modules, which are temporally coordinated by a signaling mechanism", p. 3, emphasis added (Beer et al, 2020). Indeed, when assuming some kind of independence while accounting for the fact that gesture and speech are tightly temporally dependent in neurotypicals, one needs to invoke some "signaling" mechanism that keeps two putatively independent processes in sync.…”