Gesture's body orientation modulates the N400 for visual sentences primed by gestures

Abstract

Body orientation of gesture entails social‐communicative intention, and may thus influence how gestures are perceived and comprehended together with auditory speech during face‐to‐face communication. To date, despite the emergence of neuroscientific literature on the role of body orientation on hand action perception, limited studies have directly investigated the role of body orientation in the interaction between gesture and language. To address this research question, we carried out an electroencephalography (EEG) experiment presenting to participants (n = 21) videos of frontal and lateral communicative hand gestures of 5 s (e.g., raising a hand), followed by visually presented sentences that are either congruent or incongruent with the gesture (e.g., “the mountain is high/low…”). Participants underwent a semantic probe task, judging whether a target word is related or unrelated to the gesture‐sentence event. EEG results suggest that, during the perception phase of handgestures, while both frontal and lateral gestures elicited a power decrease in both the alpha (8–12 Hz) and the beta (16–24 Hz) bands, lateral versus frontal gestures elicited reduced power decrease in the beta band, source‐located to the medial prefrontal cortex. For sentence comprehension, at the critical word whose meaning is congruent/incongruent with the gesture prime, frontal gestures elicited an N400 effect for gesture‐sentence incongruency. More importantly, this incongruency effect was significantly reduced for lateral gestures. These findings suggest that body orientation plays an important role in gesture perception, and that its inferred social‐communicative intention may influence gesture‐language interaction at semantic level.

Will “Hybrid” Meetings Replace Face-To-Face Meetings Post COVID-19 Era? Perceptions and Views From The Urological Community

“…11 One clear advantage of face-to-face meetings is that it allows the participant the benefit of listening to information while observing the speaker's body language, facial expressions, and gestures as often these visual cues improve the ability of people to communicate effectively. 12 Similarly, they give the opportunity to ask questions, interact and network with other colleagues and specialists. Maybe, this is why 80% of the respondents think that face-to-face meetings are a good learning opportunity.…”

Modality-specific dysfunctional neural processing of social-abstract and non-social-concrete information in schizophrenia

“…In everyday social communication, patients with schizophrenia encounter a diverse spectrum of inputs from multiple modalities (Holler & Levinson, 2019). These include non-linguistic stimuli from others’ facial expressions (Gur et al, 2002; Vuilleumier et al, 2001), body movements, e.g., postures and gestures (He et al, 2020; Nagels et al, 2015b; Straube et al, 2013a), and linguistic stimuli in the form of auditory speech and written texts (Brown & Kuperberg, 2015). Importantly, the multimodal inputs comprise both social-abstract and non-social-concrete information, and patients’ social functioning heavily depends on the processing of these stimuli, which serves as the basis to further mentalize social intentions and to perform appropriate social interaction (Amodio & Frith, 2006).…”

Online, face to face and hybrid meetings. What is the new normal in the COVID-19 era? A global collaborative survey by iTRUE group and UroSoMe working group

“…One clear advantage of face-to-face meetings is that it allows the participant the bene t of listening to information while observing the speaker's body language, facial expressions, and gestures as often these visual cues improve the ability of people to communicate effectively [8]. Maybe, this is why 80% of the respondents think that face-to-face meetings are a good learning opportunity.…”

“…Competing interests: The authors declare no competing interests. Opportunities for research collaboration 0.500 (10) 0.444 (11) 0.542 (11) Cost effectiveness 0.472 (11) 0.693 (1) 0.594 (8)…”