Action-Related Speech Modulates Beta Oscillations During Observation of Tool-Use Gestures

Abstract

Language and action have been thought of as closely related. Comprehending words or phrases that are related to actions commonly activates motor and premotor areas, and this comprehension process interacts with action preparation and/or execution. However, it remains unclear whether comprehending action-related language interacts with action observation. In the current study, we examined whether the observation of tool-use gesture subjects to interaction with language. In an electroencephalography (EEG) study (n = 20), participants were presented with video clips of an actor performing tool-use (TU, e.g., hammering with a fist) and emblematic (EM, e.g., the thumb up sign for 'good job') gestures accompanied by either comprehensible German (G) or incomprehensible Russian sentences (R). Participants performed a semantic judging task, evaluating whether the co-speech gestures were object- or socially-related. Behavioral results from the semantic task showed faster response for the TU versus EM gestures only in the German condition. For EEG, we found that TU elicited beta power decrease (~ 20 Hz) when compared to EM gestures, however this effect was reduced when gestures were accompanied by German instead of Russian sentences. We concluded that the processing of action-related sentences might facilitate gesture observation, in the sense that motor simulation required for TU gestures, as indexed by reduced beta power, was modulated when accompanied by comprehensible German speech. Our results corroborate the functional role of the beta oscillations during perception of hand gestures, and provide novel evidence concerning language-motor interaction.

Gesture’s body orientation modulates the N400 during semantic integration of gesture and visual sentence

“…During gesture perception, our results showed that frontal gestures elicited a more pronounced beta band power decrease than lateral gestures. This finding not only supports the role of the beta band oscillations for perception of hand actions in general (Angelini et al, 2018;Avanzini et al, 2012;Järveläinen et al, 2004), but also corroborates the sensitivity of the beta band power to the observation of different types of gestures (He et al, 2018a;Quandt et al, 2012). Previous studies that directly tested perceptual differences of body orientation or view point of non-communicative hand actions (e.g., reaching) also showed a modulated decrease of beta power (and alpha power) for more allocentric view point (Angelini et al, 2018;Drew et al, 2015).…”

“…However importantly, in the current study, although we observed an alpha band power decrease for both frontal and lateral gestures, unlike those studies examining non-communicative hand actions (Angelini et al, 2018;Drew et al, 2015), the alpha power did not differ between frontal and lateral orientation in our study. This data pattern attests to the hypotheses that alpha and beta power are dissociable during action or gesture observation, in the sense that alpha power decrease might be more related to sensory/motor perception in general, whereas the beta band is more sensitive to finegrained motor-related features (Brinkman et al, 2014;He et al, 2018a;Salmelin et al, 1995).…”

Gesture's body orientation modulates the N400 for visual sentences primed by gestures

“…In addition, recent studies looking at the oscillatory domain of EEG also showed that both the alpha (8–13 Hz) and beta (14–26 Hz) bands may be functionally related to the integration between speech and communicative gesture (Biau, Torralba, Fuentemilla, de Diego Balaguer, & Soto‐Faraco, 2015; Drijvers, Özyürek, & Jensen, 2018a, 2018b; He et al, 2015; He, Steines, Sommer, et al, 2018). Interestingly, these frequency bands were also highly relevant to the perception of action and hand gestures in particular (Avanzini et al, 2012; He, Steines, Sammer, et al, 2018; Järveläinen, Schuermann, & Hari, 2004; Quandt, Marshall, Shipley, Beilock, & Goldin‐Meadow, 2012). It has been reported that gestures differing in social‐communicative intention (goal‐directed vs. non‐goal‐directed) (Hari et al, 1998; Järveläinen et al, 2004)—as well as the level of motor simulation (Quandt et al, 2012)—elicited differential level of alpha or beta band power decrease.…”

“…It has been reported that gestures differing in social‐communicative intention (goal‐directed vs. non‐goal‐directed) (Hari et al, 1998; Järveläinen et al, 2004)—as well as the level of motor simulation (Quandt et al, 2012)—elicited differential level of alpha or beta band power decrease. Additionally, these effects may be even modulated by accompanying auditory speech, as in the case of co‐speech gesture (He, Steines, Sammer, et al, 2018).…”

“…This line of research indicates that body orientation (or view‐point) has direct impact on hand action perception, which could be related to differential level of power decrease in the alpha and the beta bands. Of note, although alpha and beta oscillations as elicited during motor execution and imagery tasks are suggested to be functionally dissociable (Brinkman, Stolk, Dijkerman, de Lange, & Toni, 2014; Quandt et al, 2012; Stolk et al, 2019), that is, alpha power decrease might be related to sensory/motor perception in general, whereas the beta band is more sensitive to fine‐grained motor‐related features (Brinkman et al, 2014; He, Steines, Sammer, et al, 2018; Salmelin, Hámáaláinen, Kajola, & Hari, 1995). Importantly, prior studies investigated mostly non‐communicative and thus semantic‐free gestures, and participants from these studies usually only see a fraction of the actor/actress (typically one hand), but not the entire body.…”