Beat Gestures for Comprehension and Recall: Differential Effects of Language Learners and Native Listeners

Abstract

Previous work has shown how native listeners benefit from observing iconic gestures during speech comprehension tasks of both degraded and non-degraded speech. By contrast, effects of the use of gestures in non-native listener populations are less clear and studies have mostly involved iconic gestures. The current study aims to complement these findings by testing the potential beneficial effects of beat gestures (non-referential gestures which are often used for information-and discourse marking) on language recall and discourse comprehension using a narrative-drawing task carried out by native and non-native listeners. Using a within-subject design, 51 French intermediate learners of English participated in a narrative-drawing task. Each participant was assigned 8 videos to watch, where a native speaker describes the events of a short comic strip. Videos were presented in random order, in four conditions: in Native listening conditions with frequent, naturally-modeled beat gestures, in Native listening conditions without any gesture, in Non-native listening conditions with frequent, naturally-modeled beat gestures, and in Non-native listening conditions without any gesture. Participants watched each video twice and then immediately recreated the comic strip through their own drawings. Participants' drawings were then evaluated for discourse comprehension (via their ability to convey the main goals of the narrative through their drawings) and recall (via the number of gesturally-marked elements in the narration that were included in their drawings). Results showed that for native listeners, beat gestures had no significant effect on either recall or comprehension. In non-native speech, however, beat gestures led to significantly lower comprehension and recall scores. These results suggest that frequent, naturally-modeled beat gestures in longer discourses may increase cognitive load for language learners, resulting in negative effects on both memory and language understanding. These findings add to the growing body of literature that suggests that gesture benefits are not a "one-sizefits-all" solution, but rather may be contingent on factors such as language proficiency and gesture rate, particularly in that whenever beat gestures are repeatedly used in discourse, they inherently lose their saliency as markers of important information.

Toddler-directed and adult-directed gesture frequency in monolingual and bilingual caregivers

“…Our analyses of beat gestures revealed that bilinguals also used more beat gestures than monolinguals. This is consistent with the idea that bilinguals increased their gesture frequency because of higher cognitive demands, in particular given that beat gestures have been suggested to boost attention and facilitate semantic integration (e.g., Biau & Soto-Faraco, 2013;Dimitrova, Chu, Wang, Özyürek, & Hagoort, 2016; but also see discussion by Rohrer, Delais-Roussarie, & Prieto, 2020). In addition, visual inspection of the data suggests that the increased rate of beat gestures in bilingual caregivers is most pronounced in the adult-directed setting.…”

The Value of Non-Referential Gestures: A Systematic Review of Their Cognitive and Linguistic Effects in Children’s Language Development

“…Even though the empirical evidence of the value of non-referential gestures is not yet as strong as the evidence in favor of referential gestures, it is clear that there are sufficient grounds to claim that non-referential gestures play an important role in boosting children's learning and language development. It is of interest to note that the mixed findings obtained in the developmental literature resemble the contradictory patterns reported by studies assessing the role of nonreferential gestures in adult speech processing (see [85] for a review). While some research has shown that non-referential gestures positively affect adults' ability to recall information [20,86,87], this has not been true of other studies [77,79,88] (see also [85]).…”

“…It is of interest to note that the mixed findings obtained in the developmental literature resemble the contradictory patterns reported by studies assessing the role of nonreferential gestures in adult speech processing (see [85] for a review). While some research has shown that non-referential gestures positively affect adults' ability to recall information [20,86,87], this has not been true of other studies [77,79,88] (see also [85]). According to [85], a potential reason for the lack of the beneficial effect of non-referential gestures reported in some studies could be related to the stimuli used, as positive results have generally been shown when non-referential gestures are used in pragmatically natural and restricted contexts, such as for marking contrastively focused information.…”

“…While some research has shown that non-referential gestures positively affect adults' ability to recall information [20,86,87], this has not been true of other studies [77,79,88] (see also [85]). According to [85], a potential reason for the lack of the beneficial effect of non-referential gestures reported in some studies could be related to the stimuli used, as positive results have generally been shown when non-referential gestures are used in pragmatically natural and restricted contexts, such as for marking contrastively focused information. Similar to the child experiments reviewed here, it is clear that experiments involving non-referential gestures that use ecologically valid tasks and materials have reported positive results.…”

More than words: word predictability, prosody, gesture and mouth movements in natural language comprehension

“…Previous studies failed to find the same N400 effects of beat gestures [43,44]. However, these studies used artificial beat gestures (one single stroke per sentence), which are different from naturally occurring beat gestures (see also [74]). Alternatively, the lack of any meaningful gestures in previous studies could have discouraged listeners from paying attention to gestures.…”