Effects of Encouraging the Use of Gestures on Speech

Cravotta, Alice; Busà, María Grazia; Prieto, Pilar

doi:10.1044/2019_jslhr-s-18-0493

Cited by 27 publications

(26 citation statements)

References 64 publications

(68 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, recent work suggests that gesturespeech physics does generalize to fluent speech. For example, Cravotta and colleagues 47 found that encouraging participants to gesture during cartoon narration versus giving no instructions led to a 22-Hz increase in observation of max F0 and to greater F0 ranges of speech and intensity. Furthermore, computational modelers have reported interesting successes in synthesizing gesture kinematics on the basis of speech acoustics alone, 48,49 indicating that information about body movements inhabits the speech signal (see also Refs.…”

Section: Gesture-speech Physicsmentioning

confidence: 99%

Gesture–speech physics in fluent speech and rhythmic upper limb movements

Pouw

Jonge-Hoekstra

Harrison

et al. 2020

Annals of the New York Academy of Sciences

View full text Add to dashboard Cite

It is commonly understood that hand gesture and speech coordination in humans is culturally and cognitively acquired, rather than having a biological basis. Recently, however, the biomechanical physical coupling of arm movements to speech vocalization has been studied in steady-state vocalization and monosyllabic utterances, where forces produced during gesturing are transferred onto the tensioned body, leading to changes in respiratory-related activity and thereby affecting vocalization F0 and intensity. In the current experiment (n = 37), we extend this previous line of work to show that gesture-speech physics also impacts fluent speech. Compared with nonmovement, participants who are producing fluent self-formulated speech while rhythmically moving their limbs demonstrate heightened F0 and amplitude envelope, and such effects are more pronounced for higher-impulse arm versus lowerimpulse wrist movement. We replicate that acoustic peaks arise especially during moments of peak impulse (i.e., the beat) of the movement, namely around deceleration phases of the movement. Finally, higher deceleration rates of higher-mass arm movements were related to higher peaks in acoustics. These results confirm a role for physical impulses of gesture affecting the speech system. We discuss the implications of gesture-speech physics for understanding of the emergence of communicative gesture, both ontogenetically and phylogenetically.

show abstract

Section: Gesture-speech Physicsmentioning

confidence: 99%

Gesture–speech physics in fluent speech and rhythmic upper limb movements

Pouw

Jonge-Hoekstra

Harrison

et al. 2020

Annals of the New York Academy of Sciences

View full text Add to dashboard Cite

show abstract

“…However, recent work suggests that gesture-speech physics does generalize to fluent speech. For example, Cravotta and colleagues 47 found that encouraging participants to gesture during cartoon narration versus giving no instructions lead to 22Hz increase in observation of max F0 and to greater F0 ranges of speech and intensity. Furthermore, computational modelers have reported on interesting successes in synthesizing gesture kinematics based on speech acoustics alone 48,49 , indicating that information about body movements inhabits the speech signal (see also 50,51 ).…”

Section: Gesture-speech Physicsmentioning

confidence: 99%

Gesture-speech physics in fluent speech and rhythmic upper limb movements

Pouw¹,

Jonge-Hoekstra²,

Harrison³

et al. 2020

Preprint

View full text Add to dashboard Cite

Communicative hand gestures are often coordinated with prosodic aspects of speech, and salient moments of gestural movement (e.g., quick changes in speed) often co-occur with salient moments in speech (e.g., near peaks in fundamental frequency and intensity). A common understanding is that such gesture and speech coordination is culturally and cognitively acquired, rather than having a biological basis. Recently, however, the biomechanical physical coupling of arm movements to speech movements has been identified as a potentially important factor in understanding the emergence of gesture-speech coordination. Specifically, in the case of steady-state vocalization and mono-syllable utterances, forces produced during gesturing are transferred onto the tensioned body, leading to changes in respiratory-related activity and thereby affecting vocalization F0 and intensity. In the current experiment (N = 37), we extend this previous line of work to show that gesture-speech physics impacts fluent speech, too. Compared with non-movement, participants who are producing fluent self-formulated speech, while rhythmically moving their limbs, demonstrate heightened F0 and amplitude envelope, and such effects are more pronounced for higher-impulse arm versus lower-impulse wrist movement. We replicate that acoustic peaks arise especially during moments of peak-impulse (i.e., the beat) of the movement, namely around deceleration phases of the movement. Finally, higher deceleration rates of higher-mass arm movements were related to higher peaks in acoustics. These results confirm a role for physical-impulses of gesture affecting the speech system. We discuss the implications of gesture-speech physics for understanding of the emergence of communicative gesture, both ontogenetically and phylogenetically.

show abstract

“…A subset of 8 comic strip illustrations was chosen from the Simon's Cat comic series that were used in Cravotta et al (2019). These 8 comic strips were chosen based on the ease of translating the illustrations to a short narrative that could be understood by low-intermediate level language learners.…”

Section: Materials Stimuli Creation For the Drawing Taskmentioning

confidence: 99%

Beat Gestures for Comprehension and Recall: Differential Effects of Language Learners and Native Listeners

2020

Self Cite

View full text Add to dashboard Cite

Previous work has shown how native listeners benefit from observing iconic gestures during speech comprehension tasks of both degraded and non-degraded speech. By contrast, effects of the use of gestures in non-native listener populations are less clear and studies have mostly involved iconic gestures. The current study aims to complement these findings by testing the potential beneficial effects of beat gestures (non-referential gestures which are often used for information-and discourse marking) on language recall and discourse comprehension using a narrative-drawing task carried out by native and non-native listeners. Using a within-subject design, 51 French intermediate learners of English participated in a narrative-drawing task. Each participant was assigned 8 videos to watch, where a native speaker describes the events of a short comic strip. Videos were presented in random order, in four conditions: in Native listening conditions with frequent, naturally-modeled beat gestures, in Native listening conditions without any gesture, in Non-native listening conditions with frequent, naturally-modeled beat gestures, and in Non-native listening conditions without any gesture. Participants watched each video twice and then immediately recreated the comic strip through their own drawings. Participants' drawings were then evaluated for discourse comprehension (via their ability to convey the main goals of the narrative through their drawings) and recall (via the number of gesturally-marked elements in the narration that were included in their drawings). Results showed that for native listeners, beat gestures had no significant effect on either recall or comprehension. In non-native speech, however, beat gestures led to significantly lower comprehension and recall scores. These results suggest that frequent, naturally-modeled beat gestures in longer discourses may increase cognitive load for language learners, resulting in negative effects on both memory and language understanding. These findings add to the growing body of literature that suggests that gesture benefits are not a "one-sizefits-all" solution, but rather may be contingent on factors such as language proficiency and gesture rate, particularly in that whenever beat gestures are repeatedly used in discourse, they inherently lose their saliency as markers of important information.

show abstract

Effects of Encouraging the Use of Gestures on Speech

Cited by 27 publications

References 64 publications

Gesture–speech physics in fluent speech and rhythmic upper limb movements

Gesture–speech physics in fluent speech and rhythmic upper limb movements

Gesture-speech physics in fluent speech and rhythmic upper limb movements

Beat Gestures for Comprehension and Recall: Differential Effects of Language Learners and Native Listeners

Contact Info

Product

Resources

About