Word familiarity predicts temporal asynchrony of hand gestures and speech.

Abstract

Seventeen Ss were videotaped as they provided narrative descriptions of 13 photographs. Judgments from 129 naive untrained Ss were used to isolate 60 speech-related gestures and their lexical affiliates (i.e., the accompanying word or phrase judged as related in meaning) from these 221 narratives. A computer-video interface measured each gesture, and a 3rd group of Ss rated word familiarity of each lexical affiliate. Multiple regression revealed that gesture onset preceded voice onset by an interval whose magnitude was inversely related to the lexical affiliate's rated familiarity. The lexical affiliate's familiarity was also inversely related to gesture duration. Results suggest that difficulty encountered during lexical access affects both gesture and speech. Familiarity's systematic relations with gesture-speech asynchrony and gesture duration make it unlikely that speech and gesture are produced independently by autonomous modules.When people talk they often move one or both hands. Casual observers presume that these arm and hand movements-the gestures that accompany spontaneous speechare related in meaning to what is being said. Yet, despite their ubiquity and the long-standing fascination such behaviors have held for scientists and humanists alike, the functional significance of gesturing for the speaker and the nature of the relation between gesture and speech are far from clear. If speech-related gestures are merely an atavistic vestige of the evolutionary process that resulted in speech (Condillac, 1756(Condillac, / 1974Hewes, 1973;Tylor, 1878) or are essentially random movements that serve to dissipate muscular tension (Dittman & Llewelyn, 1969) and coordinate the speech articulators (Hadar, Steiner, & Rose, 1984), then one should expect that gesture production is essentially independent of the cognitive processes associated with speech.' Such a view is consistent with Fodor's (1983Fodor's ( , 1985 notion of modularity and requires that speech and gesture be produced by autonomous modules. Alternatively, as some have speculated, gesture production and speech production may be interdependent: Gestures may help a speaker retrieve elusive words from memory (De-

Do conversational hand gestures communicate?

“…For example, all of the 60 gestures in our corpus were initiated before or simultaneously with the onset of their lexical affiliate; none were initiated after the onset of the lexical affiliate. The asynchrony ranges from 0 s to 3.8 s, with a mean of 0.75 s, and there is a reliable inverse relation between the magnitude of the asynchrony and the familiarity of the lexical affiliate: Gestures tend to be displayed more-or-less simultaneously with familiar lexical affiliates and to precede unfamiliar ones (Morrel-Samuels, 1989;Morrel-Samuels & Krauss, 1991).…”

“…From a total of 193 such gesture-lexical affiliate pairs, 60 were selected for use in this (and other) experiments. ' (See MorrelSamuels, 1989;Morrel-Samuels & Krauss, 1991, for a fuller account of the process by which the stimuli used in this experiment were selected.) The 60 segments were randomly partitioned into two sets of 30 and edited in random order onto separate videotapes.…”

Lexical gestures and lexical access: a process model

“…(Butterworth & Beattie, 1978;Morrel-Samuels & Krauss, 1992;Schegloff, 1984) . Morrel-Samuels and Krauss (1992) them to retrieve a particular word form, and enter this variable into the formula that determines the gesture's duration. We know of no data that supports either proposition, and neither strikes us as plausible.…”

“…As part of a pretest for a recent study, we selected 140 gestures that seem to us iconic from narratives on a variety of topics (e.g., directions to campus destinations, descriptions of the layouts of apartments, instructions on how to make a sandwich, etc.). Following the method used by Morrel-Samuels and Krauss (1992), subjects saw the video clips containing the gestures and heard the accompanying speech, and underlined the gestures' lexical affiliates on a transcript. Despite the fact that these gestures had been selected because we judged them to be iconic, subjects' agreement on their lexical affiliates averaged 43.86% (SD=23.34%).…”

“…This path (or its equivalent) is necessary in order to account for gesture termination. Since a gesture's duration is closely timed to articulation of its lexical affiliate (Morrel-Samuels & Krauss, 1992), a mechanism that informs the motor system when to terminate the gesture is required. Essentially, we are proposing that hearing the lexical affiliate being articulated serves as the signal to terminate the gesture.…”