Gesture, Thought, and Spatial Language

Emmorey, Karen; Casey, Shannon

doi:10.1007/978-94-015-9928-3_5

Cited by 24 publications

(14 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In a series of prior studies (Fussell et al, 2000;Fussell, Setlock, & Kraut, 2003;Kraut et al, 2003;Kraut, Gergle, & Fussell, 2002;Kraut, Miller, & Siegel, 1996; see also Bolt, 1980;Emmorey & Casey, 2001), we have found that conversational grounding during collaborative physical tasks tends to follow a predictable pattern: First, collaborators come to mutual agreement upon or "ground" the objects to be manipulated using one or more referential expressions. Next, they provide instructions for procedures to be performed on those objects.…”

Section: Conversational Grounding In Collaborative Physical Tasksmentioning

confidence: 66%

“…Prior research suggests that representational gestures are common in physical tasks. Tang and Leifer (1988), for example, found that approximately 35% of the gestures produced by a sample of design teams served to either convey or represent ideas (see also Bekker et al, 1995;Emmorey & Casey, 2001). Concrete representational gestures may facilitate conversational grounding in collaborative physical tasks by allowing speakers to communicate multiple pieces of information about the task simultaneously (Clark, 1996;McNeill, 1992).…”

Section: The Role Of Gesture In Conversational Groundingmentioning

confidence: 99%

See 1 more Smart Citation

Gestures Over Video Streams to Support Remote Collaboration on Physical Tasks

Fussell¹,

Setlock²,

Yang³

et al. 2004

Human–Computer Interaction

268

205

View full text Add to dashboard Cite

This article considers tools to support remote gesture in video systems being used to complete collaborative physical tasks-tasks in which two or more individuals work together manipulating three-dimensional objects in the real world. We first discuss the process of conversational grounding during collaborative physical tasks, particularly the role of two types of gestures in the grounding process: pointing gestures, which are used to refer to task objects and locations, and rep- HUMAN-COMPUTER INTERACTION, 2004, Volume 19, pp. 273-309 Copyright © 2004 resentational gestures, which are used to represent the form of task objects and the nature of actions to be used with those objects. We then consider ways in which both pointing and representational gestures can be instantiated in systems for remote collaboration on physical tasks. We present the results of two studies that use a "surrogate" approach to remote gesture, in which images are intended to express the meaning of gestures through visible embodiments, rather than direct views of the hands. In Study 1, we compare performance with a cursor-based 274 FUSSELL ET AL.

show abstract

Section: Conversational Grounding In Collaborative Physical Tasksmentioning

confidence: 66%

Section: The Role Of Gesture In Conversational Groundingmentioning

confidence: 99%

Gestures Over Video Streams to Support Remote Collaboration on Physical Tasks

Fussell¹,

Setlock²,

Yang³

et al. 2004

Human–Computer Interaction

268

205

View full text Add to dashboard Cite

show abstract

“…Readers, in turn, cannot signal that a part of the instruction has been understood, nor ask for clarification if needed [15], nor contribute to the communication by providing complementary information reflecting their collaboration in achieving a joint goal [17]. Furthermore, because both parties are in different places they cannot rely on visual cues such as gestures [18], which are central for communication about spatial configurations and relations [19].…”

Section: Challenges For Operating Instructionsmentioning

confidence: 99%

Efficiently Connecting Textual and Visual Information in Operating Instructions

Tenbrink

Maas

2015

IEEE Trans. Profess. Commun.

View full text Add to dashboard Cite

Research problem: Most technical documents rely on a combination of text and visuals to communicate their messages. To achieve the desired effect of improved processing and comprehension of operating instructions, the text must guide readers in a clear way to the relevant visual information in order to avoid ambiguity and misinterpretation, and to ensure that the reader optimally benefits from the available information. Research questions: (1) How are textual and visual information combined in operating instructions so as to guide the user's attention precisely toward the relevant parts of a household appliance? (2) In what ways can ambiguity arise, and what kinds of strategies can be used to avoid miscommunication and conceptual problems? Literature review: Operating manuals are usually procedural instructions that tell the user how to set up an appliance, how to operate and maintain it, and how to solve any problems. The vast majority of operating instructions are multimodal in that they include visuals of some kind. But previous research has shown that visual representations of instructions can be just as difficult to interpret as textual instructions -especially if the visuals come without textual elaboration. A combination of text and visuals provides opportunities for taking advantage of both by balancing the other mode's limitations. Methodology: Our exploratory qualitative study of a small set of operating instructions identifies and classifies types of references to visuals in the documents, enriched by interviews with the technical writers of these manuals. Besides showing patterns of reference types, we examine the potential ambiguity of some types of references along with strategies to avoid communication failure. Results and Conclusions:We identified 10 distinct ways in which links from text to visuals can be established. Line drawings were referred to more than visuals showing display content. The clarity of the link between text and visuals may be affected by the use of spatial expressions (such as left/right) that presuppose an underlying perspective, as well as by the use of semantic and functional information that is not readily accessible to laypersons. Ambiguity can be avoided by using clearly defined labels, specific perceptual information, and by repetition of visual elements in the text. Also, overspecification can be useful for enhancing communication. We conclude that technical writers need to attend to the links between text and visuals so as to support the users effectively, avoiding ambiguity by purposeful strategies. Because our study is qualitative and analytic, implications are limited by the scope of our study, and by the lack of empirical user comprehension studies.

show abstract

“…Quantitative analyses of the amount and type of information in speech and gestures during cartoon narration tasks (one of the traditional paradigms in the field of gesture research) indicate that around a quarter of the overall information is represented uniquely in gesture (Holler & Beattie, 2003a). This important semantic contribution of gestures is further highlighted when speech is ambiguous (Holler & Beattie, 2003b) or when information is difficult to express verbally (Bavelas, Kenwood, Johnson, & Phillips, 2002;Bergmann & Kopp, 2006;Emmorey & Casey, 2001). …”

Section: The Role Of Co-speech Gestures In Communicationmentioning

confidence: 99%