Instructions, intentions and expectations

Abstract

Based on an ongoing attempt to integrate Natural Language instructions with human figure animation, we demonstrate that agents' understanding and use of instructions can complement what they can derive from the environment in which they act. We focus on two attitudes that contribute to agents' behavior-their intentions and their expectations-and shown how Natural Language instructions contribute to such attitudes in ways that complement the environment. We also show that instructions can require more than one context of interpretation and thus that agents' understanding of instructions can evolve as their activity progresses. A significant consequence is that Natural Language understanding in the context of behavior cannot simply be treated as "front end" processing, but rather must be integrated more deeply into the processes that guide an agent's behavior and respond to its perceptions.

Visualization of Patient Behavior from Natural Language Recommendations

“…There has been significant interest in the use of natural language (NL) input to generate 3D scenes and animations, ranging from automatic scene creation [4] to visualization of complex procedural instructions [14]. Text-to-Scene visualization systems aim primarily at using NL for image and graphics generation [2] or at rendering spatial information conveyed in a text (such as road accidents description [8]).…”

“…Other systems have aimed at generating 3D animations from NL, driving virtual agent behavior from text. The AnimNL system [14] was dedicated to the visualization of procedural instructions using a virtual agent with the aim of ultimately using instructions to control the behavior of virtual humans. SceneMaker [6] generated animations from film scripts, including some staging elements; this followed work on virtual agent animation from NL [9].…”

Agents’ Interaction in Virtual Storytelling

“…Planning is the most generic description of an embodied artificial actor's behaviour, whether from a narrative perspective [12] [25] or from a generic "cognitive" perspective [24]. As we are mainly concerned here with the narrative aspects, we should describe how planning techniques can support artificial actors' behaviours in relation with storytelling.…”

Pragmatic action

“…Mannes and Kintsch (1991) present a model of instruction usage where the instructions are presented before the agent begins to interact with the environment, and consist of a goal and a description of the current state of the world. Badler et al (1990) and Webber et al (1995) describe animated simulations which are driven by natural language instructions. The work of Ross (1989) presents psychological studies that show the importance of memory in the usage of instructional material; LeFevre and Dixon (1986) show how instructions are more easily applied to future problems when presented as specific examples; Martin (1988) uses instruction as a basis of knowledge acquisition; and Kieras and Bovair (1986) discuss a psychological model of instruction usage for made-up devices where the subjects first read the instructions in their entirety and then proceduralize the 'understanding.…”