Task-Level Object Grasping for Simulated Agents

Abstract

Simulating a human figure performing a manual task requires that the agent interact with objects in the environment in a realistic manner. Graphic or programming interfaces to control human figure animation, however, do not allow the animator to instruct the system with concise "high-level" commands. Instructions coming from a high-level planner cannot be directly given to a synthetic agent because they do not specify such details as which end-effector to use or where on the object to grasp. Because current animation systems require joint angle displacement descriptions of motion -even for motions that incorporate upwards of 15 joints -an efficient connection between high-level specifications and low-level hand joint motion is required. In this paper we describe a system that directs task-level, general-purpose, object grasping for a simulated human agent. The Object-Specific Reasoner (OSR) is a reasoning module that uses knowledge of the object of the underspecified action to generate values for missing parameters. The Grasp Behavior manages simultaneous motions of the joints in the hand, wrist, and arm, and provides a programmer with a high-level description of the desired action. When composed hierarchically, the OSR and the Grasp behavior interpret task-level commands and direct specific motions to the animation system. These modules are implemented as part of the Jack system at the University of Pennsylvania. Comments AbstractSimulating a human gure performing a task requires that the agent i n teract with objects in the environment in a realistic manner. In this paper we describe a system which directs task-level, general-purpose, object grasping for a simulated human agent.The Object Speci c Reasoner (OSR) generates parameters for underspeci ed tasklevel instructions such a s (pickup jack hammer). The Grasp behavior manages simultaneous motions of the joints in the hand, wrist and arm. When composed hierarchically, the OSR and the Grasp behavior interpret task-level commands to the animation system. These modules are implemented as part of the Jack project at the University o f P ennsylvania.

An Animation System for Imitation of Object Grasping in Virtual Reality

“…In an extension of this algorithm [5] they were also able to bias the inverse kinematics algorithm towards natural-looking poses using a motion database. In contrast to this, the work of Douville et al [6] describes a behavioral approach to motion synthesis. Here, different objects are grasped using specialized grasping behaviors and an object specific reasoner.…”

Animated agents for procedural training in virtual reality: Perception, cognition, and motor control

“…Geib et al (1994) developed an agent that integrates a highlevel planner with a search planner for nding objects and another planner for manipulating objects. The ability to realistically grasp objects in a taskdependent manner, as described by Douville et al (1996), would be an especially valuable extension to Steve. Cassell et al (1994) developed an agent that integrates speech, gestures, and facial expressions in the context of a dialog.…”

Dynamically altering agent behaviors using natural language instructions

“…An object type lists the actions that can be performed on it and what state changes they cause [11,20]. Among other fields, a list of grasp sites and directions are defined with respect to the object.…”