We describe an implemented system which automatically generates and animates conversations between multiple human-like agents with appropriate and synchronized speech, intonation, facial expressions, and hand gestures. Conversations are created by a dialogue planner that produces the text as well as the intonation of the utterances. The speaker/listener relationship, the text, and the intonation in turn drive facial expressions, lip motions, eye gaze, head motion, and arm gesture generators. Coordinated arm, wrist, and hand motions are invoked to create semantically meaningful gestures. Throughout, we will use examples from an actual synthesized, fully animated conversation.
Simulating a human figure performing a manual task requires that the agent interact with objects in the environment in a realistic manner. Graphic or programming interfaces to control human figure animation, however, do not allow the animator to instruct the system with concise "high-level" commands. Instructions coming from a high-level planner cannot be directly given to a synthetic agent because they do not specify such details as which end-effector to use or where on the object to grasp. Because current animation systems require joint angle displacement descriptions of motion -even for motions that incorporate upwards of 15 joints -an efficient connection between high-level specifications and low-level hand joint motion is required. In this paper we describe a system that directs task-level, general-purpose, object grasping for a simulated human agent. The Object-Specific Reasoner (OSR) is a reasoning module that uses knowledge of the object of the underspecified action to generate values for missing parameters. The Grasp Behavior manages simultaneous motions of the joints in the hand, wrist, and arm, and provides a programmer with a high-level description of the desired action. When composed hierarchically, the OSR and the Grasp behavior interpret task-level commands and direct specific motions to the animation system. These modules are implemented as part of the Jack system at the University of Pennsylvania. Comments AbstractSimulating a human gure performing a task requires that the agent i n teract with objects in the environment in a realistic manner. In this paper we describe a system which directs task-level, general-purpose, object grasping for a simulated human agent.The Object Speci c Reasoner (OSR) generates parameters for underspeci ed tasklevel instructions such a s (pickup jack hammer). The Grasp behavior manages simultaneous motions of the joints in the hand, wrist and arm. When composed hierarchically, the OSR and the Grasp behavior interpret task-level commands to the animation system. These modules are implemented as part of the Jack project at the University o f P ennsylvania.
Until now theories of the gesture-speech relationship have been difficult to evaluate because of their descriptive basis. In this paper we provide a tool for investigating the relationship between speech and gesture: a system that generates speech, intonation, and gesture using two copies of an identical program that have different knowledge of the world and must cooperate to accomplish a goal. The output of the dialogue generation is fed into a three-dimensional interactive animated model-two graphic figures on a computer screen who gesture according to the rules given to the system. The advantage of computer modeling in this domain is that it forces us to come up with predictive theories of the gesture-speech relationship. A felicitous outcome is a working system to realize autonomous animated conversational agents, for virtual reality and other purposes.
We describe an implementation of the simulation of human grasping for manufacturing tasks using a realistic human figure with a high-level behavioral control interface. Grasping simulations can then inform the manufacturer as to requirements on human grasping in manufacturing environments. We further describe the integration of a reach planner with grasping to provide simulation of reaching and grasping tasks. The simulation of human grasping proceeds from work in cognitive science, psychology, and robotics, in which human grasping is described as a primarily tactile activity. Information about target geometry is derived from contact with the object to be grasped, and the simulation is driven by this collision information. Reaching and grasping have been integrated to provide more automated control of the simulation of grasping. A motion planning approach controls human arm movement: given a position in 3D space, the reaching algorithm automatically computes a collision-free and strength-feasible motion sequence to move the hand to the desired position. If a tool is held in the hand, the collision of the tool with the environment can also be avoided.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.