Simulating socially intelligent agents in semantic virtual environments

Abstract

The simulation of synthetic humans inhabiting virtual environments is a current research topic with a great number of behavioral problems to be tackled. Semantical virtual environments (SVEs) have recently been proposed not only to ease world modeling but also to enhance the agent–object and agent–agent interaction. Thus, we propose the use of ontologies to define the world’s knowledge base and to introduce semantic levels of detail that help the sensorization of complex scenes—containing lots of interactive objects. The object taxonomy also helps to create general and reusable operativity for autonomous characters—for example, liquids can be poured from containers such as bottles. On the other hand, we use the ontology to define social relations among agents within an artificial society. These relations must be taken into account in order to display socially acceptable decisions. Therefore, we have implemented a market-based social model that reaches coordination and sociability by means of task exchanges. This paper presents a multi-agent framework oriented to simulate socially intelligent characters in SVEs. The framework has been successfully tested in three-dimensional (3D) dynamic scenarios while simulating a virtual university bar, where groups of waiters and customers interact with both the objects in the scene and the other virtual agents, finally displaying complex social behaviors.

An Ontology for Semantic Modelling of Virtual World

“…This approach has been used to simulate the virtual bar of a university, where groups of waiters and customers interact with both the objects in the scene and the other virtual agents finally displaying complex social behaviors [16].…”

Recognizing and learning models of social exchange strategies for the regulation of social interactions in open agent societies

“…The works by Grimaldo et al [23][24][25][26][27] presented an interesting application of the Theory of Social Exchanges in the coordination of intelligent virtual agents and sociability in a virtual university bar scenario (in a 3D dynamic environment), modeled as a market-based social model, where groups of different types of waiters (e.g., coordinated, social, egalitarian) and customers (e.g., social, lazy) interact with both the objects in the scene and the other virtual agents. In [26], they presented a multi-modal agent decision making model, called MADeM, in order to provide virtual agents with socially acceptable decisions, coordinated social behaviors (e.g., task passing or planned meetings), based on the evaluation of the social exchanges.…”

Lessons from research on interaction with virtual environments

“…Requirements Cavazza (1998) Recognises a need for a knowledge representation layer for virtual environments Pioneering works Thalmann et al (1999) Informed environments based on a database model Urban life simulation Method for building virtual scenes with semantic information NOVAE Allongue, 1997, 1998) Semantic approach to increase VW interoperability Naval simulation (case study) (Soto and Allongue, 2002) Proposes the ontology paradigm Influence/reaction model for describing behaviours and reactions Doyle (2002) Introduces the concept of annotated environment (with structured representations of its contents and purpose) Agent architecture for interacting with the annotated environment Virtual objects with semantics Smart objects (Kallmann and Thalmann, 1999) Pre-programmed possible interactions Extended with action semantics expressed by rules, to reason about the consequences of actions (Abaci et al, 2005) Starfish (Badawi and Donikian, 2004) Synoptic objects Set of actions assigned to interactive surfaces Agents get the data from these interactive surfaces NiMMiT (Vanacken et al, 2007) Semantics incorporated in a diagram based notation intended to describe multimodal interaction Driving simulator (case study) Complete architectures with ontologies Chang et al (2005) Framework with an ontology-based cognitive middle layer between agent minds and the environment manages semantic concepts This layer also represents actions through causal rules Grimaldo et al (2006) Multi-agent framework composed of: ontologies, a semantic layer, planning based agents, and a 3D Engine Grimaldo et al (2008a) Ontology is used to define social relations among agents within an artificial society Virtual university bar (Grimaldo et al, 2008b) (Pellens et al, 2005) Necessity to let the domain expert participate in the specification of the VR application (Troyer et al, 2003) Virtual bowling game (Bille et al, 2004) and virtual shops (Troyer et al, 2007) Approach to design and develop a VR application where the domain expertise is used to generate it (Bille et al, 2004) Development process composed of three sequential steps: the specification step, the mapping step and the generation step (Pellens et al, 2005) method for building virtual scenes with associated semantic information as well as for the exploitation of such scenes. The three-dimensional scene provided by the designer is divided into two parts, one for visualization and another for database construction.…”

“…These relations must be taken into account in order to display socially acceptable decisions. This approach has been used to simulate a virtual university bar, where groups of waiters and customers interact with both the objects in the scene and the other virtual agents finally displaying complex social behaviours (Grimaldo et al, 2008b).…”