Semantic Virtual Environments with Adaptive Multimodal Interfaces

Abstract

We present a system for real-time configuration of multimodal interfaces to Virtual Environments (VE). The flexibility of our tool is supported by a semantics-based representation of VEs. Semantic descriptors are used to define interaction devices and virtual entities under control. We use portable (XML) descriptors to define the I/O channels of a variety of interaction devices. Semantic description of virtual objects turns them into reactive entities with whom the user can communicate in multiple ways. This article gives details on the semantics-based representation and presents some examples of multimodal interfaces created with our system, including gestures-based and PDA-based interfaces, amongst others.

Semantic framework for interactive animation generation and its application in virtual shadow play performance

“…Unlike traditional virtual environments design, semantic virtual environments maintain richer semantic information, in which the ontological model provides an abstract and semantic description that is adequate for computer processing (Otto 2005). Different aspects of the conceptual representation of the virtual world are modelled by ontologies as high-level and semantic description, including the environment structures, entities' behaviour and domain knowledge (Gutierrez et al 2005). The concept of annotated environment together with structured representations of its contents and purposes was firstly proposed by Thalmann et al (1999).…”

“…7, once the connection between the leap motion controller and our system is established, the deep images of user's hand will be achieved. The sensor data provided by leap motion controller contain a diversity of information about hands and pointables [such as fingers or finger-like tools defined by Leap Motion Co. (Gutierrez et al 2005)], in the virtual scene, which is updated by frame and can be represented as follows:…”

Ontology-Based Interactive Animation/Game Generation for Chinese Shadow Play Preservation

“…Unlike traditional virtual environments design, semantic virtual environments maintain richer semantic information, in which the ontological model provides an abstract and semantic description that is adequate for computer processing [22]. Different aspects of the conceptual representation of the virtual world are modelled by ontologies as high level and semantic description, including the environment structures, entities' behaviour and domain knowledge [23]. The concept of annotated environment together with structured representations of its contents and purposes is firstly proposed by Thalmann et al in [24].…”

An Ontology for Semantic Modelling of Virtual World

“…We can divide the use of ontologies in the domain of Virtual Environments into three uses; the first use: Ontologies in the context of Virtual Environments [21,22], the second use: Ontology for interactive Virtual Environments [23,24], the third use: Ontology for Virtual Humans [19,25].…”