A Formal Description of Multimodal Interaction Techniques for Immersive Virtual Reality Applications

Navarre, David; Palanque, Philippe; Bastide, Rémi; Schyn, Amélie; Winckler, Marco; Nedel, Luciana; Freitas, Carla Maria Dal Sasso

doi:10.1007/11555261_17

Cited by 47 publications

(40 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Previous modelling approaches based on Petri nets include the use of: Hybrid high-level Nets (HyNets) [14], Flownets [19], Interactive Cooperative Objects (ICO) [15], and Coloured Petri Nets (CPN) [11].…”

Section: Related Literature and Goalsmentioning

confidence: 99%

The APEX Framework: Prototyping of Ubiquitous Environments Based on Petri Nets

Silva

Ribeiro

Fernandes

et al. 2010

Human-Centred Software Engineering

View full text Add to dashboard Cite

Abstract. The user experience of ubiquitous environments is a determining factor in their success. The characteristics of such systems must be explored as early as possible to anticipate potential user problems, and to reduce the cost of redesign. However, the development of early prototypes to be evaluated in the target environment can be disruptive to the ongoing system and therefore unacceptable. This paper reports on an ongoing effort to explore how model-based rapid prototyping of ubiquitous environments might be used to avoid actual deployment while still enabling users to interact with a representation of the system. The paper describes APEX, a framework that brings together an existing 3D Application Server with CPN Tools. APEX-based prototypes enable users to navigate a virtual world simulation of the envisaged ubiquitous environment. The APEX architecture and the proposed CPN-based modelling approach are described. An example illustrates their use.

show abstract

Section: Related Literature and Goalsmentioning

confidence: 99%

The APEX Framework: Prototyping of Ubiquitous Environments Based on Petri Nets

Silva

Ribeiro

Fernandes

et al. 2010

Human-Centred Software Engineering

View full text Add to dashboard Cite

show abstract

“…Another way to abstract the device from the application is to use an 'Interaction Description Model'. Several solutions exist, each with their own features or design: Interaction Object Graphs [4], InTml [15], ICO [23].…”

Section: Multimodal User Interfacesmentioning

confidence: 99%

Designing context-aware multimodal virtual environments

Vanacken

Boeck

Raymaekers

et al. 2008

Proceedings of the 10th International Conference on Multimodal Interfaces

View full text Add to dashboard Cite

Despite of decades of research, creating intuitive and easy to learn interfaces for 3D virtual environments (VE) is still not obvious, requiring VE specialists to define, implement and evaluate solutions in an iterative way, often using lowlevel programming code. Moreover, quite frequently the interaction with the virtual environment may also vary dependent on the context in which it is applied, such as the available hardware setup, user experience, or the pose of the user (e.g. sitting or standing). Lacking other tools, the context-awareness of an application is usually implemented in an ad-hoc manner, using low-level programming, as well. This may result in code that is difficult and expensive to maintain. One possible approach to facilitate the process of creating these highly interactive user interfaces is by adopting a model-based user interface design. This lifts the creation of a user interface to a higher level allowing the designer to reason more in terms of high-level concepts, rather than writing programming code. In this paper, we adopt a model-based user interface design (MBUID) process for the creation of VEs, and explain how a context system using an Event-Condition-Action paradigm is added. We illustrate our approach by means of a case study.

show abstract

“…Other notations, are extended or optimised to support user interaction in particular. Examples are InTml [7], ICon [8] and ICO [9][10].…”

Section: Describing User Interactionmentioning

confidence: 99%

Comparing NiMMiT and Data-Driven Notations for Describing Multimodal Interaction

Boeck

Raymaekers

Coninx

2007

Task Models and Diagrams for Users Interface Design

View full text Add to dashboard Cite

Abstract. In the past few years, multimodal interaction is gaining importance in virtual environments. Although multimodality makes interaction with the environment more intuitive and natural for the user, the development cycle of such an environment is often a long and expensive process. In our overall field of research, we investigate how model-based design can help shorten this process by designing the application with the use of high-level diagrams. In this scope, we developed 'NiMMiT', a graphical notation especially suitable for expressing multimodal user interaction. We have already experienced the benefits of NiMMiT in several in-house applications, and are currently assessing the value of NiMMiT with respect to existing notations. In this paper we report on our comparison of NiMMiT against some well known data-driven modeling notations.

show abstract

A Formal Description of Multimodal Interaction Techniques for Immersive Virtual Reality Applications

Cited by 47 publications

References 17 publications

The APEX Framework: Prototyping of Ubiquitous Environments Based on Petri Nets

The APEX Framework: Prototyping of Ubiquitous Environments Based on Petri Nets

Designing context-aware multimodal virtual environments

Comparing NiMMiT and Data-Driven Notations for Describing Multimodal Interaction

Contact Info

Product

Resources

About