Designing context-aware multimodal virtual environments

Abstract

Despite of decades of research, creating intuitive and easy to learn interfaces for 3D virtual environments (VE) is still not obvious, requiring VE specialists to define, implement and evaluate solutions in an iterative way, often using lowlevel programming code. Moreover, quite frequently the interaction with the virtual environment may also vary dependent on the context in which it is applied, such as the available hardware setup, user experience, or the pose of the user (e.g. sitting or standing). Lacking other tools, the context-awareness of an application is usually implemented in an ad-hoc manner, using low-level programming, as well. This may result in code that is difficult and expensive to maintain. One possible approach to facilitate the process of creating these highly interactive user interfaces is by adopting a model-based user interface design. This lifts the creation of a user interface to a higher level allowing the designer to reason more in terms of high-level concepts, rather than writing programming code. In this paper, we adopt a model-based user interface design (MBUID) process for the creation of VEs, and explain how a context system using an Event-Condition-Action paradigm is added. We illustrate our approach by means of a case study.

Adaptation in virtual environments: conceptual framework and user models

“…Particular users may perform a selection task better when the selection technique is complemented by haptics (force feedback) as such that they can have more control on how the technique behaves. In our previous work [18,19], we have already experimented with this last type of adaptation. …”

Facilitating Adaptation in Virtual Environments Using a Context-Aware Model-Based Design Process

“…In this work, we classify adaptation into three types: (I) switching between interaction techniques, (II) adapting the interaction technique itself, and (III) enhancing the interaction technique with modalities. Through our case studies, we would like to investigate the first two types of adaptation as we already experimented with the third type of adaptation in our previous work [10,11].…”

“…In previous work, we presented an approach to design context-aware multimodal virtual environments using the VR-DeMo model-based user interface design process based on an 'Event-Condition-Action' paradigm [11]. The context system made it possible to switch between different modalities for a certain interaction technique based on the context.…”

“…The third type (enhancing the interaction technique with modalities) has been realized and investigated in prior work [10,11] and the second type (adapting the interaction technique itself) can be realized in the switching part of the context system without any influence on the model-based design process. To support the first type (switching between interaction techniques), traditionally the task level would be used because of the fact that an interaction technique is similar to a task.…”

“…Due to the fact that the different contexts (i.e. different selection techniques) are chosen by evaluating the same conditions in the 'CheckViewpoint'-task, we only use one state instead of one state per context situation as we showed in previous work [11]. We want to provide the user with different selection techniques depending on the viewpoint.…”