A mixed reality approach for interactively blending dynamic models with corresponding physical phenomena

Abstract

The design, visualization, manipulation, and implementation of models for computer simulation are key parts of the discipline. Models are constructed as a means to understand physical phenomena as state changes occur over time. One issue that arises is the need to correlate models and their components with the phenomena being modeled. For example, a part of an automotive engine needs to be placed into cognitive context with the diagrammatic icon that represents that part's function. A typical solution to this problem is to display a dynamic model of the engine in one window and the engine's CAD model in another. Users are expected to, on their own, mentally blend the dynamic model and the physical phenomenon into the same context. However, this contextualization is not trivial in many applications.Our approach expands upon this form of user interaction by specifying two ways in which dynamic models and the corresponding physical phenomena may be viewed, and experimented with, within the same human interaction space. We present a methodology and implementation of contextualization for diagram-based dynamic models using an anesthesia machine, and then follow up with a human study of its effects on spatial cognition.

Experimental evaluation of user performance on two-dimensional and three-dimensional perspective displays in discrete-event simulation

“…The dramatic advances in computer hardware and software, the revolution in the Internet and its multimedia front-end, and the World Wide Web have enabled the development and implementation of sophisticated user interfaces on diverse applications including DES as a decision support system [9,13,23,33,34]. While the benefits and costs of 3D visualization and VR in DES is still debatable, the application and implementation of these technologies are already well established in training simulations, entertainment industry, communication and media, and education with great successes [29,46]. For example, 3D games, 3D film productions and training simulation are very extensive [29,46].…”

“…While the benefits and costs of 3D visualization and VR in DES is still debatable, the application and implementation of these technologies are already well established in training simulations, entertainment industry, communication and media, and education with great successes [29,46]. For example, 3D games, 3D film productions and training simulation are very extensive [29,46]. In the area of training simulation, significant benefits have been recorded in flight simulation, military training and medical and surgical procedures [29,40,42].…”

“…For example, 3D games, 3D film productions and training simulation are very extensive [29,46]. In the area of training simulation, significant benefits have been recorded in flight simulation, military training and medical and surgical procedures [29,40,42].…”

“…On the other hand, such visual display may also be simplified to highlight the important features and it may show aspects that are not visible in the real system. Quarles et al [29] describe a mixed reality approach to learning how to use an anesthesia machine. An existing virtual simulation model showed the workings inside the machine when the controls are altered, including showing the flow of invisible gases using colors.…”

Hypermodelling: an integrated approach to dynamic system modelling

“…Figures 2 and 3 are examples of this, where manipulation of one model on the left results in an immediate set of visual cues on the rightmost model, and vice-versa. Technologies to support integrative modelling are numerous, from the web-browser mouse-based hyperlink page transitions to the use of mixed reality to co-locate models with reality (Quarles et al, 2009). Perhaps the easiest way to think of co-interaction, generally, is with media controllers: gestures on a hand-held controller or tablet immediately result in changes in the media presentation.…”