A Framework for the Testing and Validation of Simulated Environments in Experimentation and Training

Abstract

New computer technologies, like virtual reality (VR), have created opportunities to study human behavior and train skills in novel ways. VR holds significant promise for maximizing the efficiency and effectiveness of skill learning in a variety of settings (e.g., sport, medicine, safety-critical industries) through immersive learning and augmentation of existing training methods. In many cases the adoption of VR for training has, however, preceded rigorous testing and validation of the simulation tool. In order for VR to be implemented successfully for both training and psychological experimentation it is necessary to first establish whether the simulation captures fundamental features of the real task and environment, and elicits realistic behaviors. Unfortunately evaluation of VR environments too often confuses presentation and function, and relies on superficial visual features that are not the key determinants of successful training outcomes. Therefore evidence-based methods of establishing the fidelity and validity of VR environments are required. To this end, we outline a taxonomy of the subtypes of fidelity and validity, and propose a variety of practical methods for testing and validating VR training simulations. Ultimately, a successful VR environment is one that enables transfer of learning to the real-world. We propose that key elements of psychological, affective and ergonomic fidelity, are the real determinants of successful transfer. By adopting an evidence-based approach to VR simulation design and testing it is possible to develop valid environments that allow the potential of VR training to be maximized.

Effects of body visualization on performance in head-mounted display virtual reality

“…The tasks were chosen because they differ in the needed abilities to complete them and we did not want to specialize in just one. VR can be seen as a useful tool to study human behavior and to examine the impact of body visualization on sports performances since the visual system plays a decisive role during experiences in virtual environments [9]. With less visible properties of the own virtual body, we assume that decreased embodiment leads to significantly worse performances in sport motoric tasks.…”

“…Fidelity, that means a simulation, which recreates the real-world system, leads to the users' feel of presence in VR [9]. The authors also described the confusion about terms like fidelity, validity, immersion, and presence and gave a good overview of how to use them in the right context.…”

“…They also presented different types of fidelity for example the physical one, which refers to the level of realism provided by the simulation. They stated that physical fidelity is important to elicit a feeling of the presence of the participant [ 9 ]. An easy and valid approach to assess the strength of the feeling of ‘being there’ is to use questionnaires (for example see [ 10 , 11 ]), which used Likert scales to gauge the participants’ impression [ 12 ].…”

Performance in complex life situations: effects of age, cognition, and walking speed in virtual versus real life environments

“…To achieve ecological validity of testing, sufficient levels of biomechanical fidelity are necessary. Biomechanical (motor) fidelity is defined as movements that are made with similar temporal and spatial parameters, muscle activation patterns, and joint forces in both the VE and a RW [ 46 ]. An additional source of demonstrating fidelity can be obtained from surrogate measures of motor performance including energy expenditure and neural activity [ 47 ].…”

Virtual Terroir and the Premium Coffee Experience

“…At a high level, VR can be a crucial tool for companies to more effectively convey the meaning and personality of their brands and help consumers grasp issues related to their products (Hollebeek et al, 2020 ). Moreover, companies can use VR to develop consumers' emotional connections with brands and products (Harris et al, 2020 ). As Chirico and Gaggioli ( 2019 ) found, the emotions and sense of presence elicited by immersive 360-degree videos are comparable to those elicited by real-life environments.…”