Immersive visual scripting based on VR software design patterns for experiential training

Abstract

Virtual reality (VR) has re-emerged as a low-cost, highly accessible consumer product, and training on simulators is rapidly becoming standard in many industrial sectors. However, the available systems are either focusing on gaming context, featuring limited capabilities or they support only content creation of virtual environments without any rapid prototyping and modification. In this project, we propose a code-free, visual scripting platform to replicate gamified training scenarios through rapid prototyping and VR software design patterns. We implemented and compared two authoring tools: a) visual scripting and b) VR editor for the rapid reconstruction of VR training scenarios. Our visual scripting module is capable of generating training applications utilizing a node-based scripting system, whereas the VR editor gives user/developer the ability to customize and populate new VR training scenarios directly from the virtual environment. We also introduce action prototypes, a new software design pattern suitable to replicate behavioral tasks for VR experiences. In addition, we present the training scenegraph architecture as the main model to represent training scenarios on a modular, dynamic and highly adaptive acyclic graph based on a structured educational curriculum. Finally, a user-based evaluation of the proposed solution indicated that users-regardless of their programming expertise-can effectively use the tools to create and modify training scenarios in VR.

Virtual Reality Medical Training for COVID-19 Swab Testing and Proper Handling of Personal Protective Equipment: Development and Usability

“…The Action coding is minimal due to the (Kätsyri et al, 2015). Right: The same phenomenon is observed when comparing the user experience vs the interactivity (level of detail and automation of every action) (Zikas et al, 2020 programming design patterns utilized in MAGES. One of the most useful features of MAGES SDK is the Action classification mechanism.…”

“…FIGURE 7 | Left: The Uncanny Valley (UV) effect: affinity (empathy, likeness, attractiveness) vs human-likeness from(Kätsyri et al, 2015). Right: The same phenomenon is observed when comparing the user experience vs the interactivity (level of detail and automation of every action)(Zikas et al, 2020).…”

“…Given the storyboard, the next step is to translate this visual information into a training scenegraph (Zikas et al, 2020). This acyclic graph holds the structure of our training scenario describing all the steps and possible branching due to deviations from the expected path (see Figure 8).…”

“…Other manifestations of the Uncanny Valley such as UV in haptics (Berger et al, 2018) started to gain attention proving that If the fidelity of the haptic sensation increases but is not rendered in accordance with other sensory feedback the subjective impression of realism actually gets worse, not better. The phenomenon of UV has been proved to exist in medical VR training simulations too (Zikas et al, 2020).…”

Covid-19 - VR Strikes Back: innovative medical VR training

“…According to the uncanny valley of interactivity for VR training phenomenon [Zikas et al 2019], the user experience augments as the interactivity of the VR application grows but only up to a point -called the "uncanny valley" (see Figure 2). The design of CVRSB was done bearing in mind this point at which the application is too advanced and complex for the user to understand and perform the tasks with ease.…”