XRSISE: An XR Training System for Interactive Simulation and Ergonomics Assessment

Abstract

The use of virtual reality (VR) techniques for industrial training provides a safe and cost effective solution that contributes to increased engagement and knowledge retention levels. However, the process of experiential learning in a virtual world without biophysical constraints might contribute to muscle strain and discomfort, if ergonomic risk factors are not considered in advance. Under this scope, we have developed a digital platform which employs extended reality (XR) technologies for the creation and delivery of industrial training programs, by taking into account the users and workplace specificities through the adaptation of the 3D virtual world to the real environment. Our conceptual framework is composed of several inter-related modules: 1) the XR tutorial creation module, for automatic recognition of the sequence of actions composing a complex scenario while this is demonstrated by the educator in VR, 2) the XR tutorial execution module, for the delivery of visually guided and personalized XR training experiences, 3) the digital human model (DHM) based simulation module for creation and demonstration of job task simulations avoiding the need of an actual user and 4) the biophysics assessment module for ergonomics analysis given the input received from the other modules. Three-dimensional reconstruction and aligned projection of the objects situated in the real scene facilitated the imposition of inherent physical constraints, thereby allowed to seamlessly blend the virtual with the real world without losing the sense of presence.

Machine Learning Approaches for 3D Motion Synthesis and Musculoskeletal Dynamics Estimation: A Survey

“…In computer graphics research on the other hand, such approaches may facilitate the creation of more realistic human movements in video games, adjusted to the estimated energy expenditure [119] and the character's long-term fatigue induced by selected actions [120]. Therefore, they open the path for modeling temporally evolving, ergonomic effects [121], which may allow in the future to synthesize fatigue-driven motion, and thereby produce more empathic computer graphics characters' animation.…”

A User Evaluation Study of Augmented and Virtual Reality Tools for Training and Knowledge Transfer

“…Lifelong learning tools can help the worker adapt to changes in the working environment through corresponding ICT-empowered training processes. These tools consist of components that support the creation and execution of a tutorial process in virtual environments [7,9]. The software implementation was performed in the Unity3D real time development platform, so the tutorial environment can be encoded in Unity Asset Bundles which are archive files that contain platformspecific assets (such as models, textures, audio clips) that can be loaded by Unity at run time.…”

“…In this paper we present and evaluate a series of highly adaptive ICT (Information and Communications Technology) tools, including augmented and virtual reality (AR/VR), developed as part of the Ageing@Work project, that envisions to counteract for crucial issues hindering the ageing workers' workability and well-being by facilitating them to remain active and productive for longer [6][7][8]. Specifically, we address the problem of lifelong learning and knowledge sharing between older and younger workers, as well as remote collaboration.…”

Dynamic Human–Robot Collision Risk Based on Octree Representation

“…While onsite, hands-on safety training is very important for significantly decreasing the risk of injury. On the other hand, virtual training using state-of-the-art technologies, such as virtual, augmented or mixed reality (VR/AR/MR) [10], offers many possibilities to train workers who will be able to work safely in shared or collaborative working environments. VR technology provides a complete virtual environment/test-bed, while AR technology is able to combine the best of both worlds by enhancing real world training in a shared manufacturing environment with visuals, hints, sounds and other additional information.…”