Augmented and virtual reality based monitoring and safety system: A prototype IoT platform

Abstract

This paper presents an Augmented and Virtual Reality (AR/VR) based IoT prototype system. Performing maintenance tasks in a complex environment is quite challenging and difficult due to complex, and possibly, underground facilities, uneasy access, human factors, heavy machineries, etc. Current technology is not acceptable because of significant delays in communication and data transmission, missing multi-input interfaces, and simultaneous supervision of multiple workers who are working in the extreme environment. The aim is to technically advance and combine several technologies and integrate them as integral part of a personnel safety system to improve safety, maintain availability, reduce errors and decrease the time needed for scheduled or ad hoc interventions. We emphasize on the aspects that were made "feasible" on the worker's side due to the equipment used (mobile computing equipment). We present that the demanding tasks that previously were simply undertaken on the fixed infrastructure are now possible on the mobile end. The research challenges lie in the development of real-time datatransmission, instantaneous analysis of data coming from different inputs, local intelligence in low power embedded systems, interaction with multiple on-site users, complex user interfaces, portability and wearability. This work is part EDUSAFE, a Marie Curie ITN (Initial Training Network) project focusing on research into the use of Augmented and Virtual Reality (AR/VR) during planned and ad hoc maintenance in extreme work environments.

NIST Big Data Interoperability Framework:

“…Safety systems incorporating voluminous sensor streams represent this family of use cases. Preliminary research addressing IoT safety is already under way [84], [32], [85]. The latter work was reported during an international conference now more than a decade old, the International Conference on System Safety and Cybersecurity.…”

BIM-Based AR Maintenance System (BARMS) as an Intelligent Instruction Platform for Complex Plumbing Facilities

“…Maintenance has become one of the frequently emphasized aspects of construction [1]. For instance, a Head-Mounted Displays (HMDs)-based AR system was developed for maintenance tasks in a complex environment [2]. A natural marker-based AR framework has also been proposed to support on-site maintenance tasks in facility management [3,4].…”

Key Research Priorities for Factories of the Future—Part I: Missions

“…Augmented and Virtual Reality (AR/VR) represent promising technologies that can be further enhanced for industrial applications (see also Sect. 20.2.5) [54,55]. • Evolutionary planning and control that does not take the production facilities as a fixed asset but as an evolving tissue connecting several sites [56].…”

“…The development of IoA devices and systems with processors, sensors, and actuators will require the further development and integration of several enabling technologies, e.g. Augmented and Virtual Reality (AR/VR) [54,55,123], High Performance Computing [124], Cloud and Fog Computing [125][126][127], Cyber-Physical Production Systems [128], Big Data Analytics [129], ultrafast communication infrastructures and standards, artificial intelligence [130], data storage, sensors and monitoring [131], wearable devices [68], and actuator technologies.…”