An Integrated Mobile Augmented Reality Digital Twin Monitoring System

Abstract

The increasing digitalization and advancement in information communication technologies has greatly changed how humans interact with digital information. Nowadays, it is not sufficient to only display relevant data in production activities, as the enormous amount of data generated from smart devices can overwhelm operators without being fully utilized. Operators often require extensive knowledge of the machines in use to make informed decisions during processes such as maintenance and production. To enable novice operators to access such knowledge, it is important to reinvent the way of interacting with digitally enhanced smart devices. In this research, a mobile augmented reality remote monitoring system is proposed to help operators with low knowledge and experience level comprehend digital twin data of a device and interact with the device. It analyses both historic logs as well as real-time data through a cloud server and enriches 2D data with 3D models and animations in the 3D physical space. A cloud-based machine learning algorithm is applied to transform learned knowledge into live presentations on a mobile device for users to interact with. A scaled-down case study is conducted using a tower crane model to demonstrate the potential benefits as well as implications when the system is deployed in industrial environments. This user study verifies that the proposed solution yields consistent measurable improvements for novice users in human-device interaction that is statistically significant.

Exploring the Applications of Digital Twin Technology in Enhancing Sustainability in Civil Engineering: A Review

“…The fidelity of the model can be improved by adding physical properties (hardness, surface roughness, tolerances, density, etc.) [4].…”

“…Finite element analysis (FEA) can be used to model changes such as object bending and deformation. Predictive analysis can be applied to larger datasets through regression and machine learning techniques, combining real-time and historical data to provide diagnostic scenarios that continuously improve the predictive internal logic of the model by identifying the fields that best explain the results for further optimisation [4].…”

“…By combining various data sources and integrating intelligent capabilities (such as artificial intelligence (AI), machine learning (ML), and data analytics) and digital models (such as building information modelling (BIM) models), a DT can represent and forecast current and future conditions. DTs date back to NASA's development of two identical vehicles in the late 1960s, in which it used twins to simulate real-time behaviour for the remote updating, maintenance, and monitoring of the Apollo 13 spacecraft [4]. In 2013, Michael Grieves introduced the DT concept while at the University of Michigan [5].…”

Extended Reality Application Framework for a Digital-Twin-Based Smart Crane

“…Burghardt et al [34] presented a method of programming robots using virtual reality and digital twins. He et al [35] proposed a mobile augmented reality remote monitoring system to help operators with low knowledge.…”

Development of a Smart City Platform Based on Digital Twin Technology for Monitoring and Supporting Decision-Making

“…Monitoring and decision support within a smart city platform entails the utilization of technological tools to gather and analyse data from diverse sensors and information sources dispersed across the urban environment. The data gathered encompass several aspects, such as traffic patterns, air quality measurements, noise pollution levels, energy usage, and other related factors [1,2]. By utilizing precise and up-to-date data analysis, municipal authorities can make well-informed decisions to effectively address emergencies, maximize infrastructure development, improve public services, and establish a more efficient and sustainable urban environment.…”