Industry 4.0 is moving forward under technology upgrades, utilizing information technology to improve the intelligence of the industry, whereas Industry 5.0 is value-driven, aiming to focus on essential societal needs, values, and responsibility. The manufacturing industry is currently moving towards the integration of productivity enhancements and sustainable human employment. Such a transformation has deeply changed the human–machine interaction (HMI), among which digital twin (DT) and extended reality (XR) are two cutting-edge technologies. A manufacturing DT offers an opportunity to simulate, monitor, and optimize the machine. In the meantime, XR empowers HMI in the industrial field. This paper presents an XR application framework for DT-based services within a manufacturing context. This work aims to develop a technological framework to improve the efficiency of the XR application development and the usability of the XR-based HMI systems. We first introduce four layers of the framework, including the perception layer with the physical machine and its ROS-based simulation model, the machine communication layer, the network layer containing three kinds of communication middleware, and the Unity-based service layer creating XR-based digital applications. Subsequently, we conduct the responsiveness test for the framework and describe several XR industrial applications for a DT-based smart crane. Finally, we highlight the research challenges and potential issues that should be further addressed by analyzing the performance of the whole framework.

Digital twin technology empowers the digital transformation of the industrial world with an increasing amount of data, which meanwhile creates a challenging context for designing a human–machine interface (HMI) for operating machines. This work aims at creating an HMI for digital twin based services. With an industrial crane platform as a case study, we presented a mixed reality (MR) application running on a Microsoft HoloLens 1 device. The application, consisting of visualization, interaction, communication, and registration modules, allowed crane operators to both monitor the crane status and control its movement through interactive holograms and bi-directional data communication, with enhanced mobility thanks to spatial registration and tracking of the MR environment. The prototype was quantitatively evaluated regarding the control accuracy in 20 measurements following a step-by-step protocol that we defined to standardize the measurement procedure. The results suggested that the differences between the target and actual positions were within the 10 cm range in three dimensions, which were considered sufficiently small regarding the typical crane operation use case of logistics purposes and could be improved with the adoption of robust registration and tracking techniques in our future work.

Digital twins (DTs) and eXtended Reality (XR) are two core technological enablers for engineering in the Metaverse that can accelerate the human-centric Industry 5.0 transformation. The digital twin technology provides a digital representation of a physical asset with data linkages for inspection, monitoring, and prediction of complex processes or systems, while eXtended reality offers real-and-virtual combined environments for human users to interact with machines. However, the synergies between digital twins and eXtended reality remain understudied. This work addresses this research gap by introducing a novel method “TwinXR” that leverages ontology-based descriptions of Digital twins, i.e., digital twin documents, in industrial eXtended reality applications. To ease the use of the TwinXR method, we publish a Unity package that allows data flow and conversion between eXtended reality applications and digital twin documents on the server. Finally, the work applies the TwinXR method in two industrial eXtended reality applications involving overhead cranes and a robot arm to demonstrate the use and indicate the validity of the method. We conclude that the TwinXR method is a promising way to advance the synergies between digital twins and eXtended reality: For eXtended reality, TwinXR enables efficient and scalable eXtended reality development; For digital twins, TwinXR unlocks and demonstrates the potential of digital twins for data interchange and system interoperation. Future work includes introducing more detailed principles of Semantic Web and Knowledge Graph, as well as developing factory-level TwinXR-compatible applications.