X-Reality System Architecture for Industry 4.0 Processes

Abstract

Information visualization has been widely adopted to represent and visualize data patterns as it offers users fast access to data facts and can highlight specific points beyond plain figures and words. As data comes from multiple sources, in all types of formats, and in unprecedented volumes, the need intensifies for more powerful and effective data visualization tools. In the manufacturing industry, immersive technology can enhance the way users artificially perceive and interact with data linked to the shop floor. However, showcases of prototypes of such technology have shown limited results. The low level of digitalization, the complexity of the required infrastructure, the lack of knowledge about Augmented Reality (AR), and the calibration processes that are required whenever the shop floor configuration changes hinders the adoption of the technology. In this paper, we investigate the design of middleware that can automate the configuration of X-Reality (XR) systems and create tangible in-site visualizations and interactions with industrial assets. The main contribution of this paper is a middleware architecture that enables communication and interaction across different technologies without manual configuration or calibration. This has the potential to turn shop floors into seamless interaction spaces that empower users with pervasive forms of data sharing, analysis and presentation that are not restricted to a specific hardware configuration. The novelty of our work is due to its autonomous approach for finding and communicating calibrations and data format transformations between devices, which does not require user intervention. Our prototype middleware has been validated with a test case in a controlled digital-physical scenario composed of a robot and industrial equipment.

Projection-Based Augmented Reality Assistance for Manual Electronic Component Assembly Processes

“…Moreover, as more and more visual computing solutions are integrated into industrial shop floors, the complexity of communication and interaction across different peripherals and industrial devices increases. Nonetheless, [21] has recently proposed a middleware architecture that enables communication and interaction across different technologies without manual configuration or calibration.…”

Photorealism and Kinematics for Web-based CAD data

“…The biggest challenge remains the need for including more automated tools to adapt static industrial content to meet interactivity [Simões et al 2020]. Recent works are tackling automation for various viewpoints [Simões et al 2018[Simões et al , 2019. For instance, Park and Oh [Park and Oh 2015] worked on an automatic method to extract kinematic data directly from CAD models.…”

Cross reality to enhance worker cognition in industrial assembly operations

“…Kollatsch et al [34] developed a prototype for the visualization of information from control systems (e.g., PLC, CNC) directly on-site. Simões et al [60] proposed a middleware to create tangible in-site visualizations and interactions with industrial assets. Gauglitz et al [20] proposed a software for a tablet that augments airplane cockpits with AR instructions.…”

“…However, their use requires programming skills to develop AR applications. An alternative is the use of AR authoring tools, which were proposed a decade ago [25,33,51,60,62]. Their main advantage is that they do not built upon cost and time consuming recompilation steps, and consequently the updates to the application are fast and can be completed efficiently.…”

“…Their main advantage is that they do not built upon cost and time consuming recompilation steps, and consequently the updates to the application are fast and can be completed efficiently. Other authors [14,40,60,63] automated certain aspects of content creation, so assembly instructions can be automatically generated from CAD files, thus reducing the authoring burden. However, fine-tuning existent tools to solve a specific domain problem remains a open challenge.…”