Utilising low cost RGB-D cameras to track the real time progress of a manual assembly sequence

Abstract

Purpose The purpose of this paper is to explore the role that computer vision can play within new industrial paradigms such as Industry 4.0 and in particular to support production line improvements to achieve flexible manufacturing. As Industry 4.0 requires “big data”, it is accepted that computer vision could be one of the tools for its capture and efficient analysis. RGB-D data gathered from real-time machine vision systems such as Kinect ® can be processed using computer vision techniques. Design/methodology/approach This research exploits RGB-D cameras such as Kinect® to investigate the feasibility of using computer vision techniques to track the progress of a manual assembly task on a production line. Several techniques to track the progress of a manual assembly task are presented. The use of CAD model files to track the manufacturing tasks is also outlined. Findings This research has found that RGB-D cameras can be suitable for object recognition within an industrial environment if a number of constraints are considered or different devices/techniques combined. Furthermore, through the use of a HMM inspired state-based workflow, the algorithm presented in this paper is computationally tractable. Originality/value Processing of data from robust and cheap real-time machine vision systems could bring increased understanding of production line features. In addition, new techniques that enable the progress tracking of manual assembly sequences may be defined through the further analysis of such visual data. The approaches explored within this paper make a contribution to the utilisation of visual information “big data” sets for more efficient and automated production.

Human in the Loop: Industry 4.0 Technologies and Scenarios for Worker Mediation of Automated Manufacturing

“…The quality of visual processing has improved considerably wit h the popularity of software/hardware toolkits. Vision can a lso be used in a digital assistive system that digitizes manufacturing ta sks in real time and provide feedback to workers [89]. Teke et.…”

Process modeling for smart factories: using science mapping to understand the strategic themes, main challenges and future trends

“…The highly developed cluster “INDUSTRIAL-ENVIRONMENTS” discusses modeling knowledge and business intelligence as crucial factors for the success of industries that want to adopt higher technological levels and achieve flexibility and productivity (Neuböck and Schrefl, 2015). In industrial scenarios, the modeling and application of an Industrial Internet of Things (IIoT) is also widely discussed (Aazam et al , 2018) as well as the concern with computer vision, processing and data management in smart factories (Oyekan et al , 2019). In addition, the cluster “ADMINISTRATIVE-DATA-PROCESSING” includes studies that propose flexible monitoring systems to monitor and analyze flexible and dynamic business processes, strongly relating the human factor to organizational processes and technologies (Repta et al , 2017).…”

Machine Vision—Moving from Industry 4.0 to Industry 5.0

“…The ability to monitor manual work is also an important aspect of smart industry based on machine vision. Oyekan et al [34], in the assembly processes in a production line, employed an RGB-D camera, such as a Kinect camera, along with machine vision techniques to acquire big data in real time, towards improving the production line and enabling a more flexible construction.…”