Status Recognition Using Pre-Trained YOLOv5 for Sustainable Human-Robot Collaboration (HRC) System in Mold Assembly

Abstract

Molds are still assembled manually because of frequent demand changes and the requirement for comprehensive knowledge related to their high flexibility and adaptability in operation. We propose the application of human-robot collaboration (HRC) systems to improve manual mold assembly. In the existing HRC systems, humans control the execution of robot tasks, and this causes delays in the operation. Therefore, we propose a status recognition system to enable the early execution of robot tasks without human control during the HRC mold assembly operation. First, we decompose the mold assembly operation into task and sub-tasks, and define the actions representing the status of sub-tasks. Second, we develop status recognition based on parts, tools, and actions using a pre-trained YOLOv5 model, a one-stage object detection model. We compared four YOLOv5 models with and without a freezing backbone. The YOLOv5l model without a freezing backbone gave the optimal performance with a mean average precision (mAP) value of 84.8% and an inference time of 0.271 s. Given the success of the status recognition, we simulated the mold assembly operations in the HRC environment and reduced the assembly time by 7.84%. This study improves the sustainability of the mold assembly from the point of view of human safety, with reductions in human workload and assembly time.

The Influence of YOLOv5 Hyperparameters for Construction Details Detection

“…Based on other research [17][18][19] and our pilot studies, the results have shown that augmentation has a positive impact on detection accuracy. Furthermore, experiments have shown that by freezing backbones, the accuracy increases about 1.5 times.…”

Satellite Imageries for Detection of Bangladesh’s Rural and Urban Areas Using YOLOv5 and CNN

“…Te YOLO family of models, including YOLOv2, YOLOv3, YOLOv4, YOLOv5, YOLOv6, and the recent YOLOv7, are widely employed for recognition tasks. Te variations in size among the diferent models of the YOLOv5 family, such as YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x, are determined by the width and depth of the BottleneckCSP module [36]. Te primary function of the BottleneckCSP module is to extract features from the feature map, enabling the extraction of valuable information from the input image.…”

Working toward Solving Safety Issues in Human–Robot Collaboration: A Case Study for Recognising Collisions Using Machine Learning Algorithms

“…Next, by applying machine learning algorithms, collisions in shared workspaces can be detected before they occur. In [10] the framework for task and status recognition for HRC in Mold Assembly using YOLOv5 was developed. The results indicated the performance had a mean average precision value of 84.8%.…”

“…Currently it is difficult to compare the effectiveness of the results obtained, due to the specific and strictly defined common workplace of humans and cobots with specific behavioural scenarios. According to the notes in the introduction, the results in works [9,10] achieved an accuracy of 90% in recognising an object and 96.4% in predicting collisions, which can be said to be satisfactory. In [23] the scenario covers the HRC space, where an operator exchanges components with one cobot.…”