Online Assembly Inspection Integrating Lightweight Hybrid Neural Network With Positioning Box Matching

Abstract

Assembly inspection methods have been widely used in the process of mechanical product assembly for quality issues. However, some challenges remain to be solved, such as low detection efficiency, poor accuracy and sensitive to camera view. This paper proposes an online assembly inspection scheme based on hybrid neural network and positioning box matching. A hybrid multi-task learning neural network with transformer attention mechanism is proposed to simultaneously detect key points and assembly parts with high accuracy and strong robustness. Utilizing the key points detection results, the transformation relationships between the in-site assembly images and the standard templates are solved. According to the results of assembly parts detection, the detected 2D positioning bounding boxes are matched with those in the standard assembly templates, so as to evaluate whether the current step has quality problems. In addition, the proposed method is tested on an assembly dataset constructed in this paper. For key points detection, the average error is less than 1 pixel. For parts detection, the mean average precision is 97.66%. The missing and wrong assembly inspection results show that the average F1 score reaches 93.96%. This inspection method can be employed to detect the missing and wrong assembly faults of each assembly step online, improving the assembly quality of products.

Validating the Use of Smart Glasses in Industrial Quality Control: A Case Study

“…Zhao et al [52] introduced an innovative approach to online assembly inspection, merging a lightweight hybrid neural network with precise positioning box matching techniques. Navigating the complexities of real-time assembly assessments, the authors delivered a comprehensive and integrated methodology promising to enhance the accuracy and efficiency of product assembly inspections significantly.…”