Inline Defective Laser Weld Identification by Processing Thermal Image Sequences with Machine and Deep Learning Techniques

Abstract

The non-destructive testing methods offer great benefit in detecting and classifying the weld defects. Among these, infrared (IR) thermography stands out in the inspection, characterization, and analysis of the defects from the camera image sequences, particularly with the recent advent of deep learning. However, in IR, the defect classification becomes a cumbersome task because of the exposure to the inconsistent and unbalanced heat source, which requires additional supervision. In light of this, authors present a fully automated system capable of detecting defective welds according to the electrical resistance properties in the inline mode. The welding process is captured by an IR camera that generates a video sequence. A set of features extracted by such video feeds supervised machine learning and deep learning algorithms in order to build an industrial diagnostic framework for weld defect detection. The experimental study validates the aptitude of a customized convolutional neural network architecture to classify the malfunctioning weld joints with mean accuracy of 99% and median f1 score of 73% across five-fold cross validation on our locally acquired real world dataset. The outcome encourages the integration of thermographic-based quality control frameworks in all applications where fast and accurate recognition and safety assurance are crucial industrial requirements across the production line.

A Novel Framework Based on Deep Learning Architecture for Continuous Human Activity Recognition with Inertial Sensors

“…For each combination of sensors, the dataset of the separate execution of ADLs was split into ten stratified folds, 75% of which is assigned for training and the remaining 25% for validation. Afterward, a 10-fold-cross-validation methodology is employed to ensure a fair and unbiased evaluation of the model [48]. All investigations in this study are conducted on the Google Colab-Pro framework to train the model on a Tesla T4.…”

“…Convolutional Neural Networks (CNNs) are the most widely employed among the DL architectures proposed in the studies addressing HAR [8,22,23,27,32,38,43,46]. CNNs usually work on images by means of two-dimensional convolutions for practical problems as defect detection [47,48]; notwithstanding, their one-dimensional variant is preferred because it allows working directly on time-series signals instead of their corresponding images, thus reducing the computational cost [32]. Furthermore, CNNs employed in HAR frameworks may have either a sequential or a multi-branch structure: in the former case, layers process all the IMU signals of the input dataset [23], whereas in the latter case each branch, which may be fed by one of the IMUs included in the experimental setup, is computed in parallel with the others [8,27].…”

Optimal Transducer Placement for Deep Learning-Based Non-Destructive Evaluation

“…Among deep learning artificial neural networks, CNN (Convolutional Neural Network) is one of the effective algorithms in representing and extracting spatial patterns. Because of the efficiency and high accuracy in image classification, the CNN algorithm is widely used in plant disease diagnosis [ 20 , 21 , 22 , 23 , 24 ] and medical fields [ 25 , 26 , 27 , 28 , 29 ], as well as fault diagnosis of mechanical systems [ 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ]. Typically, bearings are the biggest cause of motor failures, so plenty of CNN-based fault diagnosis studies have been conducted for monitoring the bearing condition [ 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 ].…”

“…Zhong et al applied SVM (Support Vector Machine) to identify small faults in the gas turbine by training changes in performance parameters such as exhaust gas temperature and fan speed when a fault occurred in a gas turbine [ 40 ]. In addition, researches have been conducted on detecting defects in composites using CNN training [ 41 ], determining defects in weld joints [ 42 ], and recognizing cracks in asphalt pavement [ 43 ].…”

Study on Porosity Defect Detection in Narrow Gap Laser Welding Based on Spectral Diagnosis

“…Post-welding nondestructive testing technologies such as X-ray detection and ultrasonic inspection can effectively detect the internal defects of welds, but the test results are subject to the component size and the subjective judgment of inspectors [ 16 ]. In order to realize real-time detection of defects, acoustic emission, infrared photography, visual imaging, and other sensing means have been used to monitor the laser welding process [ 17 , 18 ]. Will et al [ 19 ] used optical coherence tomography (OCT) to conduct online monitoring of the keyhole state during laser welding.…”