Swin Transformer Combined with Convolution Neural Network for Surface Defect Detection

Abstract

Surface defect detection aims to classify and locate a certain defect that exists in the target surface area. It is an important part of industrial quality inspection. Most of the research on surface defect detection are currently based on convolutional neural networks (CNNs), which are more concerned with local information and lack global perception. Thus, CNNs are unable to effectively extract the defect features. In this paper, a defect detection method based on the Swin transformer is proposed. The structure of the Swin transformer has been fine-tuned so that it has five scales of output, making it more suitable for defect detection tasks with large variations in target size. A bi-directional feature pyramid network is used as the feature fusion part to efficiently fuse to the extracted features. The focal loss is used as a loss function to weight the hard- and easy-to-distinguish samples, potentially making the model fit the surface defect data better. To reduce the number of parameters in the model, a shared detection head was chosen for result prediction. Experiments were conducted on the flange surface defect dataset and the steel surface defect dataset, respectively. Compared with the classical CNNs target detection algorithm, our method improves the mean average precision (mAP) by about 15.4%, while the model volume and detection speed are essentially the same as those of the CNNs-based method. The experimental results show that our proposed method is more competitive compared with CNNs-based methods and has some generality for different types of defects.

A New Pest Detection Method Based on Improved YOLOv5m

“…With Transformer, features can be adaptively aggregated from a global view using self-attention mechanisms rather than convolution operations, which extract features from a local fixed field [ 31 ]. The form of standard self-attention operation can be formulated (see Equations (1) and (2)) as follows [ 32 ]: where X is the input feature map; are the linear transformation functions; is the channel dimension; are abbreviations of query, key, and weight value, respectively. We can calculate the dot products of the query with all keys, divide by , and apply a SoftMax function to obtain the weights on the values.…”

“…Here, we need one more projection there that the cost will be hwc 2 . Hence, MSA cost is calculated below in total (see Equation (3)) [ 32 ]: …”

“…It can be observed that the complexity degrades from quadratic square to linear . Thus, W-MSA cost is calculated as below (see in Equation (4)) [ 32 ]: …”