PGA-Net: Pyramid Feature Fusion and Global Context Attention Network for Automated Surface Defect Detection

Abstract

Surface defect detection is a critical task in industrial production process. Nowadays, there are lots of detection methods based on computer vision and have been successfully applied in industry, they also achieved good results. However, achieving full automation of surface defect detection remains a challenge, due to the complexity of surface defect, in intra-class, while the defects between inter-class contain similar parts, there are large differences in appearance of the defects. To address these issues, this paper proposes a pyramid feature fusion and global context attention network for pixel-wise detection of surface defect, called PGA-Net. In the framework, the multi-scale features are extracted at first from backbone network. Then the pyramid feature fusion module is used to fuse these features into five resolutions through some efficient dense skip connections. Finally, the global context attention module is applied to the fusion feature maps of adjacent resolution, which allows effective information propagate from low-resolution fusion feature maps to high-resolution fusion ones. In addition, the boundary refinement block is added to the framework to refine the boundary of defect and improve the result of predict. The final prediction is the fusion of the five resolutions fusion feature maps. The results of evaluation on four real-world defect datasets demonstrate that the proposed method outperforms the state-of-the-art methods on mean Intersection of Union and mean Pixel Accuracy

Research Progress of Automated Visual Surface Defect Detection for Industrial Metal Planar Materials

“…After the model was tested on the Northeastern University (NEU) surface defect dataset, the F measure was 0.606, and the calculation time on each image was 0.713 s. Yan et al [ 95 ] developed a probabilistic saliency framework based on a feature enhancement mechanism for realizing robust defect detection on a micro 3D texture surface of industrial products, which designed the absolute strength deviation and local strength aggregation to represent the initial saliency of the pixel level while all pixels are classified as defective or non-defective. To address the issues of intra-class defects having large differences in appearance while inter-class defects contain similar parts, Song et al have studied many approaches to combine visual salience with other ideas, such as Encoder–Decoder Residual network (EDRNet) [ 96 ], multiple constraints and improve texture feature (MCITF) [ 97 ], attention mechanism [ 98 ], and pyramid feature (PGA-Net) [ 99 ], and the experimental results show that they are both effective and outperform the state-of-the-art methods. The advantages of introducing visual saliency are mainly manifested in two aspects: first, limited computing resources are allocated to more important information in images and videos; second, the results of introducing visual saliency are more in line with people’s visual cognition needs.…”

A Light-Weight Deep-Learning Model with Multi-Scale Features for Steel Surface Defect Classification

“…The core of the model is a simple feed-forward neural network including convolutional layers and pooling layers. Dong et al [ 3 ] utilized the feature fusion and context attention strategies to address the complexity of surface defects in both intra-class and inter-class. Specifically, multi-scale features were fused by skip connections first.…”

“…With the recent advancement in deep learning, surface defect classification with deep learning-based techniques has become popular. For example, the works of [ 3 , 4 , 5 ] utilize deep learning models for surface defects classification which proves that deep learning models are far more accurate than traditional image processing-based and machine learning methods. To enhance the performance, data augmentation methods are usually utilized [ 6 ].…”

“…To improve the accuracy of defects classification when there is few labeled data and a lot of unlabeled data, semi-supervised learning method like [ 7 ] is developed to take advantage of the unlabeled data, which are easier to obtain than labeled data, with good performance. Moreover, feature fusion and context attention strategies are also employed to achieve better accuracy [ 3 ], which particularly address the problem of low performance on images with large intra-class inconsistency and inter-class similarity.…”

A Deep-Learning-based 3D Defect Quantitative Inspection System in CC Products Surface

“…Veitch-Michaelis et al [24] studied the 3D cracks recognition method through the combination of morphological detection and SVM classifier. Hongwen Dong in Northeastern University proposed a pyramid feature fusion and a global context attention network for pixel-wise detection of surface defect in the industrial production process [25]. Fatima A. Saiz et al [26] reported a deep-learning based automatic defects recognition system in which CNN was utilized in the model design, which achieved an outstanding classification rate.…”