Multi-scale GAN with transformer for surface defect inspection of IC metal packages

Chen, Kaiqiong; Cai, Nian; Wu, Zhenshuang; Xia, Hao; Zhou, Shuai; Wang, Han

doi:10.1016/j.eswa.2022.118788

Cited by 13 publications

(6 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, training GANs is known to pose difficulties, as the two-player objective function often leads to artifacts and mode collapse, especially when generating high-resolution images. An automated optical inspection system is also built on semi-supervised deep learning for the inspection of IC metal package surface defects [26]. In contrast to earlier inspection techniques, a fully multiscale inspection framework is suggested to conduct assessments of flaws at various scales.…”

Section: Defect Inspection Using Ganmentioning

confidence: 99%

Enhancing Battery Exterior Defect Inspection Accuracy Through Defect-Background Separated GAN Development

Ku,

Pahk

2024

IEEE Access

View full text Add to dashboard Cite

This paper aims to develop a defect-background separated generative adversarial network (GAN) using deep learning and GAN to enhance the accuracy of battery exterior defect inspection. In actual battery production lines, the occurrence rates of defects vary by defect type, making it challenging to create a large, uniform defect dataset due to the time required for defect acquisition. This leads to a reduction in the accuracy of battery exterior defect inspection. To construct a large, uniform defect dataset, this paper proposes a defect-background separated GAN based on the principles of GANs. The defect-background separated GAN performs effective defect and background separation learning by referring to the segmentation labeling of defects. Through dataset augmentation using the defect-background separated GAN, the performance quality of newly generated synthetic defect images has improved, and accuracy in battery exterior defect inspection can be enhanced through extensive dataset training. Experimental results show the lowest Fréchet inception distance score among various other methods for the battery exterior defect dataset, and it generates clear synthetic defects perceptible to the human eye. By training defect segmentation on this large, uniform defect dataset, an accuracy of 96.1% and an intersection over union value of 0.71 were achieved. Ultimately, applying this defect inspection network to the actual production line demonstrated a 72% improvement in time efficiency. This demonstrates the stability and robustness of the large, uniform defect dataset generated through the defect-background separated GAN.

show abstract

Section: Defect Inspection Using Ganmentioning

confidence: 99%

Enhancing Battery Exterior Defect Inspection Accuracy Through Defect-Background Separated GAN Development

Ku,

Pahk

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Ho et al [148] proposed a segmentation task based on ResNet 50 performing feature extraction and concatenation to combine the multilevel features, followed by binary classification of image patches a little bigger than a pixel; in so doing, the system detects and locates defective pixels precisely, even if surrounded by a complex background. Chen et al [149] proposed a multi-scale adaptive thresholding to support their GAN, highlighting potential defective pixels in the weighted difference image. More in detail, they adopt a smaller threshold to focus the inspection more on small defects in a large-scale sample and vice versa.…”

Section: ) Conceptualizationmentioning

confidence: 99%

“…With this kind of loss, a prototype of image (e.g., non-defective) can be obtained with translation from a different image content (e.g., with defects) giving some guidelines. Chen et al [149] developed a multi-scale VOLUME 11, 2023 GAN with transformer to reconstruct non-defective patches at different scales, comparing them with input patches to find pixel-level differences. In particular, the loss function of the generator involves three different loss terms: multi-scale feature loss, content loss and adversarial loss.…”

Section: A: Image-level Supervisionmentioning

confidence: 99%

Deep Learning for Automatic Vision-Based Recognition of Industrial Surface Defects: A Survey

et al. 2023

View full text Add to dashboard Cite

Automatic vision-based inspection systems have played a key role in product quality assessment for decades through the segmentation, detection, and classification of defects. Historically, machine learning frameworks, based on hand-crafted feature extraction, selection, and validation, counted on a combined approach of parameterized image processing algorithms and explicated human knowledge. The outstanding performance of deep learning (DL) for vision systems, in automatically discovering a feature representation suitable for the corresponding task, has exponentially increased the number of scientific articles and commercial products aiming at industrial quality assessment. In such a context, this article reviews more than 220 relevant articles from the related literature published until February 2023, covering the recent consolidation and advances in the field of fully-automatic DL-based surface defects inspection systems, deployed in various industrial applications. The analyzed papers have been classified according to a bi-dimensional taxonomy, that considers both the specific defect recognition task and the employed learning paradigm. The dependency on large and high-quality labeled datasets and the different neural architectures employed to achieve an overall perception of both well-visible and subtle defects, through the supervision of fine or/and coarse data annotations have been assessed. The results of our analysis highlight a growing research interest in defect representation power enrichment, especially by transferring pre-trained layers to an optimized network and by explaining the network decisions to suggest trustworthy retention or rejection of the products being evaluated.

show abstract

“…Defect-GAN employs adaptive noise insertion to capture the stochastic variations within defects. Kaiqiong et al [ 33 ] proposed an entirely multiscale GAN with a transformer to capture the intrinsic patterns of qualified samples of IC metal package images at multiple scales. The proposed GAN model is designed to improve the quality of the generated images by capturing the patterns of the images at multiple scales.…”

Section: Related Workmentioning

confidence: 99%

Intraclass Image Augmentation for Defect Detection Using Generative Adversarial Neural Networks

Sampath

Maurtua

Martín

et al. 2023

Sensors

View full text Add to dashboard Cite

Surface defect identification based on computer vision algorithms often leads to inadequate generalization ability due to large intraclass variation. Diversity in lighting conditions, noise components, defect size, shape, and position make the problem challenging. To solve the problem, this paper develops a pixel-level image augmentation method that is based on image-to-image translation with generative adversarial neural networks (GANs) conditioned on fine-grained labels. The GAN model proposed in this work, referred to as Magna-Defect-GAN, is capable of taking control of the image generation process and producing image samples that are highly realistic in terms of variations. Firstly, the surface defect dataset based on the magnetic particle inspection (MPI) method is acquired in a controlled environment. Then, the Magna-Defect-GAN model is trained, and new synthetic image samples with large intraclass variations are generated. These synthetic image samples artificially inflate the training dataset size in terms of intraclass diversity. Finally, the enlarged dataset is used to train a defect identification model. Experimental results demonstrate that the Magna-Defect-GAN model can generate realistic and high-resolution surface defect images up to the resolution of 512 × 512 in a controlled manner. We also show that this augmentation method can boost accuracy and be easily adapted to any other surface defect identification models.

show abstract

Multi-scale GAN with transformer for surface defect inspection of IC metal packages

Cited by 13 publications

References 32 publications

Enhancing Battery Exterior Defect Inspection Accuracy Through Defect-Background Separated GAN Development

Enhancing Battery Exterior Defect Inspection Accuracy Through Defect-Background Separated GAN Development

Deep Learning for Automatic Vision-Based Recognition of Industrial Surface Defects: A Survey

Intraclass Image Augmentation for Defect Detection Using Generative Adversarial Neural Networks

Contact Info

Product

Resources

About