Improving automated visual fault inspection for semiconductor manufacturing using a hybrid multistage system of deep neural networks

Abstract

In the semiconductor industry, automated visual inspection aims to improve the detection and recognition of manufacturing defects by leveraging the power of artificial intelligence and computer vision systems, enabling manufacturers to profit from an increased yield and reduced manufacturing costs. Previous domain-specific contributions often utilized classical computer vision approaches, whereas more novel systems deploy deep learning based ones. However, a persistent problem in the domain stems from the recognition of very small defect patterns which are often in the size of only a few $$\mu $$ μ m and pixels within vast amounts of high-resolution imagery. While these defect patterns occur on the significantly larger wafer surface, classical machine and deep learning solutions have problems in dealing with the complexity of this challenge. This contribution introduces a novel hybrid multistage system of stacked deep neural networks (SH-DNN) which allows the localization of the finest structures within pixel size via a classical computer vision pipeline, while the classification process is realized by deep neural networks. The proposed system draws the focus over the level of detail from its structures to more task-relevant areas of interest. As the created test environment shows, our SH-DNN-based multistage system surpasses current approaches of learning-based automated visual inspection. The system reaches a performance (F1-score) of up to 99.5%, corresponding to a relative improvement of the system’s fault detection capabilities by 8.6-fold. Moreover, by specifically selecting models for the given manufacturing chain, runtime constraints are satisfied while improving the detection capabilities of currently deployed approaches.

A Fault Detection System for Wiring Harness Manufacturing Using Artificial Intelligence

“…Its ability to learn from labeled data and predict outcomes makes it a powerful tool for fault detection and classification, particularly in complex manufacturing processes [10]. This approach has been used in sectors like semiconductor manufacturing, where the early detection of faults can offer significant time and cost savings [5,11]. The advantages of supervised learning extend to various applications within the manufacturing sector.…”

Towards improving challenging stochastic defect detection in SEM images based on improved YOLOv5

“…In, 8 authors proposed a useful framework as single shot detector for the detection of mixed-type defect patterns, which in general, concurrently occur in a wafer bin map. T. Schlosser et al, 9 have proposed a novel hybrid multistage system of stacked deep neural networks, as "SH-DNN", which allows localization of small defect patterns (few μm) on the much larger wafer surface. An extensive discussion, on existing research approaches…”

Automatic detection of wire bonding defects in microwave components using multi-stage hybrid methods based on deep learning

“…Davtalab et al [11] proposed an automatic layer defect detection method for architectural 3D printing using semantic segmentation models and image processing methods, including edge detection and Hough transform. Schlosser et al [12] used image processing techniques to segment chip and street areas in wafer images and subsequently designed a defect classification network to complete the detection. This approach led to a significant improvement in the accuracy of wafer defect detection compared with the direct use of object detection networks.…”