Towards improving challenging stochastic defect detection in SEM images based on improved YOLOv5

Abstract

With the progression of deep learning algorithms in computer vision, a lot of research is taking place in the semiconductor industry towards improving real-time defect detection and classification analysis. An Automated Defect Classification and Detection (ADCD) framework not only enables rapid measurement of dimensions and classification of defects, but also helps minimize production costs, engineering time as well as tool cycle time associated with the defect inspection process. As we continue to shrink the pitch (below 36nm), defect characterization at wafer scale becomes a key issue as it demands rapid measurement but without losing accuracy and repeatability. Also, in the context of high NA lithography (thin resist), accurate metrology becomes difficult with very noisy as well as low contrast images (No BKM exists till now). Human eyes generally demonstrate close to the Bayesian Error limit in detecting smaller objects (for example, extracting contextual information instantaneously from nanoscale defects in SEM images). However, for most One-stage and Two-stage object detectors, this is still a very challenging task due to variable image resolution and SEM (scanning electron microscope) image quality (low SNR). In this research work, we have experimented with different modified YOLOv5 object detectors to improve challenging stochastic defect detection precision. In this work, we have proposed an ensemble strategy by empirically combining multiple custom-trained models (YOLOv5n, YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x) together at the test and inference time. We have noticed four YOLOv5 architecture variants are outperforming against our previous Ensemble ResNets model with improvements of the average precision metric (AP) of the most difficult defect classes as p gap and microbridges as well as overall mAP accuracy. With Ensemble YOLOv5, the p gap AP and microbridge AP metrices have been improved by 35% and 25.33%, respectively, whereas the overall mAP has been improved by 6.25%. The proposed Automated Defect Classification and Detection (ADCD) framework can also be used for high resolution and high-speed metrology, providing rapid identification of defects with improved certainty and further root cause investigation.

Optimizing YOLOv7 for Semiconductor Defect Detection

“…Future work should instead employ hyperparameter optimization algorithms such as genetic algorithms, grid search, or bayesian optimization. 6,13 Certain models do achieve better APs for the bridge and p-gap classes. Most notably, the Tiny model achieves a 36% better bridge AP than the default model.…”

“…1 This has spurred interest in machine learning, particularly deep learning (DL), methods that are able to handle noise and changes in scale better. [1][2][3][4][5][6] In this study, we focus on defect detection, which involves localizing and classifying defect instances, in SEM images. Dey et al 1 showed that a DL framework based on RetinaNet detects semiconductor defects more robustly than rule-based image processing techniques.…”

YOLOv8 for defect inspection of hexagonal directed self-assembly patterns: a data-centric approach

“…ML-based SEM defect detection methods, especially Convolutional Neural Network (CNN) models, are becoming popular due to their improved robustness to these variations. Two main paradigms for training defect inspection models exist, supervised 1,[17][18][19] and unsupervised [20][21][22] learning. Supervised learning, where an ML model learns by comparing its predictions to labels provided by a human expert, usually leads to the best performance for defect detection and classification.…”

SEMI-CenterNet: a machine learning facilitated approach for semiconductor defect inspection

“…The aforementioned object detection model families have been successfully applied to the semiconductor defect detection task as demonstrated in Ref. 3 and Ref. 16.…”

“…While several works have investigated the use of DL methods for semiconductor defect detection, [2][3][4] little work has pursued using DL techniques with lower inference time and smaller models. Ref.…”