Generative and Contrastive Combined Support Sample Synthesis Model for Few-/Zero-Shot Surface Defect Recognition

Abstract

Surface defect detection is one of the most important vision-based measurements for intelligent manufacturing. Existing detection methods mainly require massive numbers of defect samples to train the model to detect the defects. Nowadays, inadequate defect samples and labels are inevitably encountered in industrial data environments due to the highly automated and stable production lines escalatingly deployed, causing fewer and fewer defective products to be produced. Consequently, manual interventions are deeply required to analyze the abnormal sample once an unseen defect accidentally emerges that significantly decreases productivity. To this end, this paper proposes a novel few/zero-shot compatible surface defect detection method without requiring massive or even any defect samples to detect surface defects. First, a novel contrastive generator is proposed to use defects' text descriptions to synthesize "fake" visual features for those rare defects. Then, the synthesized visual features (for support samples) are fused with "real" visual features (for query samples) into a similarity graph to align the relationships between support samples and query samples. After, a class center optimization method is proposed to iteratively update the similarity matrix of the graph to obtain the classification probabilities for the query samples. Eventually, the proposed method solves the problem of the lack of defect samples and the inability of few-shot learning-based methods to recognize unseen classes. Massive experiments on eight fine-grained datasets show that our method gains an average of +8.29% improvements on few-shot recognition tasks and achieves an average of +8.23% improvements on zero-shot recognition tasks compared with the state-of-the-art method. Moreover, the proposed method is deployed in a real-world prototype system, and the method's feasibility is finally demonstrated. The core code of the proposed method is available at: https://github.com/NDYBSNDY/AsC