Multiresolution Knowledge Distillation for Anomaly Detection

Salehi, Mohammadreza; Sadjadi, Niousha; Baselizadeh, Soroosh; Rohban, Mohammad Hossein; Rabiee, Hamid R.

doi:10.48550/arxiv.2011.11108

Cited by 4 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that the idea of feature reconstruction was first proposed without the need of a frozen pre-trained network, in the "Uninformed Students" paper, 41 where an ensemble of CNNs is trained to mimic a teacher network. The idea has since been extended by using several feature maps from a single network, 42,43 as well as by blurring the input. 44 The methods benefiting from frozen pre-trained networks often perform very well because they cannot forget the richness of the pre-trained feature representations, which often happens when fine-tuning such networks on different data due to catastrophic forgetting.…”

Section: Modelling Normality With Deep Pre-trained Featuresmentioning

confidence: 99%

Data refinement for fully unsupervised visual inspection using pre-trained networks

Cordier¹,

Missaoui²,

Gutierrez³

2022

Preprint

View full text Add to dashboard Cite

Anomaly detection has recently seen great progress in the field of visual inspection. More specifically, the use of classical outlier detection techniques on features extracted by deep pre-trained neural networks have been shown to deliver remarkable performances on the MVTec Anomaly Detection (MVTec AD) dataset. However, like most other anomaly detection strategies, these pre-trained methods assume all training data to be normal. As a consequence, they cannot be considered as fully unsupervised. There exists to our knowledge no work studying these pre-trained methods under fully unsupervised setting. In this work, we first assess the robustness of these pre-trained methods to fully unsupervised context, using polluted training sets (i.e. containing defective samples), and show that these methods are more robust to pollution compared to methods such as CutPaste. We then propose SROC, a Simple Refinement strategy for One Class classification. SROC enables to remove most of the polluted images from the training set, and to recover some of the lost AUC. We further show that our simple heuristic competes with, and even outperforms much more complex strategies from the existing literature.

show abstract

Section: Modelling Normality With Deep Pre-trained Featuresmentioning

confidence: 99%

Data refinement for fully unsupervised visual inspection using pre-trained networks

Cordier¹,

Missaoui²,

Gutierrez³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…It was introduced in the "Uninformed Students" paper, 30 where an ensemble of CNNs is trained to mimic a teacher network. The idea has since been extended by using several feature maps from a single network, 31,32 as well as by blurring the input. 33 The methods benefiting from frozen pre-trained networks often perform very well because they cannot forget the richness of the pre-trained feature representations, which often happens when fine-tuning such networks on different data due to catastrophic forgetting.…”

Section: Related Workmentioning

confidence: 99%

Data augmentation and pre-trained networks for extremely low data regimes unsupervised visual inspection

Gutierrez¹,

Cordier²,

Caldeira³

et al. 2021

Preprint

View full text Add to dashboard Cite

The use of deep features coming from pre-trained neural networks for unsupervised anomaly detection purposes has recently gathered momentum in the computer vision field. In particular, industrial inspection applications can take advantage of such features, as demonstrated by the multiple successes of related methods on the MVTec Anomaly Detection (MVTec AD) dataset. These methods make use of neural networks pre-trained on auxiliary classification tasks such as ImageNet. However, to our knowledge, no comparative study of robustness to the low data regimes between these approaches has been conducted yet. For quality inspection applications, the handling of limited sample sizes may be crucial as large quantities of images are not available for small series. In this work, we aim to compare three approaches based on deep pre-trained features when varying the quantity of available data in MVTec AD: KNN, Mahalanobis, and PaDiM. We show that although these methods are mostly robust to small sample sizes, they still can benefit greatly from using data augmentation in the original image space, which allows to deal with very small production runs.

show abstract

“…With the development of deep learning technology, the powerful feature extraction capability of deep networks has greatly enriched the technical methods of unsupervised anomaly detection. Positive sample anomaly detection methods using deep learning networks can be classified into two main categories [ 23 ]: image-reconstruction-based methods [ 24 , 25 , 26 ] and discriminative-embedding-based methods [ 27 , 28 ]. Image-reconstruction-based methods tend to learn the structural information of positive samples for image reconstruction and compare the reconstruction differences between the image to be detected and the abnormality-free map to achieve abnormality detection.…”

Section: Introductionmentioning

confidence: 99%

Anomaly Detection in Annular Metal Turning Surfaces Based on a Priori Information and a Multi-Scale Self-Referencing Template

Suo,

Zhang,

Liu

et al. 2023

Sensors

View full text Add to dashboard Cite

To solve the problem of anomaly detection in annular metal turning surfaces, this paper develops an anomaly detection algorithm based on a priori information and a multi-scale self-referencing template by combining the imaging characteristics of annular workpieces. First, the annular metal turning surface is unfolded into a rectangular expanded image using bilinear interpolation to facilitate subsequent algorithm development. Second, the grayscale information from the positive samples is used to obtain the a priori information, and a multi-scale self-referencing template method is used to obtain its own multi-scale information. Then, the phase error and large-size anomaly interference problems of the self-referencing method are overcome by combining the a priori information with its own information, and an accurate response to anomalous regions of various sizes is realized. Finally, the segmentation completeness of the anomalous region is improved by utilizing the region growing method. The experimental results show that the proposed method achieves a mean pixel AUROC of 0.977, and the mean M_IOU of segmentation reaches 0.788. In terms of efficiency, this method is also much more efficient than the commonly used anomaly detection algorithms. The proposed method can achieve rapid and accurate detection of defects in annular metal turning surfaces and has good industrial application value.

show abstract

Multiresolution Knowledge Distillation for Anomaly Detection

Cited by 4 publications

References 0 publications

Data refinement for fully unsupervised visual inspection using pre-trained networks

Data refinement for fully unsupervised visual inspection using pre-trained networks

Data augmentation and pre-trained networks for extremely low data regimes unsupervised visual inspection

Anomaly Detection in Annular Metal Turning Surfaces Based on a Priori Information and a Multi-Scale Self-Referencing Template

Contact Info

Product

Resources

About