Transformaly -- Two (Feature Spaces) Are Better Than One

Abstract

Anomaly detection is a well-established research area that seeks to identify samples outside of a predetermined distribution. An anomaly detection pipeline is comprised of two main stages: (1) feature extraction and (2) normality score assignment. Recent papers used pre-trained networks for feature extraction achieving state-of-the-art results. However, the use of pre-trained networks does not fully-utilize the normal samples that are available at train time. This paper suggests taking advantage of this information by using teacher-student training. In our setting, a pretrained teacher network is used to train a student network on the normal training samples. Since the student network is trained only on normal samples, it is expected to deviate from the teacher network in abnormal cases. This difference can serve as a complementary representation to the pre-trained feature vector. Our method -T ransf ormaly -exploits a pre-trained Vision Transformer (ViT) to extract both feature vectors: the pre-trained (agnostic) features and the teacher-student (fine-tuned) features. We report stateof-the-art AUROC results in both the common unimodal setting, where one class is considered normal and the rest are considered abnormal, and the multimodal setting, where all classes but one are considered normal, and just one class is considered abnormal 1 .

Deep Learning for Computer Vision in Smart Cities

“…Cohen and Avidan [2021] is a ViT-based architecture which currently achieves the best AUROC (Area Under the Receiver Operating Characteristic) results of the state of the art in anomaly detection on CIFAR-10 and CIFAR-100, in both the common unimodal setting and the multimodal setting. It works by using two independent feature spaces: a teacher-student network where the student is only trained on normal examples, and a pre-trained feature extractor.…”