MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space

Huang, Rui; Li, Yixuan

doi:10.48550/arxiv.2105.01879

Cited by 3 publications

(1 citation statement)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unknown classes detection: we use two challenging benchmarks for the detection of unknown classes. iNaturalist using the subset in Huang & Li (2021) made of plants with classes that do not intersect ImageNet. Wang et al (2022) noted that this dataset is particularly challenging due to proximity of its classes.…”

Section: Methodsmentioning

confidence: 99%

CADet: Fully Self-Supervised Anomaly Detection With Contrastive Learning

Guille-Escuret¹,

Rodríguez²,

Vázquez³

et al. 2022

Preprint

View full text Add to dashboard Cite

Handling out-of-distribution (OOD) samples has become a major stake in the realworld deployment of machine learning systems. This work explores the application of self-supervised contrastive learning to the simultaneous detection of two types of OOD samples: unseen classes and adversarial perturbations. Since in practice the distribution of such samples is not known in advance, we do not assume access to OOD examples. We show that similarity functions trained with contrastive learning can be leveraged with the maximum mean discrepancy (MMD) two-sample test to verify whether two independent sets of samples are drawn from the same distribution. Inspired by this approach, we introduce CADet (Contrastive Anomaly Detection), a method based on image augmentations to perform anomaly detection on single samples. CADet compares favorably to adversarial detection methods to detect adversarially perturbed samples on ImageNet. Simultaneously, it achieves comparable performance to unseen label detection methods on two challenging benchmarks: ImageNet-O and iNaturalist. CADet is fully self-supervised and requires neither labels for in-distribution samples nor access to OOD examples. IntroductionWhile modern machine learning systems have achieved countless successful real-world applications, handling out-of-distribution (OOD) inputs remains a tough challenge of significant importance. The problem is especially acute for high-dimensional problems like image classification. Models are typically trained in a close-world setting but inevitably faced with novel input classes when deployed in the real world. The impact can range from displeasing customer experience to dire consequences in the case of safety-critical applications such as autonomous driving (Kitt et al., 2010) Preprint. Under review.

show abstract

Section: Methodsmentioning

confidence: 99%

CADet: Fully Self-Supervised Anomaly Detection With Contrastive Learning

Guille-Escuret¹,

Rodríguez²,

Vázquez³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

On the Impact of Spurious Correlation for Out-of-Distribution Detection

Yifei

Yin

2022

AAAI

View full text Add to dashboard Cite

Modern neural networks can assign high confidence to inputs drawn from outside the training distribution, posing threats to models in real-world deployments. While much research attention has been placed on designing new out-of-distribution (OOD) detection methods, the precise definition of OOD is often left in vagueness and falls short of the desired notion of OOD in reality. In this paper, we present a new formalization and model the data shifts by taking into account both the invariant and environmental (spurious) features. Under such formalization, we systematically investigate how spurious correlation in the training set impacts OOD detection. Our results suggest that the detection performance is severely worsened when the correlation between spurious features and labels is increased in the training set. We further show insights on detection methods that are more effective in reducing the impact of spurious correlation, and provide theoretical analysis on why reliance on environmental features leads to high OOD detection error. Our work aims to facilitate better understanding of OOD samples and their formalization, as well as the exploration of methods that enhance OOD detection. Code is available at https://github.com/deeplearning-wisc/Spurious_OOD.

show abstract