CIDER: Exploiting Hyperspherical Embeddings for Out-of-Distribution Detection

Abstract

Out-of-distribution (OOD) detection is a critical task for reliable machine learning. Recent advances in representation learning give rise to developments in distance-based OOD detection, where testing samples are detected as OOD if they are relatively far away from the centroids or prototypes of in-distribution (ID) classes. However, prior methods directly take off-theshelf loss functions that suffice for classifying ID samples, but are not optimally designed for OOD detection. In this paper, we propose CIDER, a simple and effective representation learning framework by exploiting hyperspherical embeddings for OOD detection. CIDER jointly optimizes two losses to promote strong ID-OOD separability: (1) a dispersion loss that promotes large angular distances among different class prototypes, and (2) a compactness loss that encourages samples to be close to their class prototypes. We show that CIDER is effective under various settings and establishes state-of-the-art performance. On a hard OOD detection task CIFAR-100 vs. CIFAR-10, our method substantially improves the AUROC by 14.20% compared to the embeddings learned by the cross-entropy loss.

C2I-CAT: Class-to-Image Cross Attention Transformer for Out-of-Distribution Detection

“…Score-based methods measure the likelihood of how far a given sample originates from in-distribution (ID) ( [7], [8], [9], [10], [11], [12]). Training-based methods are based on representation learning in embedding space ( [13], [14]) or reduce overconfidence in a network [15] by normalizing model's logits during training. Thus, OOD performance of these methods based on both scoring functions and training heavily depends on the quality of the features or model's outputs (i.e., logits) obtained from the trained model.…”

RainProof: An Umbrella to Shield Text Generator from Out-Of-Distribution Data

“…The second one, called plug-in methods aims to distinguish regular samples in the in-distribution (IN) from OOD samples based on the model's behaviour on a new input. Plug-in methods include Maximum Softmax Probabilities (MSP) (Hendrycks and Gimpel, 2016) or Energy (Liu et al, 2020) or featurebased anomaly detectors that compute a per-class anomaly score (Ming et al, 2022;Ryu et al, 2017;Huang et al, 2020;Ren et al, 2021a). Although plug-in methods from classification settings seem attractive, their adaptation to text generation tasks is more involved.…”

Natural robustness of machine learning in the open world

“…The energy-based regularization has a direct theoretical interpretation as shaping the log-likelihood, hence naturally suits OOD detection. Contrastive learning methods are also employed for the OOD detection task [139][140][141], which can be computationally expensive.…”

“…The energy-based regularization has a direct theoretical interpretation as shaping the log-likelihood, hence naturally suits OOD detection. Contrastive learning methods are also employed for the OOD detection task [139][140][141], which can be computationally more expensive to train than ours. In this work, we focus on exploring classification-based loss functions for OOD detection, which only requires in-distribution data in training.…”