Outlier exposure with confidence control for out-of-distribution detection

Papadopoulos, Aristotelis-Angelos; Rajati, Mohammad Reza; Shaikh, Nazim; Wang, Jiamian

doi:10.1016/j.neucom.2021.02.007

Cited by 56 publications

(28 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We build upon the existing leading methods: Baseline (Hendrycks and Gimpel, 2017), ODIN (Liang et al, 2018), Gram (Sastry and Oore, 2020), OECC (Papadopoulos et al, 2021), and Energy (Liu et al, 2020). We use the following pretrained models: ResNet-34 (He et al, 2016), DenseNet-BC- Table 3 shows the results of the Energy method and our method when combined with the pretrained Energy model (Energy+pNML) with WideResNet-40 model on CIFAR-100 and CIFAR-10 as IND sets.…”

Section: Resultsmentioning

confidence: 99%

“…The Energy method (Liu et al, 2020) adds a cost function to the training phase to shape the energy surface explicitly for OOD detection. Papadopoulos et al (2021) suggested the outlier exposure with confidence control (OECC) method in which a regularization term is added to the loss function such that the model produces a uniform distribution for OOD samples.…”

Section: Dnn Adaptationmentioning

confidence: 99%

“…However, they do not consider the test sample that is presented to the model (Jiang et al, 2020), which makes them useless for OOD detection. Other approaches build heuristics over the empirical risk minimization (ERM) learner, by post-processing the model output (Sastry and Oore, 2020) or modifying the training process (Papadopoulos et al, 2021;Vyas et al, 2018). Regardless of the approach, these methods choose the learner that minimizes the loss over the training set.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detection

Bibas¹,

Feder²,

Hassner³

2021

Preprint

View full text Add to dashboard Cite

Detecting out-of-distribution (OOD) samples is vital for developing machine learning based models for critical safety systems. Common approaches for OOD detection assume access to some OOD samples during training which may not be available in a real-life scenario. Instead, we utilize the predictive normalized maximum likelihood (pNML) learner, in which no assumptions are made on the tested input. We derive an explicit expression of the pNML and its generalization error, denoted as the regret, for a single layer neural network (NN). We show that this learner generalizes well when (i) the test vector resides in a subspace spanned by the eigenvectors associated with the large eigenvalues of the empirical correlation matrix of the training data, or (ii) the test sample is far from the decision boundary. Furthermore, we describe how to efficiently apply the derived pNML regret to any pretrained deep NN, by employing the explicit pNML for the last layer, followed by the softmax function. Applying the derived regret to deep NN requires neither additional tunable parameters nor extra data. We extensively evaluate our approach on 74 OOD detection benchmarks using DenseNet-100, ResNet-34, and WideResNet-40 models trained with CIFAR-100, CIFAR-10, SVHN, and ImageNet-30 showing a significant improvement of up to 15.6% over recent leading methods.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Dnn Adaptationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detection

Bibas¹,

Feder²,

Hassner³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…By training one class versus all the other classes, the DNN learns in some sense the out of distribution classes, however with the significant advantage of not relying on explicitly provided OOD data, in contrast to other strategies [46], [53]. Thanks to this strategy, the DNN learns to better distinguish between objects from known classes and unknown objects from classes not seen during training.…”

Section: From One Versus All (Ova) To Ovnnimentioning

confidence: 99%

“…We train it on AVA logits. Note that we have not compared our OVNNI to techniques trained to learn OOD such as [46], [53], since in these case the OOD data are in the training set, making this technique able to detect just with trained OOD data. To balance OVA training which typically has more samples available for the "All" class, we use weighted cross-entropy to train for each class, with weights for a given class based on 1 − τ class , where τ class is the proportion of data samples of this class in the training set.…”

Section: A Experimental Protocolmentioning

confidence: 99%

One Versus All for Deep Neural Network for Uncertainty (OVNNI) Quantification

Franchi¹,

Bursuc²,

Aldea³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Deep neural networks (DNNs) are powerful learning models yet their results are not always reliable. This drawback results from the fact that modern DNNs are usually overconfident, and consequently their epistemic uncertainty cannot be straightforwardly characterized. In this work, we propose a new technique to quantify the epistemic uncertainty of data easily. This method consists in mixing the predictions of an ensemble of DNNs trained to classify One class vs. All the other classes (OVA) with predictions from a standard DNN trained to perform All vs. All (AVA) classification. First of all, the adjustment provided by the AVA DNN to the score of the base classifiers allows for a more fine-grained inter-class separation. Moreover, the two types of classifiers enforce mutually their detection of out-of-distribution (OOD) samples, circumventing entirely the requirement of using such samples during training. The additional cost involved by the construction of the ensemble is offset by the ease of use of our proposed strategy and by its enhanced generalization potential, as it does not bind its performance in a given context to specific OOD datasets. The extensive experiments confirm the wide applicability of our approach, and our method achieves state of the art performance in quantifying OOD data across multiple datasets and architectures while requiring little hyper-parameter tuning.INDEX TERMS Uncertainty estimation, DNN ensembles, one vs all classification, all vs all classification.

show abstract

ATOM: Robustifying Out-of-Distribution Detection Using Outlier Mining

Chen

et al. 2021

Machine Learning and Knowledge Discovery in Databases. Research Track

View full text Add to dashboard Cite

Outlier exposure with confidence control for out-of-distribution detection

Cited by 56 publications

References 6 publications

Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detection

Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detection

One Versus All for Deep Neural Network for Uncertainty (OVNNI) Quantification

ATOM: Robustifying Out-of-Distribution Detection Using Outlier Mining

Contact Info

Product

Resources

About