Hybrid Models for Open Set Recognition

Zhang, Hongjie; Li, Ang; Guo, Jie; Guo, Yanwen

doi:10.1007/978-3-030-58580-8_7

Cited by 99 publications

(66 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Jiang et al (2018) propose to use scores based on the relative distance of the predicted class to the second class. Recently, residual flow-based methods were used to obtain a density model for OOD detection (Zhang et al, 2020). Ji et al (2021) proposed a method based on subfunction error bounds to compute unreliability per sample.…”

Section: Prior Workmentioning

confidence: 99%

Leveraging Unlabeled Data to Predict Out-of-Distribution Performance

Garg¹,

Balakrishnan²,

Neyshabur³

et al. 2022

Preprint

View full text Add to dashboard Cite

Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions that may cause performance drops. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples for which model confidence exceeds that threshold. ATC outperforms previous methods across several model architectures, types of distribution shifts (e.g., due to synthetic corruptions, dataset reproduction, or novel subpopulations), and datasets (WILDS, ImageNet, BREEDS, CIFAR, and MNIST). In our experiments, ATC estimates target performance 2-4ˆmore accurately than prior methods. We also explore the theoretical foundations of the problem, proving that, in general, identifying the accuracy is just as hard as identifying the optimal predictor and thus, the efficacy of any method rests upon (perhaps unstated) assumptions on the nature of the shift. Finally, analyzing our method on some toy distributions, we provide insights concerning when it works.

show abstract

Section: Prior Workmentioning

confidence: 99%

Leveraging Unlabeled Data to Predict Out-of-Distribution Performance

Garg¹,

Balakrishnan²,

Neyshabur³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Density-based anomaly detection is applied in practice [ 25 , 28 , 34 , 41 , 42 , 43 ] as follows: first, learn a density estimator

to approximate the data density

, and then plug that estimate in the density-based methods from Section 2.2 and Section 2.3 to discriminate between inliers and outliers. Recent empirical failures [ 3 , 26 , 27 ] of this procedure applied to density scoring have been attributed to the discrepancy between

and

[ 28 , 33 , 34 , 35 , 48 ]. Instead, we choose in this paper to question the fundamental assumption that these density-based methods should result in a correct classification between outliers and inliers.…”

Section: The Role Of Reparametrizationmentioning

confidence: 99%

“…For instance [ 26 , 27 , 28 ] noticed that generative models trained on a benchmark dataset (e.g., CIFAR-10, [ 29 ]) and tested on another (e.g., SVHN, [ 30 ]) are not able to identify the latter as an outlier with current methods. Different hypotheses have been formulated to explain that discrepancy, ranging from the curse of dimensionality [ 31 ] to a significant mismatch between

and

[ 26 , 32 , 33 , 34 , 35 , 36 ].…”

Section: Introductionmentioning

confidence: 99%

Perfect Density Models Cannot Guarantee Anomaly Detection

Lan

Dinh

2021

Entropy

View full text Add to dashboard Cite

Thanks to the tractability of their likelihood, several deep generative models show promise for seemingly straightforward but important applications like anomaly detection, uncertainty estimation, and active learning. However, the likelihood values empirically attributed to anomalies conflict with the expectations these proposed applications suggest. In this paper, we take a closer look at the behavior of distribution densities through the lens of reparametrization and show that these quantities carry less meaningful information than previously thought, beyond estimation issues or the curse of dimensionality. We conclude that the use of these likelihoods for anomaly detection relies on strong and implicit hypotheses, and highlight the necessity of explicitly formulating these assumptions for reliable anomaly detection.

show abstract

“…Improving OOD detection has seen progress by training generative models [6; 7; 2; 8], and modifying objective and loss functions [9]. Exposure to a number of OOD samples during training has also lead to improvements [10].…”

Section: Introductionmentioning

confidence: 99%

Adversarial vulnerability of powerful near out-of-distribution detection

Fort¹

2022

Preprint

View full text Add to dashboard Cite

There has been a significant progress in detecting out-of-distribution (OOD) inputs in neural networks recently, primarily due to the use of large models pretrained on large datasets, and an emerging use of multi-modality. We show a severe adversarial vulnerability of even the strongest current OOD detection techniques. With a small, targeted perturbation to the input pixels, we can change the image assignment from an in-distribution to an out-distribution, and vice versa, easily. In particular, we demonstrate severe adversarial vulnerability on the challenging near OOD CIFAR-100 vs CIFAR-10 task, as well as on the far OOD CIFAR-100 vs SVHN. We study the adversarial robustness of several post-processing techniques, including the simple baseline of Maximum of Softmax Probabilities (MSP), the Mahalanobis distance, and the newly proposed Relative Mahalanobis distance. By comparing the loss of OOD detection performance at various perturbation strengths, we demonstrate the beneficial effect of using ensembles of OOD detectors, and the use of the Relative Mahalanobis distance over other postprocessing methods. In addition, we show that even strong zero-shot OOD detection using CLIP and multi-modality suffers from a severe lack of adversarial robustness as well. Our code is available on GitHub.

show abstract

Hybrid Models for Open Set Recognition

Cited by 99 publications

References 14 publications

Leveraging Unlabeled Data to Predict Out-of-Distribution Performance

Leveraging Unlabeled Data to Predict Out-of-Distribution Performance

Perfect Density Models Cannot Guarantee Anomaly Detection

Adversarial vulnerability of powerful near out-of-distribution detection

Contact Info

Product

Resources

About