One Versus All for Deep Neural Network for Uncertainty (OVNNI) Quantification

Abstract

Deep neural networks (DNNs) are powerful learning models yet their results are not always reliable. This drawback results from the fact that modern DNNs are usually overconfident, and consequently their epistemic uncertainty cannot be straightforwardly characterized. In this work, we propose a new technique to quantify the epistemic uncertainty of data easily. This method consists in mixing the predictions of an ensemble of DNNs trained to classify One class vs. All the other classes (OVA) with predictions from a standard DNN trained to perform All vs. All (AVA) classification. First of all, the adjustment provided by the AVA DNN to the score of the base classifiers allows for a more fine-grained inter-class separation. Moreover, the two types of classifiers enforce mutually their detection of out-of-distribution (OOD) samples, circumventing entirely the requirement of using such samples during training. The additional cost involved by the construction of the ensemble is offset by the ease of use of our proposed strategy and by its enhanced generalization potential, as it does not bind its performance in a given context to specific OOD datasets. The extensive experiments confirm the wide applicability of our approach, and our method achieves state of the art performance in quantifying OOD data across multiple datasets and architectures while requiring little hyper-parameter tuning.INDEX TERMS Uncertainty estimation, DNN ensembles, one vs all classification, all vs all classification.

On Monocular Depth Estimation and Uncertainty Quantification Using Classification Approaches for Regression

“…6 is much smaller, which results in the evener distributed labels, and we consider this is the main cause of its worse performance. Yang et al [9] based model outperforms the others, which indicates that the Multi-BCE loss is more suitable for uncertainty estimation, which is similar to the one-versus-all strategy [29].…”

On Monocular Depth Estimation and Uncertainty Quantification using Classification Approaches for Regression

“…6 is much smaller, which results in the evener distributed labels, and we consider this is the main cause of its worse performance. Yang et al [9] based model outperforms the others, which indicates that the Multi-BCE loss is more suitable for uncertainty estimation, which is similar to the one-versus-all strategy [29].…”

UQGAN: A Unified Model for Uncertainty Quantification of Deep Classifiers trained via Conditional GANs

“…One-vs-All methods in the context of OoD detection have been recently studied by Franchi et al [2020], Padhy et al [2020], Saito and Saenko [2021]. In the work by Franchi et al [2020] an ensemble of binary neural networks is trained to perform one-vs-all classification on the in-distribution data which are then weighted by a standard softmax classifier.…”

“…One-vs-All methods in the context of OoD detection have been recently studied by Franchi et al [2020], Padhy et al [2020], Saito and Saenko [2021]. In the work by Franchi et al [2020] an ensemble of binary neural networks is trained to perform one-vs-all classification on the in-distribution data which are then weighted by a standard softmax classifier. Padhy et al [2020] use a DNN with a single sigmoid binary output for every class and explore the possibility of training the one-vs-all network with a distance based loss function instead of the binary cross entropy.…”