Ensemble-Based Out-of-Distribution Detection

Abstract

To design an efficient deep learning model that can be used in the real-world, it is important to detect out-of-distribution (OOD) data well. Various studies have been conducted to solve the OOD problem. The current state-of-the-art approach uses a confidence score based on the Mahalanobis distance in a feature space. Although it outperformed the previous approaches, the results were sensitive to the quality of the trained model and the dataset complexity. Herein, we propose a novel OOD detection method that can train more efficient feature space for OOD detection. The proposed method uses an ensemble of the features trained using the softmax-based classifier and the network based on distance metric learning (DML). Through the complementary interaction of these two networks, the trained feature space has a more clumped distribution and can fit well on the Gaussian distribution by class. Therefore, OOD data can be efficiently detected by setting a threshold in the trained feature space. To evaluate the proposed method, we applied our method to various combinations of image datasets. The results show that the overall performance of the proposed approach is superior to those of other methods, including the state-of-the-art approach, on any combination of datasets.

Out-of-Distribution (OOD) Detection and Generalization Improved by Augmenting Adversarial Mixup Samples

“…Additional OOD detection studies [20][21][22] have recently been proposed to focus only on detecting semantic shifts, such as most previous OOD detectors sensitive to covariate shift. This paper proposes a distance-aware AM method detecting semantic OOD samples while being robust to negligible covariate shift.…”

“…Lots of previous studies have been conducted in various directions to solve the problems related to OOD detection and OOD generalization [11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28]. For OOD detection, a decision boundary that distinguishes OOD data by training the model is automatically discovered to induce a low confidence score of OOD data, thus producing reliable inference results.…”

“…Several training or inference methods have recently been studied to solve the above problems of OOD detection and OOD generalization [11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28]. However, most of the previous works only focus on one side of distributional shifts, either semantic OOD detection or covariate OOD generalization.…”

Implementation and Validation of non-semantic Out-of-Distribution Detection on Image Data in Manufacturing

“…In recent years, with the widespread application of deep learning techniques, some methods for detecting OOD in high-dimensional images have been investigated. In 2021, in a survey on OOD detection methods published in [22], the authors summarized various methods for detecting OOD, such as Post-hoc Detection [20][12], Confidence Enhancement [14], outlier exposure [8], etc., which are applicable to detecting semantic OOD in multidimensional images. Another approach by combining statistical hypothesis testing with feature extraction methods for detecting drift in high-dimensional image distributions was proposed in [17].…”

“…Another approach by combining statistical hypothesis testing with feature extraction methods for detecting drift in high-dimensional image distributions was proposed in [17]. The performance of this method in recognizing semantic OODs was validated on an academic benchmark [22], using CIFAR-10 as In-Distribution (ID) in training and distinguishing CIFAR images from other datasets such as MNIST [17].…”