A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges

Abstract

Machine learning models often encounter samples that are diverged from the training distribution. Failure to recognize an out-of-distribution (OOD) sample, and consequently assign that sample to an in-class label significantly compromises the reliability of a model. The problem has gained significant attention due to its importance for safety deploying models in open-world settings. Detecting OOD samples is challenging due to the intractability of modeling all possible unknown distributions. To date, several research domains tackle the problem of detecting unfamiliar samples, including anomaly detection, novelty detection, one-class learning, open set recognition, and out-of-distribution detection. Despite having similar and shared concepts, out-of-distribution, open-set, and anomaly detection have been investigated independently. Accordingly, these research avenues have not cross-pollinated, creating research barriers. While some surveys intend to provide an overview of these approaches, they seem to only focus on a specific domain without examining the relationship between different domains. This survey aims to provide a cross-domain and comprehensive review of numerous eminent works in respective areas while identifying their commonalities. Researchers can benefit from the overview of research advances in different fields and develop future methodology synergistically. Furthermore, to the best of our knowledge, while there are surveys in anomaly detection or one-class learning, there is no comprehensive or up-to-date survey on out-of-distribution detection, which our survey covers extensively. Finally, having a unified cross-domain perspective, we discuss and shed light on future lines of research, intending to bring these fields closer together.

On the link between conscious function and general intelligence in humans and machines

“…To make this more concrete, we consider the notion of "out-of-distribution" (OoD) generalization (Salehi et al, 2021). A typical assumption within machine learning is that the distribution of the test data matches that of the data used to train a model.…”

“…When examining the three proposed theories of conscious function, we find that each describes a system which supports greater domain-general skill acquisition, and thus greater intelligence by our adopted definition. Utilizing this insight, we turn to current state-of-the-art AI systems, and find that systems which utilize specific aspects of each theory can result in greater generalization than other similar models (Hendrycks et al, 2021). Other prominent lines of research to address limitations in general intelligence of artificial agents include research into modularity and causality, learning world models, and meta-learning.…”

A Survey on Unsupervised Industrial Anomaly Detection Algorithms

“…The article [83] focuses on the distinction between several different domain concepts like anomaly detection (AD), novelty detection (ND), open set recognition (OSR), outlier detection (OD), and Out-of-Distribution detection (OOD). The article [84] focuses on the intersection of different research fields, providing extended cross-cutting ideas, exhaustively introducing the algorithms and frameworks of some typical methods, de-emphasizing methodological schools and blurring domain boundaries. It intends to bring these fields closer together.…”

Understanding the Challenges When 3D Semantic Segmentation Faces Class Imbalanced and OOD Data

“…The problem is very severe for safety-critical applications such as autonomous driving, where some of the categories are unseen or rare in the datasets but need to be handled in the system [13]. OOD detection has also been studied as a general problem in machine learning [14]. However, the task is more difficult when 3DSS is faced with the dual challenges from class imbalances and OOD data, where the model could show a high confidence for wrong predictions [15] on either the object's semantic class or the judgment of its in-distribution (ID) or OOD.…”

“…Learning models for autonomous driving and robotics applications need to address the open-world problem [54], which requires the models to deal with both the seen (ID) and unseen (OOD) objects in the training datasets. OOD detectors [14] have been developed as an auxiliary module combined with the main task model for this purpose. Several mainstream methods for OOD detection have been developed.…”