Entropy Maximization and Meta Classification for Out-of-Distribution Detection in Semantic Segmentation

Chan, Robin; Rottmann, Matthias; Gottschalk, Hanno

doi:10.1109/iccv48922.2021.00508

Cited by 77 publications

(55 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In road traffic, the detection of new and unknown objects, anomalies or obstacles, which must also be evaluated as 'outside the operating parameters', is essential. To measure the performance of methods for detecting such objects, the benchmark suite "SegmentMeIfYouCan" was created [7], [8]. In addition, the authors present two datasets for anomaly and obstacle segmentation to help autonomous vehicles better assess safetycritical situations.…”

Section: A Corner Casesmentioning

confidence: 99%

A-Eye: Driving with the Eyes of AI for Corner Case Generation

Kowol¹,

Bracke²,

Gottschalk³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

The overall goal of this work is to enrich training data for automated driving with so called corner cases. In road traffic, corner cases are critical, rare and unusual situations that challenge the perception by AI algorithms. For this purpose, we present the design of a test rig to generate synthetic corner cases using a human-in-the-loop approach. For the test rig, a realtime semantic segmentation network is trained and integrated into the driving simulation software CARLA in such a way that a human can drive on the network's prediction. In addition, a second person gets to see the same scene from the original CARLA output and is supposed to intervene with the help of a second control unit as soon as the semantic driver shows dangerous driving behavior. Interventions potentially indicate poor recognition of a critical scene by the segmentation network and then represents a corner case. In our experiments, we show that targeted enrichment of training data with corner cases leads to improvements in pedestrian detection in safety relevant episodes in road traffic.

show abstract

Section: A Corner Casesmentioning

confidence: 99%

A-Eye: Driving with the Eyes of AI for Corner Case Generation

Kowol¹,

Bracke²,

Gottschalk³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Concerning computational costs, it is preferred to approximate Bayesian inference using Monte Carlo dropout [19,44] or ensembles [31]. Uncertaintybased approaches can be further improved by integrating anomalous data into the training procedure [8,14]. Another line of works employs generative models such as autoencoders (AEs) [2,3,12,36] or generative adversarial models (GANs) [1,47,55,70] to reconstruct or synthesise images and measure the reconstruction quality.…”

Section: Related Workmentioning

confidence: 99%

Towards Unsupervised Open World Semantic Segmentation

Svenja¹,

Rottmann²,

Gottschalk³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

For the semantic segmentation of images, state-of-theart deep neural networks (DNNs) achieve high segmentation accuracy if that task is restricted to a closed set of classes. However, as of now DNNs have limited ability to operate in an open world, where they are tasked to identify pixels belonging to unknown objects and eventually to learn novel classes, incrementally. Humans have the capability to say: "I don't know what that is, but I've already seen something like that". Therefore, it is desirable to perform such an incremental learning task in an unsupervised fashion. We introduce a method where unknown objects are clustered based on visual similarity. Those clusters are utilized to define new classes and serve as training data for unsupervised incremental learning. More precisely, the connected components of a predicted semantic segmentation are assessed by a segmentation quality estimate. connected components with a low estimated prediction quality are candidates for a subsequent clustering. Additionally, the component-wise quality assessment allows for obtaining predicted segmentation masks for the image regions potentially containing unknown objects. The respective pixels of such masks are pseudo-labeled and afterwards used for retraining the DNN, i.e. without the use of ground truth generated by humans. In our experiments we demonstrate that, without access to ground truth and even with few data, a DNN's class space can be extended by a novel class, achieving considerable segmentation accuracy.

show abstract

“…This will likely introduce some inlier content into negative images, however the resulting noise can be alleviated by careful batch composition. The training can be implemented through a separate OOD head [15] or by maximizing softmax entropy in negative pixels [13]. Recent work shows that anomaly detector can also be trained on instance classes of an auxiliary semantic segmentation dataset [13].…”

Section: Dense Open-set Recognitionmentioning

confidence: 99%

“…The training can be implemented through a separate OOD head [15] or by maximizing softmax entropy in negative pixels [13]. Recent work shows that anomaly detector can also be trained on instance classes of an auxiliary semantic segmentation dataset [13].…”

Section: Dense Open-set Recognitionmentioning

confidence: 99%

See 1 more Smart Citation

Dense anomaly detection by robust learning on synthetic negative data

Grcić¹,

Bevandić²,

Šegvić³

2021

Preprint

View full text Add to dashboard Cite

Standard machine learning is unable to accommodate inputs which do not belong to the training distribution. The resulting models often give rise to confident incorrect predictions which may lead to devastating consequences. This problem is especially demanding in the context of dense prediction since input images may be only partially anomalous. Previous work has addressed dense anomaly detection by discriminative training on mixed-content images. We extend this approach with synthetic negative patches which simultaneously achieve high inlier likelihood and uniform discriminative prediction. We generate synthetic negatives with normalizing flows due to their outstanding distribution coverage and capability to generate samples at different resolutions. We also propose to detect anomalies according to a principled information-theoretic criterion which can be consistently applied through training and inference. The resulting models set the new state of the art on standard benchmarks and datasets in spite of minimal computational overhead and refraining from auxiliary negative data.

show abstract

Entropy Maximization and Meta Classification for Out-of-Distribution Detection in Semantic Segmentation

Cited by 77 publications

References 26 publications

A-Eye: Driving with the Eyes of AI for Corner Case Generation

A-Eye: Driving with the Eyes of AI for Corner Case Generation

Towards Unsupervised Open World Semantic Segmentation

Dense anomaly detection by robust learning on synthetic negative data

Contact Info

Product

Resources

About