Entropy Maximization and Meta Classification for Out-Of-Distribution Detection in Semantic Segmentation

Abstract

Deep neural networks (DNNs) for the semantic segmentation of images are usually trained to operate on a predefined closed set of object classes. This is in contrast to the "open world" setting where DNNs are envisioned to be deployed to. From a functional safety point of view, the ability to detect so-called "out-of-distribution" (OoD) samples, i.e., objects outside of a DNN's semantic space, is crucial for many applications such as automated driving. A natural baseline approach to OoD detection is to threshold on the pixel-wise softmax entropy. We present a two-step procedure that significantly improves that approach. Firstly, we utilize samples from the COCO dataset as OoD proxy and introduce a second training objective to maximize the softmax entropy on these samples. Starting from pretrained semantic segmentation networks we re-train a number of DNNs on different in-distribution datasets and consistently observe improved OoD detection performance when evaluating on completely disjoint OoD datasets. Secondly, we perform a transparent post-processing step to discard false positive OoD samples by so-called "meta classification". To this end, we apply linear models to a set of hand-crafted metrics derived from the DNN's softmax probabilities. In our experiments we consistently observe a clear additional gain in OoD detection performance, cutting down the number of detection errors by 52% when comparing the best baseline with our results. We achieve this improvement sacrificing only marginally in original segmentation performance. Therefore, our method contributes to safer DNNs with more reliable overall system performance.

An Application-Driven Conceptualization of Corner Cases for Perception in Highly Automated Driving

“…Here, we aim to provide a brief overview of existing and applied methods to contribute to harmonizing this field of research. For corner case detection on camera images, often, the area under the receiver operator characteristic (AUC) is used to determine separability between a normal and anomalous corner case class (e.g., [50]). Moreover, the area under the precision-recall curve (AUPRC) determines the separability of one class, e.g., the corner case class [50].…”

“…For corner case detection on camera images, often, the area under the receiver operator characteristic (AUC) is used to determine separability between a normal and anomalous corner case class (e.g., [50]). Moreover, the area under the precision-recall curve (AUPRC) determines the separability of one class, e.g., the corner case class [50]. One of few existing online benchmarks related to corner case detection on automotive camera images, the Fishyscapes online benchmark [37], reports results in the average precision (AP) and the false positive rate at 95% true positive rate (FPR 95 ).…”

Standardized Max Logits: A Simple yet Effective Approach for Identifying Unexpected Road Obstacles in Urban-Scene Segmentation

“…Several studies [2,5,3] utilize samples of unexpected objects from external datasets during the training phase. For example, by assuming that the objects cropped from Im-ageNet dataset [32] are anomalous objects, they are overlaid on original training images [2] (e.g., Cityscapes) to provide samples of unexpected objects.…”

“…For example, by assuming that the objects cropped from Im-ageNet dataset [32] are anomalous objects, they are overlaid on original training images [2] (e.g., Cityscapes) to provide samples of unexpected objects. Similarly, another previous work [5] utilizes the objects from COCO dataset [21] as samples of unexpected objects. However, such methods require retraining the network by using the additional datasets, which hampers to utilize a given pre-trained segmentation network directly.…”

“…Several studies [3,23,22,5,31,2,13] tackle the problem of detecting such unexpected objects on roads. Some approaches [2,5] utilize external datasets [32,21] as samples of unexpected objects while others [23,35,22,29] leverage image resynthesis models for erasing the regions of such objects. However, such approaches require a considerable amount of labor intensity or necessitate a lengthy inference time.…”