SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation

Abstract

State-of-the-art semantic or instance segmentation deep neural networks (DNNs) are usually trained on a closed set of semantic classes. As such, they are ill-equipped to handle previously-unseen objects. However, detecting and localizing such objects is crucial for safety-critical applications such as perception for automated driving, especially if they appear on the road ahead. While some methods have tackled the tasks of anomalous or out-of-distribution object segmentation, progress remains slow, in large part due to the lack of solid benchmarks; existing datasets either consist of synthetic data, or suffer from label inconsistencies. In this paper, we bridge this gap by introducing the "Seg-mentMeIfYouCan" benchmark. Our benchmark addresses two tasks: Anomalous object segmentation, which considers any previously-unseen object category; and road obstacle segmentation, which focuses on any object on the road, may it be known or unknown. We provide two corresponding datasets together with a test suite performing an in-depth method analysis, considering both established pixel-wise performance metrics and recent component-wise ones, which are insensitive to object sizes. We empirically evaluate multiple state-of-the-art baseline methods, including several specifically designed for anomaly / obstacle segmentation, on our datasets as well as on public ones, using our benchmark suite. The anomaly and obstacle segmentation results show that our datasets contribute to the diversity and challengingness of both dataset landscapes.

Self-Supervised Variational Autoencoder for Unsupervised Object Counting from Very-High-Resolution Satellite Imagery: Applications in Dwelling Extraction in FDP Settlement Areas

“…5, the implemented approaches, especially self-supervision and latent space conditioning, have yielded better localization and counting of dwellings. Compared to recent anomaly detection works [86]- [88], though the datasets and anomaly localization pipelines are different from the current study, our approach has reached the best localization performance where AUC values reach approximately 98% for some datasets (e.g. see Table II for Nguenygiel-2017 and Minawao-2016 datasets).…”

Statistical modeling of deep features reduces false alarms in video change detection

“…In one hand, we take the approach of Chan et al [42] and use a variation of the traditional intersection over union (IoU) first introduced by Rottman et al [43]. Unlike the conventional IoU, which penalizes cases where a ground truth region is fragmented into multiple predictions by assigning each prediction a moderate IoU score, the adapted metric, named sIoU, does not penalize predictions of a segment when the remaining ground truth is sufficiently covered by other predicted segments.…”

A Survey of FPGA-Based Vision Systems for Autonomous Cars

“…Some specific datasets for obstacle detection are available: Lost and Found [205], Street Hazards [206] and RoadObsta-cle21 [207]). They typically provide a sequences of images and a corresponding depth map from the scene.…”