What's in the Black Box? The False Negative Mechanisms Inside Object Detectors

Abstract

In object detection, false negatives arise when a detector fails to detect a target object. To understand why object detectors produce false negatives, we identify five 'false negative mechanisms', where each mechanism describes how a specific component inside the detector architecture failed. Focusing on two-stage and one-stage anchor-box object detector architectures, we introduce a framework for quantifying these false negative mechanisms. Using this framework, we investigate why Faster R-CNN and RetinaNet fail to detect objects in benchmark vision datasets and robotics datasets. We show that a detector's false negative mechanisms differ significantly between computer vision benchmark datasets and robotics deployment scenarios. This has implications for the translation of object detectors developed for benchmark datasets to robotics applications.

Monitoring of Perception Systems: Deterministic, Probabilistic, and Learning-based Fault Detection and Identification

“…Previous works on runtime fault detection and identification focused on components of the perception system [51]. Miller et al [52] propose a framework for quantifying false negatives in object detection. For semantic segmentation, Besnier et al [13] propose an out-of-distribution detection mechanism, while Rahman et al [53] propose a failure detection framework to identify pixellevel misclassifications.…”