A Normalized Gaussian Wasserstein Distance for Tiny Object Detection

Abstract

Detecting tiny objects is a very challenging problem since a tiny object only contains a few pixels in size. We demonstrate that state-of-the-art detectors do not produce satisfactory results on tiny objects due to the lack of appearance information. Our key observation is that Intersection over Union (IoU) based metrics such as IoU itself and its extensions are very sensitive to the location deviation of the tiny objects, and drastically deteriorate the detection performance when used in anchor-based detectors. To alleviate this, we propose a new evaluation metric using Wasserstein distance for tiny object detection. Specifically, we first model the bounding boxes as 2D Gaussian distributions and then propose a new metric dubbed Normalized Wasserstein Distance (NWD) to compute the similarity between them by their corresponding Gaussian distributions. The proposed NWD metric can be easily embedded into the assignment, non-maximum suppression, and loss function of any anchor-based detector to replace the commonly used IoU metric. We evaluate our metric on a new dataset for tiny object detection (AI-TOD) in which the average object size is much smaller than existing object detection datasets. Extensive experiments show that, when equipped with NWD metric, our approach yields performance that is 6.7 AP points higher than a standard fine-tuning baseline, and 6.0 AP points higher than state-of-the-art competitors.

A Guide to Image and Video based Small Object Detection using Deep Learning : Case Study of Maritime Surveillance

“…We can cite RetinaNet [39], which is designed to focus on the most challenging samples (e.g., small objects) by multiplying a term proportional to the network's confidence into the classical cross-entropy loss. Other methods modify the standard IoU loss, including Intersection over Detection, Generalized IoU [141], Wasserstein distance [142], and Complete IoU [143]; The detailed explanation of these methods can be found in Section 6.2.1.…”

“…In this case, the ER is defined as the total number of miss classified pixels over the total number of pixels. Normalized Wasserstein Distance (NWD) [142]: As opposed to the aforementioned metrics, which treat bounding boxes as deterministic variables, here the bounding boxes are represented by multivariate Gaussian densities. The similarity is then calculated by an exponential function of the existing Optimal Transport (OT) theory (i.e., Wasserstein distance).…”

Towards Large-Scale Small Object Detection: Survey and Benchmarks

“…FPN [2] PANet [74] TDM [84] DarkNet-RI [85] SNIP [78] Sniper [79] AutoFocus [80] DST [81] YOLOv3 [45] SSD [49] HyperNet [66] MS-CNN [68] DSFD [69] M2Det [70] SSH [72] TridentNet [73] MFR-CNN [75] QueryDet [76] RCNN for SOD [28] PyramidBox [105] FS-SSD [109] SINet [110] IONet [111] R2CNN [129] SCRDet [130] FBR-Net [131] TinaFace [132] NWD [135] CDMNet [57] ClusDet [136] DMNet [137] TLL [138] SML [139] Augmentation for SOD [53] RRNet [54] YOLOv4 [55] AMRNet [56] LearningDataAugmentation [58] Randaugment [59] MTGAN [98] FLSR [101] PerceptualGAN [103] STDN [82] DSSD [89] EFPN [96] StairNet [86] Fusion Factor [87] SSPNet…”

“…Observing that IoU metric changes drastically due to the slight offsets of predicted boxes for tiny objects, Xu et al [134] proposed a novel metric, Dot Distance, to alleviate this situation. Similarly, NWD [135] introduces the Normalized Wasserstein Distance to optimize the location metric for tiny object detectors.…”

MStrans: Multiscale Vision Transformer for Aerial Objects Detection

“…The high percentage of tiny instances and the multi-scale multidirectional instances with arbitrary aspect ratios make this task more challenging. Inspired by the dense object detection frameworks, Chen et al [39] [41] proposed a new indicator (Normalized Wasserstein Distance, NWD) by replacing the standard IoU with Wasserstein Distance, which improves the detection efficiency of anchor-based object detectors for tiny aerial object detection. Meanwhile, the research to handle the aerial objects of multiple scales, directions, and aspect ratios is also conducted.…”