Detecting Rotated Objects as Gaussian Distributions and Its 3-D Generalization

Abstract

Existing detection methods commonly use a parameterized bounding box (BBox) to model and detect (horizontal) objects and an additional rotation angle parameter is used for rotated objects. We argue that such a mechanism has fundamental limitations in building an effective regression loss for rotation detection, especially for high-precision detection with high IoU (e.g. 0.75). Instead, we propose to model the rotated objects as Gaussian distributions. A direct advantage is that our new regression loss regarding the distance between two Gaussians e.g. Kullback-Leibler Divergence (KLD), can well align the actual detection performance metric, which is not well addressed in existing methods. Moreover, the two bottlenecks i.e. boundary discontinuity and square-like problem also disappear. We also propose an efficient Gaussian metric-based label assignment strategy to further boost the performance. Interestingly, by analyzing the BBox parameters' gradients under our Gaussian-based KLD loss, we show that these parameters are dynamically updated with interpretable physical meaning, which help explain the effectiveness of our approach, especially for high-precision detection. We extend our approach from 2-D to 3-D with a tailored algorithm design to handle the heading estimation, and experimental results on twelve public datasets (2-D/3-D, aerial/text/face images) with various base detectors show its superiority.

Oriented Cell Dataset: efficient imagery analyses using angular representation

“…To solve the discontinuous boundaries issue originated by the angular periodicity or corner ordering, Yang et al proposed the Circular Smooth Label model (CSL) [21] and the Densely Coded Labels model (DCL) [22], which transform the angular prediction task from a regression to a classification problem. Although these methods have been broadly used for detecting oriented objects in aerial/satellite images [15], [16], we are unaware of existing datasets that provide OBB annotations for brightfield microscopic images. As mentioned before, using OBBs might severely reduce the annotation burden when compared to segmentation approaches, and still provide enough geometrical information to enable cell confluence and polarity estimation, which are broadly used on biological applications [1]- [3], [8].…”

“…1a. Meanwhile, the literature regarding object detection with OBBs is recent and scarce, and focuses mostly on niche applications such as aerial/satellite imagery [15], [16]. Although HBB and OBB detectors share the same main concepts in terms of network architecture (e.g., both can be achieved using two- or one-stage methods), moving from HBBs to OBBs adds some challenges, such as the ambiguous parametrization of OBBs, the difficulty in regressing angular information, and the adaptation of anchor-based methods [16].…”

“…As in works that use aerial images [15], [16], our cell images are usually too large for training the detectors directly. Hence, we first cropped those images to 512 × 512 patches with a 100-pixels stride, resulting in a total of 5,590 patches of images to train our models.…”

Recalibrating Features and Regression for Oriented Object Detection

“…Qian et al [24] directly regressed the four vertices of a quadrangle and devised a modulated rotation loss to optimize the regression process, largely eliminating the problems of angular periodicity and bounding box boundary discontinuity [2,22]. Apart from the above strategies, a series of works [34,35] model the rotated objects as Gaussian distributions and build the regression loss with certain distributional distance measurements, e.g., Wasserstein distance and Kullback-Leibler Divergence.…”

Multi-scale rotating target detection in high resolution remote sensing image based on improved YOLOv5

“…This paper proposes a loss function based on KFIoU: compressibility IoU (CPIoU). The process of rotating IoU calculation is strengthened as follows: in Figure 4, the GT box and predicted box are first converted into Gaussian distribution [9] 𝐺 , 𝐺 ；Kalman filter [6] was used to calculate the Gaussian distribution 𝐺 of the overlapping region between these two boxes.…”