Deformable ConvNet with Aspect Ratio Constrained NMS for Object Detection in Remote Sensing Imagery

Abstract

Convolutional neural networks (CNNs) have demonstrated their ability object detection of very high resolution remote sensing images. However, CNNs have obvious limitations for modeling geometric variations in remote sensing targets. In this paper, we introduced a CNN structure, namely deformable ConvNet, to address geometric modeling in object recognition. By adding offsets to the convolution layers, feature mapping of CNN can be applied to unfixed locations, enhancing CNNs' visual appearance understanding. In our work, a deformable region-based fully convolutional networks (R-FCN) was constructed by substituting the regular convolution layer with a deformable convolution layer. To efficiently use this deformable convolutional neural network (ConvNet), a training mechanism is developed in our work. We first set the pre-trained R-FCN natural image model as the default network parameters in deformable R-FCN. Then, this deformable ConvNet was fine-tuned on very high resolution (VHR) remote sensing images. To remedy the increase in lines like false region proposals, we developed aspect ratio constrained non maximum suppression (arcNMS). The precision of deformable ConvNet for detecting objects was then improved. An end-to-end approach was then developed by combining deformable R-FCN, a smart fine-tuning strategy and aspect ratio constrained NMS. The developed method was better than a state-of-the-art benchmark in object detection without data augmentation.

SCRDet: Towards More Robust Detection for Small, Cluttered and Rotated Objects

“…The R-P-Faster R-CNN framework is developed in [14] for small objects. While both deformable convolution layers [6] and R-FCN are combined by [40] to improve detection accuracy. More recently, the authors in [40] adopt top-down and skipped connections to produce a single high-level feature map of a fine resolution, improving the performance of the deformable Faster R-CNN.…”

“…While both deformable convolution layers [6] and R-FCN are combined by [40] to improve detection accuracy. More recently, the authors in [40] adopt top-down and skipped connections to produce a single high-level feature map of a fine resolution, improving the performance of the deformable Faster R-CNN. However such horizontal region based detectors still are confronted with the challenges for the aforementioned bottlenecks in terms of scale, orientation and density, which call for more principled methods beyond the setting for horizontal region detection.…”

Multi-Scale Geospatial Object Detection Based on Shallow-Deep Feature Extraction

“…As a result, the proposed SD-MS framework outperforms all comparison approaches for all ten classes of the NWPU VHR-10 dataset, which demonstrates the superiority of the proposed method compared with the eight other methods. In addition, to quantitatively evaluate the proposed SD-MS model, this study compared it with eight existing methods: rotation-invariant CNN (RICNN) [15], region proposal networks with faster R-CNN (R-P-faster R-CNN) (R-P-F-R-CNN ) [50], deformable R-FCN (D-R-FCN) [51], collection of part detectors (COPD) [11], position-sensitive balancing (PSB) [20], deformable faster R-CNN (D-F-R-CNN) [52], recurrent detection with activated semantics (RDAS512) [53], and multi-scale CNN (MS-CNN) [19]. As can be seen from Table 4, the proposed SD-MS obtains the best mAP.…”

A Novel Adaptive Joint Time Frequency Algorithm by the Neural Network for the ISAR Rotational Compensation

“…It will be done before it is applied to compensate radar image. For example, in other neural network related papers [24,35,37], they do not consider the training time when listing total performance. Thus, the time consumption of training procedure should not be considered in the total performance.…”