Small Object Detection in Remote Sensing Images with Residual Feature Aggregation-Based Super-Resolution and Object Detector Network

Abstract

This paper deals with detecting small objects in remote sensing images from satellites or any aerial vehicle by utilizing the concept of image super-resolution for image resolution enhancement using a deep-learning-based detection method. This paper provides a rationale for image super-resolution for small objects by improving the current super-resolution (SR) framework by incorporating a cyclic generative adversarial network (GAN) and residual feature aggregation (RFA) to improve detection performance. The novelty of the method is threefold: first, a framework is proposed, independent of the final object detector used in research, i.e., YOLOv3 could be replaced with Faster R-CNN or any object detector to perform object detection; second, a residual feature aggregation network was used in the generator, which significantly improved the detection performance as the RFA network detected complex features; and third, the whole network was transformed into a cyclic GAN. The image super-resolution cyclic GAN with RFA and YOLO as the detection network is termed as SRCGAN-RFA-YOLO, which is compared with the detection accuracies of other methods. Rigorous experiments on both satellite images and aerial images (ISPRS Potsdam, VAID, and Draper Satellite Image Chronology datasets) were performed, and the results showed that the detection performance increased by using super-resolution methods for spatial resolution enhancement; for an IoU of 0.10, AP of 0.7867 was achieved for a scale factor of 16.

A comprehensive review of deep learning-based single image super-resolution

“…GAN-based methods provide superior performance for remote sensing image SR; Liu et al developed a novel cascaded conditional Wasserstein generative adversarial network (CCWGAN) to generate HR images for remote sensing (Liu et al, 2020a). Bashir et al proposed a YOLOv3-based small-object detection framework SRCGAN-RFA-YOLO (Bashir & Wang, 2021b), where the authors used residual feature aggregation and cyclic GAN to improve the resolution of remote sensing images before performing object detection.…”

Inversion of Soil Heavy Metal Content Based on Spectral Characteristics of Peach Trees

“…In remote sensing of vegetation, the spectral index has always been regarded as an effective indicator that can monitor or evaluate growth and development [31]. It can amplify the characteristic information of the plant spectrum and reduce the interference of external factors such as the atmosphere and underlying surfaces on the spectral information.…”

Saliency-Guided Remote Sensing Image Super-Resolution

“…In [16], the authors proposed RRDGAN to acquire better-quality images by incorporating denoising and SR into a unified framework. The authors of [17] combined a cyclic generative network with residual feature aggregation to improve image super-resolution performance. Furthermore, the notion of GANs has been draw on a variety of fields, such as style-transfer [18][19][20], image dehazing [21], image completion [22], music generation [23], image segmentation [24], image classification [25], and image change detection [26].…”