Recent Advances and Opportunities in Scene Classification of Aerial Images with Deep Models

Abstract

Scene classification is a fundamental task in interpretation of remote sensing images, and has become an active research topic in remote sensing community due to its important role in a wide range of applications. Over the past years, tremendous efforts have been made for developing powerful approaches for scene classification of remote sensing images, evolving from the traditional bag-of-visual-words model to the new generation deep convolutional neural networks (CNNs). The deep CNN based methods have exhibited remarkable breakthrough on performance, dramatically outperforming previous methods which strongly rely on hand-crafted features. However, performance with deep CNNs has gradually plateaued on existing public scene datasets, due to the notable drawbacks of these datasets, such as the small scale and low-diversity of training samples. Therefore, to promote the development of new methods and move the scene classification task a step further, we deeply discuss the existing problems in scene classification task, and accordingly present three open directions. We believe these potential directions will be instructive for the researchers in this field.

MRENet: Simultaneous Extraction of Road Surface and Road Centerline in Complex Urban Scenes from Very High-Resolution Images

“…Given the small scale and low diversity in existing datasets [49], we developed a new challenging dataset by collecting VHR images from the GF-2 satellite in complex scenes. We manually marked the accurate reference map of road surface and road centerlines.…”

Remote sensing scene classification based on high-order graph convolutional network

“…Remote sensing scene classification plays an important role in remote sensing image understanding, which aims at labeling a remote sensing scene image according to the semantic classes, e.g., forest, river and park. (Cheng, Han et al, 2017a;Cheng, Li et al, 2017b;Cheng et al, 2020Cheng et al, , 2018F. Hu et al, 2018;Li et al, 2018).…”

An End-to-End Local-Global-Fusion Feature Extraction Network for Remote Sensing Image Scene Classification

“…However, RSISC is still a big challenge for the large differences between natural and remote sensing images, as well as inapplicability of the deep learning-based methods in representing remote sensing images. Particularly, remote sensing images are more complex than natural ones, which cover a large area from the "view of God", contain many types of contents and objects, and their semantics are very ambiguous [10]. As some samples for the task of RSISC shown in Figure 1, there are some images from different categories sharing many similar contents and semantics, such as the farmland in Figure 1 (a) and (b), and the runway in Figure 1 (d) and (e).…”

“…Whereas some scene images from the same category may show high diversity in content, such as the Figure 1 (b) and (c) and the Figure 1 (e) and (f). Furthermore, the terms of semantic categories (e.g., farmland and airport) summarily describe the content of scene images in high-level abstraction [10], and some attributes (e.g., the farmland in Figure 1 (b) and the runway in Figure 1 (e)) in the scene images are not fully described by the category labels. Such inter-class similarity and intra-class diversity introduce higher requirements for the discriminative ability of feature representation with many solutions now being proposed to generate more discriminative feature representations to tackle this problem.…”