A Dual-Path and Lightweight Convolutional Neural Network for High-Resolution Aerial Image Segmentation

Abstract

Semantic segmentation on high-resolution aerial images plays a significant role in many remote sensing applications. Although the Deep Convolutional Neural Network (DCNN) has shown great performance in this task, it still faces the following two challenges: intra-class heterogeneity and inter-class homogeneity. To overcome these two problems, a novel dual-path DCNN, which contains a spatial path and an edge path, is proposed for high-resolution aerial image segmentation. The spatial path, which combines the multi-level and global context features to encode the local and global information, is used to address the intra-class heterogeneity challenge. For inter-class homogeneity problem, a Holistically-nested Edge Detection (HED)-like edge path is employed to detect the semantic boundaries for the guidance of feature learning. Furthermore, we improve the computational efficiency of the network by employing the backbone of MobileNetV2. We enhance the performance of MobileNetV2 with two modifications: (1) replacing the standard convolution in the last four Bottleneck Residual Blocks (BRBs) with atrous convolution; and (2) removing the convolution stride of 2 in the first layer of BRBs 4 and 6. Experimental results on the ISPRS Vaihingen and Potsdam 2D labeling dataset show that the proposed DCNN achieved real-time inference speed on a single GPU card with better performance, compared with the state-of-the-art baselines.by these methods are not good at discriminating: (1) two objects which are classified into the same semantic label but with different appearances, named intra-class heterogeneity, as shown in Figure 1a, where the houses (or cars) have different shapes, sizes, and colors, but they belong to the same semantic label; and (2) two adjacent objects which are categorized into two different semantic labels but with similar appearances, named inter-class homogeneity, as shown in Figure 1b, where the low vegetation and trees are similar in colors, but their semantic labels are distinct. To tackle these two challenges, we need to consider each category of pixels as a whole, instead of assigning semantic label to each single pixel independently. To address the intra-class heterogeneity issue, we need to combine the multi-level and global context features to encode the local and global information, which can learn the discriminative and effective features to correctly categorize variant objects belonged to the same semantic label. Semantic boundaries can detect the feature variations on adjacent objects with similar appearance but different semantic labels. We can integrate it into the training process to help the network to learn the discriminative features to enlarge the inter-class differences. Based on the above two points, we propose a novel Deep Convolutional Neural Network (DCNN) that contains a spatial path and an edge path to tackle the problems of intra-class heterogeneity and inter-class homogeneity in high-resolution aerial images simultaneously.(a) intra-class heterogeneity (b) inter-class homogeneity

A Review of Environmental Context Detection for Navigation Based on Multiple Sensors

“…Like many fields of research, AP classification can be performed using CNN, as done in [17], where AP images were classified into five classes: building, low vegetation, tree, car, impervious surface. To do so, the authors built a network where features (which were obtained by a slightly modified version of the MobileNetV2 network [113]) were shared to extract two different channels: semantic and boundary maps.…”

An Efficient and Lightweight Convolutional Neural Network for Remote Sensing Image Scene Classification

“…Due to the addition of handcrafted features, the classification pipeline is also divided into two stages, which cannot be trained or implemented in an end-to-end manner. There are still few methods available for remote sensing image scene classification and object detection (such as methods proposed by Zhang et al [60], Zhang et al [61] Teimouri et al [62], etc.,) so far.…”

CSE-HRNet: A Context and Semantic Enhanced High-Resolution Network for Semantic Segmentation of Aerial Imagery

“…Meanwhile, objects of the different categories having the same colors or interacted with cast shadows would present very similar visual characteristics. Therefore, these confusing objects lead to the issues of intra-class heterogene-ity and inter-class homogeneity, both of which pose extreme challenges for accurate and coherent segmentation [5]- [7].…”

“…Semantic segmentation of high-resolution aerial images suffers from the problems of intra-class heterogeneity and inter-class homogeneity. It is claimed that the intra-class heterogeneity issue is mainly derived from the lack of contextual information [7]- [9]. Consequently, multi-scale contextual information is essential to categorize the objects with significant intra-class variances, such as vehicles with various colors and shapes, into the same semantic class [7]- [10].…”

“…It is claimed that the intra-class heterogeneity issue is mainly derived from the lack of contextual information [7]- [9]. Consequently, multi-scale contextual information is essential to categorize the objects with significant intra-class variances, such as vehicles with various colors and shapes, into the same semantic class [7]- [10]. Moreover, multi-level semantic information could be exploited to discriminate the targets with similar appearances but belonging to distinct semantic categories [11]- [17], owing to the consideration that the problem of interclass homogeneity results from inadequate semantic information [14]- [16].…”