Multi-Branch Adaptive Hard Region Mining Network for Urban Scene Parsing of High-Resolution Remote-Sensing Images

Abstract

Scene parsing of high-resolution remote-sensing images (HRRSIs) refers to parsing different semantic regions from the images, which is an important fundamental task in image understanding. However, due to the inherent complexity of urban scenes, HRRSIs contain numerous object classes. These objects present large-scale variation and irregular morphological structures. Furthermore, their spatial distribution is uneven and contains substantial spatial details. All these features make it difficult to parse urban scenes accurately. To deal with these dilemmas, in this paper, we propose a multi-branch adaptive hard region mining network (MBANet) for urban scene parsing of HRRSIs. MBANet consists of three branches, namely, a multi-scale semantic branch, an adaptive hard region mining (AHRM) branch, and an edge branch. First, the multi-scale semantic branch is constructed based on a feature pyramid network (FPN). To reduce the memory footprint, ResNet50 is chosen as the backbone, which, combined with the atrous spatial pyramid pooling module, can extract rich multi-scale contextual information effectively, thereby enhancing object representation at various scales. Second, an AHRM branch is proposed to enhance feature representation of hard regions with a complex distribution, which would be difficult to parse otherwise. Third, the edge-extraction branch is introduced to supervise boundary perception training so that the contours of objects can be better captured. In our experiments, the three branches complemented each other in feature extraction and demonstrated state-of-the-art performance for urban scene parsing of HRRSIs. We also performed ablation studies on two HRRSI datasets from ISPRS and compared them with other methods.

Development of a two-stage method for segmenting the color images of urban terrain acquired from space optic-electronic observation systems based on the ant algorithm and the hough algorithm

“…In [14], convolutional neural networks are used to solve the problem of highlighting objects of interest in images of remote sensing of the earth. In [14], the application of three algorithms of convolutional neural networks with transfer training to a publicly available dataset of images of remote sensing of the Earth of different resolutions is considered. The advantage of [14] is a high indicator of the accuracy of the identification of objects of interest from the set of training.…”

“…In [14], the application of three algorithms of convolutional neural networks with transfer training to a publicly available dataset of images of remote sensing of the Earth of different resolutions is considered. The advantage of [14] is a high indicator of the accuracy of the identification of objects of interest from the set of training. The main disadvantage of [14] is the omission of unknowns objects that were not part of the training set and, as a result, the need to retrain the convolutional neural network.…”

“…The advantage of [14] is a high indicator of the accuracy of the identification of objects of interest from the set of training. The main disadvantage of [14] is the omission of unknowns objects that were not part of the training set and, as a result, the need to retrain the convolutional neural network. In addition, the identification of objects of urbanized area is not considered in [14].…”

“…The main disadvantage of [14] is the omission of unknowns objects that were not part of the training set and, as a result, the need to retrain the convolutional neural network. In addition, the identification of objects of urbanized area is not considered in [14].…”

“…The advantage of [16] is the efficient use of memory and obtaining explicit and clear boundaries of objects. The disadvantage of [14] is the need to constantly add the selected boundaries to the architecture of the encoder-decoder and the impossibility of using [16] to determine the elements of an urbanized area.…”