Multi-Object Segmentation in Complex Urban Scenes from High-Resolution Remote Sensing Data

Abstract

Terrestrial features extraction, such as roads and buildings from aerial images using an automatic system, has many usages in an extensive range of fields, including disaster management, change detection, land cover assessment, and urban planning. This task is commonly tough because of complex scenes, such as urban scenes, where buildings and road objects are surrounded by shadows, vehicles, trees, etc., which appear in heterogeneous forms with lower inter-class and higher intra-class contrasts. Moreover, such extraction is time-consuming and expensive to perform by human specialists manually. Deep convolutional models have displayed considerable performance for feature segmentation from remote sensing data in the recent years. However, for the large and continuous area of obstructions, most of these techniques still cannot detect road and building well. Hence, this work’s principal goal is to introduce two novel deep convolutional models based on UNet family for multi-object segmentation, such as roads and buildings from aerial imagery. We focused on buildings and road networks because these objects constitute a huge part of the urban areas. The presented models are called multi-level context gating UNet (MCG-UNet) and bi-directional ConvLSTM UNet model (BCL-UNet). The proposed methods have the same advantages as the UNet model, the mechanism of densely connected convolutions, bi-directional ConvLSTM, and squeeze and excitation module to produce the segmentation maps with a high resolution and maintain the boundary information even under complicated backgrounds. Additionally, we implemented a basic efficient loss function called boundary-aware loss (BAL) that allowed a network to concentrate on hard semantic segmentation regions, such as overlapping areas, small objects, sophisticated objects, and boundaries of objects, and produce high-quality segmentation maps. The presented networks were tested on the Massachusetts building and road datasets. The MCG-UNet improved the average F1 accuracy by 1.85%, and 1.19% and 6.67% and 5.11% compared with UNet and BCL-UNet for road and building extraction, respectively. Additionally, the presented MCG-UNet and BCL-UNet networks were compared with other state-of-the-art deep learning-based networks, and the results proved the superiority of the networks in multi-object segmentation tasks.

Architecture of Deep Convolutional Encoder-Decoder Networks for Building Footprint Semantic Segmentation

“…Te U-Net family [24] suggested two innovative classifers for multi-object segmentation to extract roads and buildings. Te multi-level context gating U-Net (MCG-U-Net) and the bi-directional ConvLSTM U-Net model are the two models discussed.…”

Ship images detection and classification based on convolutional neural network with multiple feature regions

“…Nie et al [40] proposed an improved Mask R-CNN model that can accurately detect and segment ships from remote sensing images at the pixel level. The literature [41] used two new deep learning-based networks namely BCL-UNet and MGG-UNet, which can detect and segment buildings and roads very well. For the characteristics of high-resolution SAR ship images, Sun et al [42] proposed a detector based on bi-directional feature fusion and angular classification (BiFA-YOLO), which can be adapted to SAR ships of arbitrary orientation.…”

“…[40] proposed an improved Mask R‐CNN model that can accurately detect and segment ships from remote sensing images at the pixel level. The literature [41] used two new deep learning‐based networks namely BCL‐UNet and MGG‐UNet, which can detect and segment buildings and roads very well. For the characteristics of high‐resolution SAR ship images, Sun et al.…”

Edge Guided Context Aggregation Network for Semantic Segmentation of Remote Sensing Imagery

“…Semantic segmentation is a typical computer vision problem that processes raw data such as RGB images, to be specific, converting them into masks with different highlighted regions of interest where each pixel of the image is assigned as a unique category label. In recent years, semantic segmentation has become one of the key issues in remote sensing imagery parsing for its widespread applications, including road extraction [1,2], urban planning [3,4], object detection [5,6], and change detection [7], to name a few.…”