Small Manhole Cover Detection in Remote Sensing Imagery with Deep Convolutional Neural Networks

Liu, Wei; Cheng, Dayu; Yin, Pengcheng; Yang, Mengyuan; Li, Erzhu; Xie, Mei; Zhang, Lianpeng

doi:10.3390/ijgi8010049

Cited by 20 publications

(22 citation statements)

References 20 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chen et al [20] proposed a DCNN based on DeepLabv3 [8], which adopted modified ASPP, a fully connected fusion path and pre-trained encoder for high-resolution remote sensing images segmentation. Liu et al [21] introduced an effective method to detect manhole cover objects in remote sensing images. They designed two sub-networks: a multi-scale output network for manhole cover object-like edge generation, and a multi-level convolution matching network for object detection based on fused feature maps.…”

Section: Related Workmentioning

confidence: 99%

A Dual-Path and Lightweight Convolutional Neural Network for High-Resolution Aerial Image Segmentation

Zhang

Leí

Cui

et al. 2019

IJGI

View full text Add to dashboard Cite

Semantic segmentation on high-resolution aerial images plays a significant role in many remote sensing applications. Although the Deep Convolutional Neural Network (DCNN) has shown great performance in this task, it still faces the following two challenges: intra-class heterogeneity and inter-class homogeneity. To overcome these two problems, a novel dual-path DCNN, which contains a spatial path and an edge path, is proposed for high-resolution aerial image segmentation. The spatial path, which combines the multi-level and global context features to encode the local and global information, is used to address the intra-class heterogeneity challenge. For inter-class homogeneity problem, a Holistically-nested Edge Detection (HED)-like edge path is employed to detect the semantic boundaries for the guidance of feature learning. Furthermore, we improve the computational efficiency of the network by employing the backbone of MobileNetV2. We enhance the performance of MobileNetV2 with two modifications: (1) replacing the standard convolution in the last four Bottleneck Residual Blocks (BRBs) with atrous convolution; and (2) removing the convolution stride of 2 in the first layer of BRBs 4 and 6. Experimental results on the ISPRS Vaihingen and Potsdam 2D labeling dataset show that the proposed DCNN achieved real-time inference speed on a single GPU card with better performance, compared with the state-of-the-art baselines.by these methods are not good at discriminating: (1) two objects which are classified into the same semantic label but with different appearances, named intra-class heterogeneity, as shown in Figure 1a, where the houses (or cars) have different shapes, sizes, and colors, but they belong to the same semantic label; and (2) two adjacent objects which are categorized into two different semantic labels but with similar appearances, named inter-class homogeneity, as shown in Figure 1b, where the low vegetation and trees are similar in colors, but their semantic labels are distinct. To tackle these two challenges, we need to consider each category of pixels as a whole, instead of assigning semantic label to each single pixel independently. To address the intra-class heterogeneity issue, we need to combine the multi-level and global context features to encode the local and global information, which can learn the discriminative and effective features to correctly categorize variant objects belonged to the same semantic label. Semantic boundaries can detect the feature variations on adjacent objects with similar appearance but different semantic labels. We can integrate it into the training process to help the network to learn the discriminative features to enlarge the inter-class differences. Based on the above two points, we propose a novel Deep Convolutional Neural Network (DCNN) that contains a spatial path and an edge path to tackle the problems of intra-class heterogeneity and inter-class homogeneity in high-resolution aerial images simultaneously.(a) intra-class heterogeneity (b) inter-class homogeneity

show abstract

Section: Related Workmentioning

confidence: 99%

A Dual-Path and Lightweight Convolutional Neural Network for High-Resolution Aerial Image Segmentation

Zhang

Leí

Cui

et al. 2019

IJGI

View full text Add to dashboard Cite

show abstract

“…DL methods have been successfully used to object detection [ 20 ] in several applications, such as agriculture and environmental studies [ 21 , 22 ], urban infrastructure [ 23 ] and health analysis [ 24 ]. Thus far, solely few works have been developed to detect manholes using DL ([ 25 ] and [ 26 ]). Reference [ 25 ] perform manhole detection in aerial images.…”

Section: Introductionmentioning

confidence: 99%

Storm-Drain and Manhole Detection Using the RetinaNet Method

Santos

Marcato

Silva

et al. 2020

Sensors

View full text Add to dashboard Cite

As key-components of the urban-drainage system, storm-drains and manholes are essential to the hydrological modeling of urban basins. Accurately mapping of these objects can help to improve the storm-drain systems for the prevention and mitigation of urban floods. Novel Deep Learning (DL) methods have been proposed to aid the mapping of these urban features. The main aim of this paper is to evaluate the state-of-the-art object detection method RetinaNet to identify storm-drain and manhole in urban areas in street-level RGB images. The experimental assessment was performed using 297 mobile mapping images captured in 2019 in the streets in six regions in Campo Grande city, located in Mato Grosso do Sul state, Brazil. Two configurations of training, validation, and test images were considered. ResNet-50 and ResNet-101 were adopted in the experimental assessment as the two distinct feature extractor networks (i.e., backbones) for the RetinaNet method. The results were compared with the Faster R-CNN method. The results showed a higher detection accuracy when using RetinaNet with ResNet-50. In conclusion, the assessed DL method is adequate to detect storm-drain and manhole from mobile mapping RGB images, outperforming the Faster R-CNN method. The labeled dataset used in this study is available for future research.

show abstract

“…Recent researches show that deep convolutional neural networks (DCNNs) could reach an impressive advanced performance for scene classification [16,17], object detection [18][19][20], and semantic segmentation [6,21,22] while using remote sensing imagery. DCNNs can accurately extract semantic features not only the low-levels and middle-levels, but also the high-level features from the input image [23].…”

Section: Introductionmentioning

confidence: 99%

Accurate Building Extraction from Fused DSM and UAV Images Using a Chain Fully Convolutional Neural Network

et al. 2019

Self Cite

View full text Add to dashboard Cite

Accurate extraction of buildings using high spatial resolution imagery is essential to a wide range of urban applications. However, it is difficult to extract semantic features from a variety of complex scenes (e.g., suburban, urban and urban village areas) because various complex man-made objects usually appear heterogeneous with large intra-class and low inter-class variations. The automatic extraction of buildings is thus extremely challenging. The fully convolutional neural networks (FCNs) developed in recent years have performed well in the extraction of urban man-made objects due to their ability to learn state-of-the-art features and to label pixels end-to-end. One of the most successful FCNs used in building extraction is U-net. However, the commonly used skip connection and feature fusion refinement modules in U-net often ignore the problem of feature selection, and the ability to extract smaller buildings and refine building boundaries needs to be improved. In this paper, we propose a trainable chain fully convolutional neural network (CFCN), which fuses high spatial resolution unmanned aerial vehicle (UAV) images and the digital surface model (DSM) for building extraction. Multilevel features are obtained from the fusion data, and an improved U-net is used for the coarse extraction of the building. To solve the problem of incomplete extraction of building boundaries, a U-net network is introduced by chain, which is used for the introduction of a coarse building boundary constraint, hole filling, and "speckle" removal. Typical areas such as suburban, urban, and urban villages were selected for building extraction experiments. The results show that the CFCN achieved recall of 98.67%, 98.62%, and 99.52% and intersection over union (IoU) of 96.23%, 96.43%, and 95.76% in suburban, urban, and urban village areas, respectively. Considering the IoU in conjunction with the CFCN and U-net resulted in improvements of 6.61%, 5.31%, and 6.45% in suburban, urban, and urban village areas, respectively. The proposed method can extract buildings with higher accuracy and with clearer and more complete boundaries.

show abstract

Small Manhole Cover Detection in Remote Sensing Imagery with Deep Convolutional Neural Networks

Cited by 20 publications

References 20 publications

A Dual-Path and Lightweight Convolutional Neural Network for High-Resolution Aerial Image Segmentation

A Dual-Path and Lightweight Convolutional Neural Network for High-Resolution Aerial Image Segmentation

Storm-Drain and Manhole Detection Using the RetinaNet Method

Accurate Building Extraction from Fused DSM and UAV Images Using a Chain Fully Convolutional Neural Network

Contact Info

Product

Resources

About