Semantic Segmentation of Aerial Images With an Ensemble of CNNS

Abstract

ABSTRACT:This paper describes a deep learning approach to semantic segmentation of very high resolution (aerial) images. Deep neural architectures hold the promise of end-to-end learning from raw images, making heuristic feature design obsolete. Over the last decade this idea has seen a revival, and in recent years deep convolutional neural networks (CNNs) have emerged as the method of choice for a range of image interpretation tasks like visual recognition and object detection. Still, standard CNNs do not lend themselves to per-pixel semantic segmentation, mainly because one of their fundamental principles is to gradually aggregate information over larger and larger image regions, making it hard to disentangle contributions from different pixels. Very recently two extensions of the CNN framework have made it possible to trace the semantic information back to a precise pixel position: deconvolutional network layers undo the spatial downsampling, and Fully Convolution Networks (FCNs) modify the fully connected classification layers of the network in such a way that the location of individual activations remains explicit. We design a FCN which takes as input intensity and range data and, with the help of aggressive deconvolution and recycling of early network layers, converts them into a pixelwise classification at full resolution. We discuss design choices and intricacies of such a network, and demonstrate that an ensemble of several networks achieves excellent results on challenging data such as the ISPRS semantic labeling benchmark, using only the raw data as input.

ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data

“…There has been a quick uptake of the approach in the remote sensing community and various solutions based on deep learning have been presented recently (e.g. Audebert et al, 2016Audebert et al, , 2017Audebert et al, , 2018Längkvist et al, 2016;Li et al, 2015;Li and Shao, 2014;Volpi and Tuia, 2017;Liu et al, 2018Liu et al, , 2017aPan et al, 2018a,b;Marmanis et al, 2016Marmanis et al, , 2018Wen et al, 2017;Zhao et al, 2017b). A comprehensive review of deep learning applications in the field of remote sensing can be found in ; Ma et al (2019); Gu et al (2019).…”

“…The estimated boundaries were then concatenated with image features and provided them as input into another CNN segmentation network, for the final classification of pixels. For the CNN segmentation network, they experimented with two architectures, the SegNet (Badrinarayanan et al, 2015) and a Fully Convolutional Network presented in Marmanis et al (2016) that uses weights from pretrained architectures. One of the key differences in our approach for boundary detection with Marmanis et al (2018), is that the boundary prediction happens at the end of our architecture, therefore the request for boundary prediction affects all features since the boundaries are strongly correlated with the extent of the predicted classes.…”

Deep Learning for Classification of Hyperspectral Data: A Comparative Review

“…Thanks to recent advances in deep learning for image processing and pattern recognition, remote sensing data classification progressed tremendously in the last few years. In particular, standard optical imagery (Red-Green-Blue -RGBand Infra-Red -IR-) benefited from using deep convolutional neural networks (CNN) for tasks such as classification, object detection or semantic segmentation [1], [2], [3]. This was made possible by the transfer of models developed in computer vision, which focuses mostly on images encoded on three channels.…”

CNNs Fusion for Building Detection in Aerial Images for the Building Detection Challenge

“…Ju et al [9] show that unweighted average (used by Marmanis et al [18] on multiple instances of the same model architecture and by Kamnitsas et al [10] on complementary architectures) is a fusion method that performs as good as, if not better than, other known methods. It is favorable for reducing variance among predictions, but suffers from overconfident models.…”