Semantic Segmentation of Aerial Images With an Ensemble of CNNS

Abstract

ABSTRACT:This paper describes a deep learning approach to semantic segmentation of very high resolution (aerial) images. Deep neural architectures hold the promise of end-to-end learning from raw images, making heuristic feature design obsolete. Over the last decade this idea has seen a revival, and in recent years deep convolutional neural networks (CNNs) have emerged as the method of choice for a range of image interpretation tasks like visual recognition and object detection. Still, standard CNNs do not lend themselves to per-pixel semantic segmentation, mainly because one of their fundamental principles is to gradually aggregate information over larger and larger image regions, making it hard to disentangle contributions from different pixels. Very recently two extensions of the CNN framework have made it possible to trace the semantic information back to a precise pixel position: deconvolutional network layers undo the spatial downsampling, and Fully Convolution Networks (FCNs) modify the fully connected classification layers of the network in such a way that the location of individual activations remains explicit. We design a FCN which takes as input intensity and range data and, with the help of aggressive deconvolution and recycling of early network layers, converts them into a pixelwise classification at full resolution. We discuss design choices and intricacies of such a network, and demonstrate that an ensemble of several networks achieves excellent results on challenging data such as the ISPRS semantic labeling benchmark, using only the raw data as input.

High-Resolution Remote Sensing Data Classification over Urban Areas Using Random Forest Ensemble and Fully Connected Conditional Random Field

“…From this figure we can see that the parameter α σ has a significant influence on the accuracy in this case, and if it is set too large, a negative effect will occur. Comparing Figure 14 with Figure 13, we can see that the accuracy when [15,25] α σ ∈ and [20,40] β σ ∈ a satisfactory result can be obtained. Otherwise, the accuracies declined sharply.…”

“…As a remedial measure, a conditional random field (CRF) is usually used to smooth the classification result. For example, both Marmanis et al [15] and Paisitkriangkrai et al [1] used deep convolutional neural networks (CNN) to classify each pixel, then used a CRF to refine the results, whereas Niemeyer et al [16] first classified each 3D point using a random forest (RF) classifier then smoothed them using a CRF.…”

Segment-before-Detect: Vehicle Detection and Classification through Semantic Segmentation of Aerial Images

“…Indeed, the FCN model has been improved to include multi-scale and spatial regularization, e.g., with Conditional Random Fields [15,16]. Although these architectures were introduced for semantic segmentation of multimedia images, usually to discriminate foreground objects versus background, they have also been successfully used for remote sensing data on several datasets [7,[17][18][19].…”

“…Therefore, using FCN for Earth Observation means we can shift from superpixel segmentation and region-based classification [4][5][6] to fully supervised semantic segmentation [7]. FCN models have been successfully applied for remote sensing data analysis, notably land cover mapping on urban areas [7,8]. For example, FCN-based models are now the state-of-the-art on the ISPRS Vaihingen Semantic Labeling dataset [9,10].…”

“…In an Earth Observation context, these FCN models are now especially appealing, as dense prediction allows us performing semantic mapping without requiring any preprocessing tricks. Therefore, using FCN for Earth Observation means we can shift from superpixel segmentation and region-based classification [4][5][6] to fully supervised semantic segmentation [7]. FCN models have been successfully applied for remote sensing data analysis, notably land cover mapping on urban areas [7,8].…”