Material Segmentation of Multi-View Satellite Imagery

Abstract

Material recognition methods use image context and local cues for pixel-wise classification. In many cases only a single image is available to make a material prediction. Image sequences, routinely acquired in applications such as mutli-view stereo, can provide a sampling of the underlying reflectance functions that reveal pixel-level material attributes. We investigate multi-view material segmentation using two datasets generated for building material segmentation and scene material segmentation from the SpaceNet Challenge satellite image dataset [9]. In this paper, we explore the impact of multi-angle reflectance information by introducing the reflectance residual encoding, which captures both the multi-angle and multispectral information present in our datasets. The residuals are computed by differencing the sparse-sampled reflectance function with a dictionary of pre-defined dense-sampled reflectance functions. Our proposed reflectance residual features improves material segmentation performance when integrated into pixel-wise and semantic segmentation architectures. At test time, predictions from individual segmentations are combined through softmax fusion and refined by building segment voting. We demonstrate robust and accurate pixelwise segmentation results using the proposed material segmentation pipeline. Figure 1. Images from the SpaceNet dataset are orthorectified and individually segmented. The segmented material masks are aggregated through softmax fusion and further refined with a separate building segment mask.

Angular Luminance for Material Segmentation

“…We evaluate and compare our AngLNet network on a material segmentation dataset derived from the SpaceNet challenge multispectral satellite image dataset [19]. The dataset contains multiple aligned and fully labeled images from three urban locations.…”