Towards Explaining Satellite Based Poverty Predictions with Convolutional Neural Networks

Abstract

Deep convolutional neural networks (CNNs) have been shown to predict poverty and development indicators from satellite images with surprising accuracy. This paper presents a first attempt at analyzing the CNNs responses in detail and explaining the basis for the predictions. The CNN model, while trained on relatively low resolution day-and night-time satellite images, is able to outperform human subjects who look at high-resolution images in ranking the Wealth Index categories. Multiple explainability experiments performed on the model indicate the importance of the sizes of the objects, pixel colors in the image, and provide a visualization of the importance of different structures in input images. A visualization is also provided of type images that maximize the network prediction of Wealth Index, which provides clues on what the CNN prediction is based on.

Beyond Human Bias: Learning from CNNs’ Superior Insights in Satellite-Based Poverty Mapping

“…The CNN is trained using the night-time-light transfer learning ideas introduced by Jean et al [17]. Our version of this training is described in detail in Sarmadi et al [13]. Also here, five-fold stratified cross validation with randomization was used to estimate the out-of-sample performance.…”

“…Convolutional neural network (CNN) models are the current state of the art for estimating wealth levels from daytime satellite images. A detailed description of the model building is given in [13]. The CNN backbone is trained via transfer learning on the MobileNetV2 [12] model, which is pretrained on ImageNet data.…”

“…This task is done in the framework of the TensorFlowV2 library 1 . There are two differences compared to the network described in [13]. First, we use a slightly updated image set for the training where we allow a maximum of only 25% of the image to be cloudy.…”

“…We used the 3 × 3 variation of the model in [13], meaning that we use the fully convolutional CNN and give the 3 × 3 neighborhood of the corresponding image as input to extract features. It is important to note that the Sentinel-2 daytime satellite images used by our CNN are much lower resolution than the high-resolution images shown to the domain experts, and also lower resolution than the satellite images used by Jean et al [17].…”