Convolutional Neural Processes for Inpainting Satellite Images

Abstract

The widespread availability of satellite images has allowed researchers to model complex systems such as disease dynamics. However, many satellite images have missing values due to measurement defects, which render them unusable without data imputation. For example, the scanline corrector for the LANDSAT 7 satellite broke down in 2003, resulting in a loss of around 20% of its data.Inpainting involves predicting what is missing based on the known pixels and is an old problem in image processing, classically based on PDEs or interpolation methods, but recent deep learning approaches have shown promise. However, many of these methods do not explicitly take into account the inherent spatiotemporal structure of satellite images. In this work, we cast satellite image inpainting as a natural meta-learning problem, and propose using convolutional neural processes (ConvNPs) where we frame each satellite image as its own task or 2D regression problem. We show ConvNPs can outperform classical methods and state-of-the-art deep learning inpainting models on a scanline inpainting problem for LANDSAT 7 satellite images, assessed on a variety of in and out-of-distribution images. * Equal contribution. † Order decided via a coin toss.

Meta-Learning Amidst Heterogeneity and Ambiguity

“…Context-based meta-learning, also referred to as Neural Processes (NPs), has been emphasized since it can properly predict unseen data inputs without huge computation costs [6]- [8]. Many papers have shown that the context-based meta-learning algorithms are effective not only for synthetic regression, image-inpainting, and few-shot classification but also for reinforcement learning [9]- [11].…”

“…Particularly, those gain better performance by constructing the context representation that considers the relationship between corresponding features from all the given context data points. To also acquire the translation invariance, convolutional neural processes employed convolutional neural networks and a kernel density estimation method with equally spaced additional data points for functional context representation [11], [14], [15]. Moreover, for noise invariance, [12], [16] proposed to apply bayesian approaches such that the bayesian aggregated function and bootstrapping method are used to improve robustness against irreducible noises.…”

A Single Image Deep Learning Approach to Restoration of Corrupted Landsat-7 Satellite Images

“…However, the vast majority of classical deep learning methods require a huge amount of training data to assure effective convergence of the model parameters. Convolutional conditional neural processes (ConvCNPs) and convolutional latent neural processes (ConvLNPs) have been shown to exhibit very good few-shot and zero-shot learning capabilities [ 24 ], but these methods still need pretrained models.…”