Towards On-Board Hyperspectral Satellite Image Segmentation: Understanding Robustness of Deep Learning through Simulating Acquisition Conditions

Abstract

Although hyperspectral images capture very detailed information about the scanned objects, their efficient analysis, transfer, and storage are still important practical challenges due to their large volume. Classifying and segmenting such imagery are the pivotal steps in virtually all applications, hence developing new techniques for these tasks is a vital research area. Here, deep learning has established the current state of the art. However, deploying large-capacity deep models on-board an Earth observation satellite poses additional technological challenges concerned with their memory footprints, energy consumption requirements, and robustness against varying-quality image data, with the last problem being under-researched. In this paper, we tackle this issue, and propose a set of simulation scenarios that reflect a range of atmospheric conditions and noise contamination that may ultimately happen on-board an imaging satellite. We verify their impact on the generalization capabilities of spectral and spectral-spatial convolutional neural networks for hyperspectral image segmentation. Our experimental analysis, coupled with various visualizations, sheds more light on the robustness of the deep models and indicate that specific noise distributions can significantly deteriorate their performance. Additionally, we show that simulating atmospheric conditions is key to obtaining the learners that generalize well over image data acquired in different imaging settings.

Graph Neural Networks Extract High-Resolution Cultivated Land Maps From Sentinel-2 Image Series

“…We are currently working on using the weighted edges (and heterogeneous graphs) to further exploit the temporal information, and on porting the models to Intuition-1. We focus on not only benchmarking such algorithms [23], but also on verifying their robustness against on-board conditions [24]. Finally, we work on utilizing other interpolation techniques and our multi-image SR algorithms, and to make them taskdriven for precise segmentation of satellite images.…”

Benchmarking Deep Learning for On-Board Space Applications

“…It could make the adoption of state-ofthe-art deep learning faster in on-board satellite data analysis faster; thus, it could be an important step towards exploiting deep learning at the edge in various applications, in both critical and non-critical missions. Finally, when coupled with the data-level digital twins that enable us to simulate on-board image acquisition [20], the proposed benchmarking technique can become a comprehensive tool for assessing the robustness of on-board AI through various simulations.…”

“…Hyperspectral imaging (HSI) provides very detailed information about the scanned objects, capturing their spectral characteristics within hundreds of contiguous wavelength bands. Classification of such data has become an active research topic due to its wide applicability in a variety of fields ranging from biology, chemistry, forensics, straight to remote sensing and Earth observation [20]. Deep learning is an ideal candidate to be leveraged in this area and has been blooming in the field by delivering state-of-the-art performance for HSI classification and segmentation.…”

Self-Supervised Denoising for Real Satellite Hyperspectral Imagery

“…Therefore, satellite hyperspectral imagery has been widely used in smart agriculture [2], urban applications [3], water quality monitoring [4], ecological sustainability [5], and applications in other fields. Compared with natural images, hyperspectral images can simultaneously obtain ground scene information in multiple bands with a low signal-to-noise ratio (SNR), which leads to the inevitable influence of various noises in the process of hyperspectral image acquisition [6]; furthermore, satellite hyperspectral images are also affected by complex space and atmospheric environments [7]. Noise reduces the quality of satellite hyperspectral images and greatly limits their subsequent processing and application [8].…”