The color out of space: learning self-supervised representations for Earth Observation imagery

Vincenzi, Stefano; Porrello, Angelo; Buzzega, Pietro; Cipriano, Marco; Fronte, Pietro; Cuccu, Roberto; Ippoliti, Carla; Conte, Annamaria; Calderara, Simone

doi:10.1109/icpr48806.2021.9413112

Cited by 32 publications

(21 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In fact, the scarcity of large capacity remote sensing dataset is mainly in the aspect of category labels instead of images. In this case, it is promising to develop self-supervised pretraining methods [46]- [49] and some related methods have been developed in the remote sensing area [50]- [53]. For instance, SeCo [50] leverages the seasonal changes to enforce consistency between positive samples, which are the unique characteristics in aerial scenes, while [52] simultaneously fuses the temporal information and geographical location into the MoCo-V2 [47] framework.…”

Section: Remote Sensing Pretrainingmentioning

confidence: 99%

An Empirical Study of Remote Sensing Pretraining

Wang¹,

Zhang²,

Du³

et al. 2022

Preprint

View full text Add to dashboard Cite

Section: Remote Sensing Pretrainingmentioning

confidence: 99%

An Empirical Study of Remote Sensing Pretraining

Wang¹,

Zhang²,

Du³

et al. 2022

Preprint

View full text Add to dashboard Cite

“…One different approach of image inspection consists of using neural networks for feature extraction, colorization or classification. Autoencoders are neural networks that have encountered a lot of appreciation over the years due to their capability of embedding data into a lower representation in an unsupervised way [29] - [31]. These works purpose is mainly to obtain the utmost classification of the obtained latent representations.…”

Section: Related Workmentioning

confidence: 99%

Cross-Bands Information Transfer to Offset Ambiguities and Atmospheric Phenomena for Multispectral Data Visualization

Neagoe

Coca

Vaduva

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Visualization of multispectral images through band selection methods determines an information loss that in utmost cases proves to be critical for the adequate understanding of the represented scene. The R -G -B representation obtained by mapping the visual bands to the R, G and B channels is highly used due to its great resemblance with the natural color one and aspects perceivable by the human eye. However, despite the similarity in terms of color code, ambiguities between classes such as water and vegetation or atmospheric phenomena like fog, clouds and smoke that have been penetrated by other bands, remain visible and hinder the process of visualization of the Earth surface. This paper presents a set of five different methods to offset the effects caused by ambiguities, fog, light clouds and smoke by transferring relevant information between bands in order to visually reconstitute those parts of the image affected by atmospheric phenomena. The general concept shared by these methods implies a stacked autoencoder that successfully encompasses the information from all spectral bands into a latent representation used for visualization. Each proposed method is defined by different combination of input and error function formula. Spectral and polar coordinates features represent the possible options for the input, while formulas based on mean squared error (MSE) or angular spectral distances determine the potential choices in terms of error function definition. The property of angular spectral distance and polar coordinates transformation to obtain illuminant invariant features determined their use in three out of five methods. We evaluate the methods through spectral signature graphical comparison and visual comparison related to the R -G -B representation. We conduct experiments on multiple Sentinel 2 full images.

show abstract

“…They train their selfsupervised models on relatively big datasets with 100,000 images, but they only test them on two downstream tasks NAIP [18] and EuroSAT [14]. In [38] the authors propose a method similar to the image colorization, but they train a model to predict information from the visible part of the spectrum from the information outside of the visible spectrum. The authors trained the model using the BigEarthNet dataset [33,34] that is quite big with 269,695 training im-ages, but they again only tested learned representation on two downstream tasks.…”

Section: Self-supervised Learning In Remote Sensingmentioning

confidence: 99%

“…However, the application of self-supervised learning methods in remote sensing has not been studied a lot. Most of these applications either used small amounts, up to 50,000 images, of unlabeled training data [31,30,35] or tested learned representations on a small number of datasets [1,19,38].…”

Section: Introductionmentioning

confidence: 99%

Self-Supervised Learning of Remote Sensing Scene Representations Using Contrastive Multiview Coding

Stojnić¹,

Risojević²

2021

Preprint

View full text Add to dashboard Cite

In recent years self-supervised learning has emerged as a promising candidate for unsupervised representation learning. In the visual domain its applications are mostly studied in the context of images of natural scenes. However, its applicability is especially interesting in specific areas, like remote sensing and medicine, where it is hard to obtain huge amounts of labeled data. In this work, we conduct an extensive analysis of the applicability of self-supervised learning in remote sensing image classification. We analyze the influence of the number and domain of images used for self-supervised pre-training on the performance on downstream tasks. We show that, for the downstream task of remote sensing image classification, using self-supervised pre-training on remote sensing images can give better results than using supervised pre-training on images of natural scenes. Besides, we also show that self-supervised pretraining can be easily extended to multispectral images producing even better results on our downstream tasks.

show abstract

The color out of space: learning self-supervised representations for Earth Observation imagery

Cited by 32 publications

References 35 publications

An Empirical Study of Remote Sensing Pretraining

An Empirical Study of Remote Sensing Pretraining

Cross-Bands Information Transfer to Offset Ambiguities and Atmospheric Phenomena for Multispectral Data Visualization

Self-Supervised Learning of Remote Sensing Scene Representations Using Contrastive Multiview Coding

Contact Info

Product

Resources

About