Self-Supervised Learning in Remote Sensing: A review

Wang, Yi; Albrecht, Conrad M; Braham, Nassim Ait Ali; Mou, Lichao; Zhu, Xiao Xiang

doi:10.1109/mgrs.2022.3198244

Cited by 105 publications

(73 citation statements)

References 273 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Advances in deep learning (DL) have resulted in a set of tools for earth monitoring [4,5]. For instance, DL models have shown their advantages for land cover classification tasks [6] and segmentation of floods [7].…”

Section: Motivationmentioning

confidence: 99%

Toward Foundation Models for Earth Monitoring: Generalizable Deep Learning Models for Natural Hazard Segmentation

Jakubik¹,

Muszyński²,

Vössing³

et al. 2023

Preprint

View full text Add to dashboard Cite

Section: Motivationmentioning

confidence: 99%

Toward Foundation Models for Earth Monitoring: Generalizable Deep Learning Models for Natural Hazard Segmentation

Jakubik¹,

Muszyński²,

Vössing³

et al. 2023

Preprint

View full text Add to dashboard Cite

“…Readers are invited to read a recent survey on self-supervised and semi-supervised approaches applied to remote sensing segmentation task in [102]. For a broader overview of self-supervised approaches in different remote sensing tasks, we refer them to another very recent preprint paper [103].…”

Section: Xu Et Al 2021 [101]mentioning

confidence: 99%

Self-Supervised Learning for Scene Classification in Remote Sensing: Current State of the Art and Perspectives

2022

View full text Add to dashboard Cite

Deep learning methods have become an integral part of computer vision and machine learning research by providing significant improvement performed in many tasks such as classification, regression, and detection. These gains have been also observed in the field of remote sensing for Earth observation where most of the state-of-the-art results are now achieved by deep neural networks. However, one downside of these methods is the need for large amounts of annotated data, requiring lots of labor-intensive and expensive human efforts, in particular for specific domains that require expert knowledge such as medical imaging or remote sensing. In order to limit the requirement on data annotations, several self-supervised representation learning methods have been proposed to learn unsupervised image representations that can consequently serve for downstream tasks such as image classification, object detection or semantic segmentation. As a result, self-supervised learning approaches have been considerably adopted in the remote sensing domain within the last few years. In this article, we review the underlying principles developed by various self-supervised methods with a focus on scene classification task. We highlight the main contributions and analyze the experiments, as well as summarize the key conclusions, from each study. We then conduct extensive experiments on two public scene classification datasets to benchmark and evaluate different self-supervised models. Based on comparative results, we investigate the impact of individual augmentations when applied to remote sensing data as well as the use of self-supervised pre-training to boost the classification performance with limited number of labeled samples. We finally underline the current trends and challenges, as well as perspectives of self-supervised scene classification.

show abstract

“…It is an eight-label classification task in which each image patch is defined to have up to eight adjacent patches. However, the feature representations learned by generative and discriminative methods are highly dependent on the used pretext task, and an ineffective pretext task might reduce the transfer-ability of a pre-trained model [45]. In stead of solving a single pretext task, contrastive approaches train models by maximising the similarity between the feature representations of two positive samples, e.g., two augmented views of the same instance.…”

Section: Introductionmentioning

confidence: 99%

“…In stead of solving a single pretext task, contrastive approaches train models by maximising the similarity between the feature representations of two positive samples, e.g., two augmented views of the same instance. However, simply following this approach will easily generate an identity map for each pair of positive samples, i.e., model collapse [45]. DINO [6], a state-of-the-art contrastive self-supervised model, utilizes knowledge distillation and centering and sharpening of the teacher network [23] to handle this issue.…”

Section: Introductionmentioning

confidence: 99%

Change Detection Based on Supervised Contrastive Learning for High-Resolution Remote Sensing Imagery

Wang

Zhong

Zhang

2023

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

Due to the costly nature of remote sensing image labeling and the large volume of available unlabeled imagery, self-supervised methods that can learn feature representations without manual annotation have received great attention. While prior works have explored self-supervised learning in remote sensing tasks, pretext tasks based on local-global view alignment remain underexplored. Inspired by DINO [6], which employs an effective representation learning structure with knowledge distillation based on global-local view alignment, we formulate two pretext tasks for use in self-supervised learning on remote sensing imagery (SSLRS). Using these tasks, we explore the effectiveness of positive temporal contrast as well as multi-sized views on SSLRS. Moreover, we extend DINO and propose DINO-MC which uses local views of various sized crops instead of a single fixed size. Our experiments demonstrate that even when pre-trained on only 10% of the dataset, DINO-MC performs on par or better than existing state of the art SSLRS methods on multiple remote sensing tasks, while using less computational resources. All codes, models and results are available at https://github.com/ WennyXY/DINO-MC.

show abstract

Self-Supervised Learning in Remote Sensing: A review

Cited by 105 publications

References 273 publications

Toward Foundation Models for Earth Monitoring: Generalizable Deep Learning Models for Natural Hazard Segmentation

Toward Foundation Models for Earth Monitoring: Generalizable Deep Learning Models for Natural Hazard Segmentation

Self-Supervised Learning for Scene Classification in Remote Sensing: Current State of the Art and Perspectives

Change Detection Based on Supervised Contrastive Learning for High-Resolution Remote Sensing Imagery

Contact Info

Product

Resources

About