RingMo: A Remote Sensing Foundation Model With Masked Image Modeling

Toward Foundation Models for Earth Monitoring: Generalizable Deep Learning Models for Natural Hazard Segmentation

“…In this work, we propose a methodology to significantly improve generalizability of DL natural hazards mappers based on pre-training on a suitable pre-task. Our approach supports the development of foundation models for earth monitoring, such as [9], with the objective of directly segmenting unseen natural hazards across unseen geographic regions. Our contributions are as follows: First, we demonstrate across four U-Net architectures that our approach significantly improves the generalizability of DL models for the segmentation of unseen natural hazards.…”

CMID: A Unified Self-Supervised Learning Framework for Remote Sensing Image Understanding

“…SatMAE [23] leveraged temporal and multi-spectral information in RS images to improve self-supervised pre-training with MIM. RingMo [14] applied the MAE [29] method and designed a new mask strategy for self-supervised representation learning on a 3 million unlabeled RS images dataset. The fine-tuning results on various downstream tasks showed that the new mask strategy was more appropriate for RS images and the learned representations by RingMo were generalized well to various RS downstream tasks.…”

“…2) Adaptation for RS Images: Although the simplicity and effectiveness of SimMIM, there are some limitations must be taken into account when applying SimMIM into RS images. One issue is that SimMIM replaces masked patches with the [MASK] token, but RS images are known for their multiobject characteristics [36] and the objects are usually densely distributed [14]. The masking operation may cause the dense and small objects in the image to be lost [14], leading to incomplete semantic meaning and making image reconstruction more difficult [14].…”

“…One issue is that SimMIM replaces masked patches with the [MASK] token, but RS images are known for their multiobject characteristics [36] and the objects are usually densely distributed [14]. The masking operation may cause the dense and small objects in the image to be lost [14], leading to incomplete semantic meaning and making image reconstruction more difficult [14]. Moreover, the zero-initialized mask token [MASK] is not originally presented in the image.…”

“…SSL methods learn from vast amounts of unlabeled data by leveraging the structure present in the data itself to create supervised signals [10], [11], [12]. This possibility to train deep learning models without human-annotated labels, combined with the impressive results achieved by SSL methods on natural images [11], [13], has prompted numerous investigations into adopting SSL for RS images [14], [15], [16]. Contrastive learning (CL) and masked image modeling (MIM) are currently the most widely-adopted SSL methods in the domain of RS [10].…”