SITS-Former: A pre-trained spatio-spectral-temporal representation model for Sentinel-2 time series classification

Yuan, Yuan; Lin, Lei; Liu, Qingshan; Hang, Renlong; Zhou, Zeng-Guang

doi:10.1016/j.jag.2021.102651

Cited by 36 publications

(48 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By predicting randomly contaminated observations given an entire time series of a pixel, the model is trained to leverage the inherent temporal structure of satellite time series to learn general-purpose spectral-temporal representations. The work was further improved in [153], where the network is asked to regress the central pixels of the masked patches for patch-based representation learning.…”

Section: B Predictive Methodsmentioning

confidence: 99%

Self-Supervised Learning in Remote Sensing: A review

Wang

Albrecht

Braham

et al. 2022

IEEE Geosci. Remote Sens. Mag.

104

View full text Add to dashboard Cite

In deep learning research, self-supervised learning (SSL) has received great attention triggering interest within both the computer vision and remote sensing communities. While there has been a big success in computer vision, most of the potential of SSL in the domain of earth observation remains locked. In this paper, we provide an introduction to, and a review of the concepts and latest developments in SSL for computer vision in the context of remote sensing. Further, we provide a preliminary benchmark of modern SSL algorithms on popular remote sensing datasets, verifying the potential of SSL in remote sensing and providing an extended study on data augmentations. Finally, we identify a list of promising directions of future research in SSL for earth observation (SSL4EO) to pave the way for fruitful interaction of both domains.

show abstract

Section: B Predictive Methodsmentioning

confidence: 99%

Self-Supervised Learning in Remote Sensing: A review

Wang

Albrecht

Braham

et al. 2022

IEEE Geosci. Remote Sens. Mag.

104

View full text Add to dashboard Cite

show abstract

“…Satellite imagery and deep learning have been widely used in crop monitoring as they provides applications in multiple areas: lands classification [3], yields predictions [4], wildfires management [5], disease detection [6] and various detection tasks [7]. In the recent years, multiple technologies have been applied to accomplish these tasks from CNNs [8] to RNNs [9] and recently transformers [10] which are becoming very popular state-of-the-art models, particularly for multimodal data such as images, text, etc.…”

Section: Introductionmentioning

confidence: 99%

Fusion of Satellite Images and Weather Data With Transformer Networks for Downy Mildew Disease Detection

2023

View full text Add to dashboard Cite

Crop diseases significantly affect the quantity and quality of agricultural production. In a context where the goal of precision agriculture is to minimize or even avoid the use of pesticides, weather and remote sensing data with deep learning can play a pivotal role in detecting crop diseases, allowing localized treatment of crops. However, combining heterogeneous data such as weather and images remains a hot topic and challenging task. Recent developments in transformer architectures have shown the possibility of fusion of data from different domains, such as text-image. The current trend is to custom only one transformer to create a multimodal fusion model. Conversely, we propose a new approach to realize data fusion using three transformers. In this paper, we first solved the missing satellite images problem, by interpolating them with a ConvLSTM model. Then, we proposed a multimodal fusion architecture that jointly learns to process visual and weather information. The architecture is built from three main components, a Vision Transformer and two transformer-encoders, allowing to fuse both image and weather modalities. The results of the proposed method are promising achieving an overall accuracy of 97%.

show abstract

“…The self-supervised pre-training approach can improve the utilization of labeled samples in land cover classification (Tarasiou and Zafeiriou, 2022). Unlabeled data as pre-training data can effectively improve crop classification accuracy and reduce the use of labeled samples (Yuan and Lin, 2021;Yuan et al, 2022). However, pre-training is still less used in scenarios such as in-season crop classification and model transfer.…”

Section: Introductionmentioning

confidence: 99%

Cropformer: A new generalized deep learning classification approach for multi-scenario crop classification

Wang

Chang²,

Yu³

et al. 2023

Front. Plant Sci.

View full text Add to dashboard Cite

Accurate and efficient crop classification using remotely sensed data can provide fundamental and important information for crop yield estimation. Existing crop classification approaches are usually designed to be strong in some specific scenarios but not for multi-scenario crop classification. In this study, we proposed a new deep learning approach for multi-scenario crop classification, named Cropformer. Cropformer can extract global features and local features, to solve the problem that current crop classification methods extract a single feature. Specifically, Cropformer is a two-step classification approach, where the first step is self-supervised pre-training to accumulate knowledge of crop growth, and the second step is a fine-tuned supervised classification based on the weights from the first step. The unlabeled time series and the labeled time series are used as input for the first and second steps respectively. Multi-scenario crop classification experiments including full-season crop classification, in-season crop classification, few-sample crop classification, and transfer of classification models were conducted in five study areas with complex crop types and compared with several existing competitive approaches. Experimental results showed that Cropformer can not only obtain a very significant accuracy advantage in crop classification, but also can obtain higher accuracy with fewer samples. Compared to other approaches, the classification performance of Cropformer during model transfer and the efficiency of the classification were outstanding. The results showed that Cropformer could build up a priori knowledge using unlabeled data and learn generalized features using labeled data, making it applicable to crop classification in multiple scenarios.

show abstract

SITS-Former: A pre-trained spatio-spectral-temporal representation model for Sentinel-2 time series classification

Cited by 36 publications

References 50 publications

Self-Supervised Learning in Remote Sensing: A review

Self-Supervised Learning in Remote Sensing: A review

Fusion of Satellite Images and Weather Data With Transformer Networks for Downy Mildew Disease Detection

Cropformer: A new generalized deep learning classification approach for multi-scenario crop classification

Contact Info

Product

Resources

About