A CNN-Transformer Hybrid Approach for Crop Classification Using Multitemporal Multisensor Images

Li, Zhengtao; Chen, Guokun; Zhang, Tianxu

doi:10.1109/jstars.2020.2971763

Cited by 88 publications

(33 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors in [46] proposed a hybrid CNN and transformer architecture for crop classification on multitemporal and multispectral data. In their research, a dataset with 65 acquiring dates were collected from Sentinel-2 A/B and Landsat-8 for a region in central California.…”

Section: Deep Learning Techniques For Hyperspectral Data Analyticsmentioning

confidence: 99%

Big Data and Machine Learning With Hyperspectral Information in Agriculture

2021

View full text Add to dashboard Cite

Hyperspectral and multispectral information processing systems and technologies have demonstrated its usefulness for the improvement of agricultural productivity and practices by providing useful information to farmers and crop managers on the factors affecting crop status and growth. These technologies are widely used in a range of agriculture applications such as crop management, crop yield forecasting, crop disease detection, and the monitoring of agriculture land usage, water, and soil conditions. Hyperspectral information sensing can acquire several hundred spectral bands that cover the electromagnetic spectrum of an observational scene in a single acquisition. The resulting hyperspectral data cube contains a large volume of spatial and spectral information. The hyperspectral sequence of images or video further increases the data generation velocity and volume which lead to the Big data challenges particularly in agricultural remote sensing applications. This paper is structured to first give a comprehensive review of representative studies to provide insights into significant research efforts in agriculture using Big data, machine learning and deep learning with the focus on frameworks or architectures, information processing and analytics with hyperspectral and multispectral data. The potential for utilizing Big data, machine learning and deep learning for hyperspectral and multispectral data in agriculture is very promising. The paper then further explores the potential of using ensemble machine learning and scalable parallel discriminant analysis which takes into consideration the spatial and spectral components for Big data in agriculture. To the best of our knowledge, no similar review study on agriculture with Big data, machine learning and deep learning for hyperspectral and multispectral information processing has been reported. Furthermore, the potential of ensemble machine learning and scalable parallel discriminant analysis has not been explored in agriculture information processing. Experiments and data analytics have been performed on hyperspectral data from agriculture for validation. The results have shown the good performance of our approach.

show abstract

Section: Deep Learning Techniques For Hyperspectral Data Analyticsmentioning

confidence: 99%

Big Data and Machine Learning With Hyperspectral Information in Agriculture

2021

View full text Add to dashboard Cite

show abstract

“…In the building extraction task, the transformer-based model may be a potential choice because of its strong feature representation capability, global receptive field, and the capability of modeling long-range dependencies between pixels. However, when dealing with high-resolution image data, transformers are usually computation-inefficient and entail high memory usage [19][20][21][22]. This greatly limits its usability in remote sensing image analysis tasks since both the volume and resolution of high-resolution remote sensing image data have recently been growing exponentially.…”

Section: Introductionmentioning

confidence: 99%

Building Extraction from Remote Sensing Images with Sparse Token Transformers

2021

View full text Add to dashboard Cite

Deep learning methods have achieved considerable progress in remote sensing image building extraction. Most building extraction methods are based on Convolutional Neural Networks (CNN). Recently, vision transformers have provided a better perspective for modeling long-range context in images, but usually suffer from high computational complexity and memory usage. In this paper, we explored the potential of using transformers for efficient building extraction. We design an efficient dual-pathway transformer structure that learns the long-term dependency of tokens in both their spatial and channel dimensions and achieves state-of-the-art accuracy on benchmark building extraction datasets. Since single buildings in remote sensing images usually only occupy a very small part of the image pixels, we represent buildings as a set of “sparse” feature vectors in their feature space by introducing a new module called “sparse token sampler”. With such a design, the computational complexity in transformers can be greatly reduced over an order of magnitude. We refer to our method as Sparse Token Transformers (STT). Experiments conducted on the Wuhan University Aerial Building Dataset (WHU) and the Inria Aerial Image Labeling Dataset (INRIA) suggest the effectiveness and efficiency of our method. Compared with some widely used segmentation methods and some state-of-the-art building extraction methods, STT has achieved the best performance with low time cost.

show abstract

“…An RNN needs to process the input time series moment by moment, which brings a lot of calculation and storage burden to training the model. With the invention of the Transformer model, researchers have applied it to extract temporal dependencies from remote sensing time series data [39], [46], [47].…”

Section: Related Workmentioning

confidence: 99%

“…• In view of the existing problems of DL processing remote-sensing time series, we use the Informer model (a new variant of Transformer) to deal with the remotesensing time-series classification task; we also explain the role of each module of the model when it is applied to this task. • Because the label of the sample is visible to the entire time series [39], we removed the mask operation in the classic Informer model. In addition, unlike the Informer, the input of the decoder module retains the data of all time steps, so that the mutual influence among all moments can be exploited.…”

Section: Introductionmentioning

confidence: 99%

Land-Cover Classification With Time-Series Remote Sensing Images by Complete Extraction of Multiscale Timing Dependence

Yan

Liu

Wang

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

The evolution rules of different land covers show some similarity, which poses a huge challenge for high-accuracy land-cover classification. Therefore, the full extraction of multiscale timing-dependency features is important for mining seasonal and phenological change laws and improving the accuracy of time-series land-cover classification. However, traditional methods are often unable to fully detect the global and local change information generated during the evolution of land covers, resulting in incomplete timing-dependency features being extracted and a low classification accuracy. The Informer network can fully capture the long-term dependence of a time series, thereby improving its classification accuracy. Therefore, we propose a high-accuracy land-cover classification method with the Informer network: (1) We continuously shorten the length of the series so that the ProbSparse self-attention mechanism can consider timing-dependencies on multi-scale, and then we can obtain the features of the local important moments; (2) we calculate the correlation between the important moments and the other moments, as well as the correlation among each moment, to fully utilize the local and global time-dependent features of the land-cover time series; and (3) we add a fully connected batch normalization module in order to use all the extracted timing dependency for classification. Finally, the proposed model is compared with traditional models on two datasets: for the reorganized BreizhCrops dataset, it achieved a performance similar to long short-term memory (LSTM); for the TiSeLaC dataset, it achieved an F1-score of 96.011%, which is 0.33% higher than the second-best model.

show abstract

A CNN-Transformer Hybrid Approach for Crop Classification Using Multitemporal Multisensor Images

Cited by 88 publications

References 27 publications

Big Data and Machine Learning With Hyperspectral Information in Agriculture

Big Data and Machine Learning With Hyperspectral Information in Agriculture

Building Extraction from Remote Sensing Images with Sparse Token Transformers

Land-Cover Classification With Time-Series Remote Sensing Images by Complete Extraction of Multiscale Timing Dependence

Contact Info

Product

Resources

About