HSI-TransUNet: A transformer based semantic segmentation model for crop mapping from UAV hyperspectral imagery

Niu, Bowen; Feng, Quanlong; Chen, Boan; Ou, Cong; Liu, Yiming; Yang, Jianyu

doi:10.1016/j.compag.2022.107297

Cited by 26 publications

(12 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…He et al proposed an SST, 20 which first acquires the spatial information of HSI though CNN and then captures the spectral information by a dense transformer. Niu et al propose a transformer based semantic segmentation model 21 for crop mapping task and prove to be effective. Yang et al proposed a convolutiontransformer fusion network for HSI classification, 22 which fuses the convolution and transformer in both serial and parallel mechanisms to achieve the full utilization of HSI features.…”

Section: Transformermentioning

confidence: 99%

See 1 more Smart Citation

HFC-SST: improved spatial-spectral transformer for hyperspectral few-shot classification

Huang

Tang

et al. 2023

J. Appl. Rem. Sens.

View full text Add to dashboard Cite

Owing to the complex environment of hyperspectral image (HSI) collecting area, it is difficult to obtain an extensive number of labeled samples for HSI. Recently, many few-shot learning (FSL) algorithms based on convolutional neural network (CNN) have been employed for HSI classification in the scenery of small-scale training samples. However, a CNN-based model is unsuitable for modeling the spatial-spectral information with long-range dependency. The transformer has proved its superiority in modeling the long-range dependency. Inspired by this, an improved spatial-spectral transformer for HSI few-shot classification (HFC-SST) is proposed to deeply extract the local spatial-spectral information with only a few labeled samples. The contribution of this letter is twofold. First, a local spatial-spectral sequence generation method based on spatial-spectral correlation analysis and adjacent position information is proposed to generate the input sequence for transformer. Second, a local spatial-spectral feature extraction network based on the transformer is proposed to further exploit the spatial-spectral feature information on the input sequence. Experimental results on HFC with four datasets confirm that our proposed HFC-SST algorithm can achieve higher classification accuracy than the traditional CNN algorithms and the HSI FSL algorithms.

show abstract

Section: Transformermentioning

confidence: 99%

“…Niu et al. propose a transformer based semantic segmentation model 21 for crop mapping task and prove to be effective. Yang et al.…”

Section: Related Workmentioning

confidence: 99%

HFC-SST: improved spatial-spectral transformer for hyperspectral few-shot classification

Huang

Tang

et al. 2023

J. Appl. Rem. Sens.

View full text Add to dashboard Cite

show abstract

“…CNNs architectures showed good results on crop-segmentation of S2 Imagery from a single image [24], [22] and from SITS [7], [23]. Transformer-based architectures have been used recently with success in crop segmentation from single S2 images, as shown by recent works [25], [21]. However, hardly anyone investigated if introducing self-attention in the encoding layers can improve results in crop mapping from SITS.…”

Section: Introductionmentioning

confidence: 99%

Enhancing crop segmentation in satellite image time-series with transformer networks

Gallo,

Gatti,

Landro

et al. 2024

Sixteenth International Conference on Machine Vision (ICMV 2023)

View full text Add to dashboard Cite

Recent studies have shown that Convolutional Neural Networks (CNNs) achieve impressive results in crop segmentation of Satellite Image Time-Series (SITS). However, the emergence of transformer networks in various vision tasks raises the question of whether they can outperform CNNs in crop segmentation of SITS. This paper presents a revised version of the Transformer-based Swin UNETR model adapted specifically for crop segmentation of SITS. The proposed model demonstrates significant advancements, achieving a validation accuracy of 96.14% and a test accuracy of 95.26% on the Munich dataset, surpassing the previous best results of 93.55% for validation and 92.94% for the test. Additionally, the model's performance on the Lombardia dataset is comparable to UNet3D and superior to FPN and DeepLabV3. Experiments of this study indicate that the model will likely achieve comparable or superior accuracy to CNNs while requiring significantly less training time. These findings highlight the potential of transformer-based architectures for crop segmentation in SITS, opening new avenues for remote sensing applications.

show abstract

“…Through further studies, researchers have made several improvements to the Transformer, resulting in networks such as DETR [ 24 ], ViT [ 25 ], and SETR-MLP [ 26 ]. These networks have been applied to various fields, such as object detection [ 27 , 28 , 29 ], semantic segmentation [ 30 , 31 , 32 ], image classification [ 33 , 34 , 35 ], and image generation [ 36 ]. In this study, the advanced Swin-Transformer [ 37 ] network is used to conduct research in the following three aspects: (1) Proposing a high-precision classification and detection model for mutton multi-parts; (2) testing the robustness, generalization, and anti-occlusion performance of the proposed model; (3) introducing other mainstream detection algorithms to evaluate the advantages and disadvantages of the proposed model and test its real-time performance.…”

Section: Introductionmentioning

confidence: 99%

Research on Automatic Classification and Detection of Mutton Multi-Parts Based on Swin-Transformer

Zhao

Bai

Wang

et al. 2023

Foods

View full text Add to dashboard Cite

In order to realize the real-time classification and detection of mutton multi-part, this paper proposes a mutton multi-part classification and detection method based on the Swin-Transformer. First, image augmentation techniques are adopted to increase the sample size of the sheep thoracic vertebrae and scapulae to overcome the problems of long-tailed distribution and non-equilibrium of the dataset. Then, the performances of three structural variants of the Swin-Transformer (Swin-T, Swin-B, and Swin-S) are compared through transfer learning, and the optimal model is obtained. On this basis, the robustness, generalization, and anti-occlusion abilities of the model are tested and analyzed using the significant multiscale features of the lumbar vertebrae and thoracic vertebrae, by simulating different lighting environments and occlusion scenarios, respectively. Furthermore, the model is compared with five methods commonly used in object detection tasks, namely Sparser-CNN, YoloV5, RetinaNet, CenterNet, and HRNet, and its real-time performance is tested under the following pixel resolutions: 576 × 576, 672 × 672, and 768 × 768. The results show that the proposed method achieves a mean average precision (mAP) of 0.943, while the mAP for the robustness, generalization, and anti-occlusion tests are 0.913, 0.857, and 0.845, respectively. Moreover, the model outperforms the five aforementioned methods, with mAP values that are higher by 0.009, 0.027, 0.041, 0.050, and 0.113, respectively. The average processing time of a single image with this model is 0.25 s, which meets the production line requirements. In summary, this study presents an efficient and intelligent mutton multi-part classification and detection method, which can provide technical support for the automatic sorting of mutton as well as for the processing of other livestock meat.

show abstract

HSI-TransUNet: A transformer based semantic segmentation model for crop mapping from UAV hyperspectral imagery

Cited by 26 publications

References 24 publications

HFC-SST: improved spatial-spectral transformer for hyperspectral few-shot classification

HFC-SST: improved spatial-spectral transformer for hyperspectral few-shot classification

Enhancing crop segmentation in satellite image time-series with transformer networks

Research on Automatic Classification and Detection of Mutton Multi-Parts Based on Swin-Transformer

Contact Info

Product

Resources

About