Transfer Representation Learning Meets Multimodal Fusion Classification for Remote Sensing Images

Ma, Mengru; Ma, Wenping; Jiao, Licheng; Liu, Xu; Liu, Fang; Yang, Shuyuan; Hou, Biao

doi:10.1109/tgrs.2022.3215177

Cited by 15 publications

(3 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Notably, ensembles of pretrained SAR and RGB models have shown optimal performance [32], with methods such as the fusion of Sentinel-1 DInSAR imagery with Sentinel-2 for urban change detection demonstrating the superiority of transfer learning by feature extraction (TLFE) and the introduction of ensemble methods (FeatSpaceEnsNet, AvgEnsNet, HybridEnsNet) over traditional models. Furthermore, the fusion of multispectral satellite data with panchromatic images using a multi-branch approach has been explored [33]. Our approach is distinguished by its the choice of spectral bands for each branch, the parameter initialization, and tailored training strategies.…”

Section: Related Workmentioning

confidence: 99%

Two-stage Transfer Learning for Airborne Multi-spectral Image Classifiers

Rise,

Uney,

Huang

2024

Preprint

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

Two-stage Transfer Learning for Airborne Multi-spectral Image Classifiers

Rise,

Uney,

Huang

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Semantic segmentation is the task of assigning semantic labels to each pixel in an image, which is a common research topic in remote sensing imagery [17], [18]. Traditional methods often use region split-merge and active contour models for this task.…”

Section: A Remote Sensing Imagery Semantic Segmentationmentioning

confidence: 99%

MMLN: Multi-directional and Multi-constraint Learning Network for Remote Sensing Imagery Semantic Segmentation

Sun,

Xie,

Ren

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Recent advances in satellite remote sensing technology and computer technology have significantly impacted practical applications in remote sensing image segmentation. However, the prevalent hybrid segmentation models that combine Convolutional Neural Networks (CNNs) and Transformers, often overlook the critical exploration of local and global feature correlations across various scales. This exploration is essential for learning more representative features and strengthening context modeling capabilities. Additionally, the decoding layers of these models do not effectively exploit the pixel-level semantic relationships within cross-layer feature maps, thereby limiting the models' ability to discern small object features. To address these challenges, this paper introduces a Multi-directional and Multiconstraint Learning Network (MMLN) designed for semantic segmentation of remote sensing imagery. This network features a Multi-directional Dynamic Complement Decoder (MDCD), which enhances the interaction between local and global features in the feature space, and subsequently improves the feature discrimination within the segmentation network. Moreover, a Multi-constraint Saliency Boundary-adaptive Module (MSBM) is implemented to reinforce the spatial constraints on saliency at the edge regions and ensure semantic consistency along the mask boundaries. This, in turn, augments the segmentation model's capability to detect small objects. The evaluation on four datasets reveals that the MMLN outperforms the existing state-of-the-art methods in remote sensing imagery segmentation. The code is available at https://github.com/zhongyas/MMLN.

show abstract

“…For example, Wang et al [44] designed an adaptive learning strategy, which transfers the knowledge from a pre-trained model for the classification of remote sensing scenes. Ma et al [45] introduced dual-branch attention into transfer learning to alleviate the issues of inter-and intra-class differences. Besides, transfer learning is also used for the classification of targets or pixels in images [46]- [47].…”

Section: E Transfer Learningmentioning

confidence: 99%

A Unified Two-Stage Spatial and Spectral Network With Few-Shot Learning for Pansharpening

Sheng

Zhang

Sun

et al. 2023

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

Recently, pan-sharpening methods based on deep learning (DL) have achieved state-of-the-art results. However, current existing DL-based pan-sharpening methods need to be trained repetitively for different satellite sensors to obtain satisfactory fusion performance and therefore require a large number of training images for each satellite. To deal with these issues, in this paper we propose a unified two-stage spatial and spectral network (UTSN) for pan-sharpening. A branch of networks is constructed for each different satellite, in which the spatial enhancement network (SEN) is shared to improve the spatial details in the fused images from different satellites. A spectral adjustment network (SAN) is employed to capture the spectral characteristics of the specific satellite. Through SAN, the spectral information in the intermediate image from SEN is refined to produce the final fusion results. Such a framework can integrate the datasets from different satellites together for sufficient training of SEN. The proposed method is able to achieve promising pan-sharpening results also for a new satellite with limited training images by only learning a new SAN on the few-shot datasets due to the simple but efficient structure of SAN. The experimental results show that the proposed method can produce state-of-the-art fusion results in both the standard and few-shot cases. The source code is publicly available at https://github.com/RSMagneto/UTSN.

show abstract

Transfer Representation Learning Meets Multimodal Fusion Classification for Remote Sensing Images

Cited by 15 publications

References 48 publications

Two-stage Transfer Learning for Airborne Multi-spectral Image Classifiers

Two-stage Transfer Learning for Airborne Multi-spectral Image Classifiers

MMLN: Multi-directional and Multi-constraint Learning Network for Remote Sensing Imagery Semantic Segmentation

A Unified Two-Stage Spatial and Spectral Network With Few-Shot Learning for Pansharpening

Contact Info

Product

Resources

About