Novel Cross-Resolution Feature-Level Fusion for Joint Classification of Multispectral and Panchromatic Remote Sensing Images

Liu, Sicong; Zhao, Hui; Du, Qian; Bruzzone, Lorenzo; Samat, Alim; Tong, Xiaohua

doi:10.1109/tgrs.2021.3127710

Cited by 13 publications

(5 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fusion of shallow artificial features and deep features has shown its effectiveness in multi-source image classification [48,49]. In order to offer a more comprehensive representation for Martian landforms from the single-band and gray-scale images, the extracted abstract convolutional features from scene-level view and the multi-texture features from local landform view are fused.…”

Section: ) Multi-view Features Fusion and Classificationmentioning

confidence: 99%

MarsMapNet: A Novel Superpixel-Guided Multiview Feature Fusion Network for Efficient Martian Landform Mapping

Zhao,

Liu,

Tong

et al. 2024

IEEE Trans. Geosci. Remote Sensing

Self Cite

View full text Add to dashboard Cite

Abstract-Landform classification and mapping of the Martian surface using Mars orbiter images can provide an important reference for landing site selection and rovers' traversability evaluation in Mars exploration. Moreover, specific Martian landforms are closely associated with the evidences of water-related activities and Martian life, thus have crucial research importance. This article proposes a novel superpixelguided multi-view feature fusion network (MarsMapNet) for efficient mapping of the Martian landforms. In particular, the proposed MarsMapNet first generates the superpixel-level segments from Mars orbiter images by considering local morphological homogeneity of landforms. Then a multi-view feature extraction and fusion (MVF) network is developed, where abstract convolutional features are extracted based on scene-level patches, and multi-textures are extracted based on local landform from shallow-to-deep feature learning. After the network being trained on scene-level samples and guided by the superpixel segmentation, Martian landforms can be correctly classified in an efficient way, whose mapping time cost sharply decreased when compared to the reference methods. The proposed MarsMapNet has been validated on three real landing sites from several Mars missions (i.e., the Jezero Crater, the Southern Utopia Planitia, the Oxia Planum) by using the Mars Reconnaissance Orbiter's Context Camera (CTX) images. Qualitative and quantitative analysis on the obtained experimental results confirm the effectiveness and efficiency of the proposed MarsMapNet when compared with the state-of-theart methods, demonstrating its potential for supporting a Martian global landform mapping in the future.

show abstract

Section: ) Multi-view Features Fusion and Classificationmentioning

confidence: 99%

MarsMapNet: A Novel Superpixel-Guided Multiview Feature Fusion Network for Efficient Martian Landform Mapping

Zhao,

Liu,

Tong

et al. 2024

IEEE Trans. Geosci. Remote Sensing

Self Cite

View full text Add to dashboard Cite

show abstract

“…Such big and rich HSI dataset, including the different spectral bands data related by the (spatial) geo-located position, may contain hidden information and patterns. HSI Classification (HSIC) [14], [15] aims at discover, detect, identify and recognize such patterns. However, the spectral dataset size usually increases combinatorially with the problem size (e.g.…”

Section: Introductionmentioning

confidence: 99%

Hybrid Dense Network With Attention Mechanism for Hyperspectral Image Classification

Khan

Mazzara

Distefano

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

The nonlinear relation between the spectral information and the corresponding objects (complex physiognomies) makes pixel-wise classification challenging for conventional methods. To deal with nonlinearity issues in Hyperspectral Image Classification (HSIC), Convolutional Neural Networks (CNN) are more suitable, indeed. However, fixed kernel sizes make traditional CNN too specific, neither flexible nor conducive to feature learning, thus impacting on the classification accuracy. The convolution of different kernel size networks may overcome this problem by capturing more discriminating and relevant information. In light of this, the proposed solution aims at combining the core idea of 3D and 2D Inception net with the Attention mechanism to boost the HSIC CNN performance in a hybrid scenario. The resulting Attention-fused Hybrid Network (AfNet) is based on three attention-fused parallel hybrid subnets with different kernels in each block repeatedly using highlevel features to enhance the final ground-truth maps. In short, AfNet is able to selectively filter out the discriminative features critical for classification. Several tests on HSI datasets provided competitive results for AfNet compared to state-of-the-art models.

show abstract

“…In line with this evolution, we propose to take the on-board processing of satellite acquired multi-spectral images to its the next logical level. Expanding from classification [4], [5] and segmentation tasks [6], [7], we seek to perform panoptic segmentation using limited on-board resources.…”

Section: Introductionmentioning

confidence: 99%

Towards On-Board Panoptic Segmentation of Multispectral Satellite Images

Fernando¹,

Fookes²,

Gammulle³

et al. 2022

Preprint

View full text Add to dashboard Cite

With tremendous advancements in low-power embedded computing devices and remote sensing instruments, the traditional satellite image processing pipeline which includes an expensive data transfer step prior to processing data on the ground is being replaced by on-board processing of captured data. This paradigm shift enables critical and time-sensitive analytic intelligence to be acquired in a timely manner onboard the satellite itself. However, at present, the on-board processing of multi-spectral satellite images is limited to classification and segmentation tasks. Extending this processing to it's next logical level, in this paper we propose a lightweight pipeline for onboard panoptic segmentation of multi-spectral satellite images. Panoptic segmentation offers major economic and environmental insights, ranging from yield estimation from agricultural lands to intelligence for complex military applications. Nevertheless, the on-board intelligence extraction raises several challenges due to the loss of temporal observations and the need to generate predictions from a single image sample. To address this challenge, we propose a multimodal teacher network based on a cross-modality attention-based fusion strategy to improve the segmentation accuracy by exploiting data from multiple modes. We also propose an online knowledge distillation framework to transfer the knowledge learned by this multi-modal teacher network to a uni-modal student which receives only a single frame input, and is more appropriate for an on-board environment. We benchmark our approach against existing state-ofthe-art panoptic segmentation models using the PASTIS multispectral panoptic segmentation dataset considering an on-board processing setting. Our evaluations demonstrate a substantial 10.7%, 11.9% and 10.6% increase in Segmentation Quality (SQ), Recognition Quality (RQ), and Panoptic Quality (PQ) metrics compared to the existing state-of-the-art model when it is evaluated in an on-board processing setting.

show abstract

Novel Cross-Resolution Feature-Level Fusion for Joint Classification of Multispectral and Panchromatic Remote Sensing Images

Cited by 13 publications

References 41 publications

MarsMapNet: A Novel Superpixel-Guided Multiview Feature Fusion Network for Efficient Martian Landform Mapping

MarsMapNet: A Novel Superpixel-Guided Multiview Feature Fusion Network for Efficient Martian Landform Mapping

Hybrid Dense Network With Attention Mechanism for Hyperspectral Image Classification

Towards On-Board Panoptic Segmentation of Multispectral Satellite Images

Contact Info

Product

Resources

About