An Adversarial Approach to Discriminative Modality Distillation for Remote Sensing Image Classification

Pande, Shivam; Banerjee, Avinandan; Kumar, Saurabh; Banerjee, Biplab; Chaudhuri, Subhasis

doi:10.1109/iccvw.2019.00558

Cited by 21 publications

(5 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, distillation loss incorporates the knowledge transfer process into the main downstream task. Additionally, architectures utilizing hallucination streams often require multistage training rather than an end-to-end approach [82], [93], [94]. Variations of CNN architecture dominate in this category.…”

Section: B Knowledge Distillationmentioning

confidence: 99%

“…In Pande et al [82], GANs are integrated into a teacher-student training paradigm. On the other hand, the works of Wei et al [84] and Li et al [83] underscore the utility of hallucination branches.…”

Section: Remote Sensingmentioning

confidence: 99%

“…Gray color means no relevant result reported for a given paper. Hall Net [81], GAN Building [77], C-GAN Distillation [82], DH-ADNet [83], DiffAlignOp [12], MSH-Net [84], Social Sensing [85], Hall Two-stream [86], DisOptNet [87], Bi-LSTM RS [64]. Table I provides more information about datasets and backbones of each reviewed method.…”

Section: A Data Noisementioning

confidence: 99%

See 2 more Smart Citations

Multimodal Colearning Meets Remote Sensing: Taxonomy, State of the Art, and Future Works

Kieu,

Nguyen,

Nazib

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

In remote sensing (RS), multiple modalities of data are usually available, e.g., RGB, Multispectral, Hyperspectral, LiDAR, and SAR. Multimodal machine learning systems, which fuse these rich multimodal data modalities, have shown better performance compared to unimodal systems. Most multimodal research assumes that all modalities are present, aligned, and noiseless during training and testing time. However, in real-world scenarios, it is common to observe that one or more modalities are missing, noisy, and non-aligned, in either training or testing or both. In addition, acquiring largescale, noise-free annotations is expensive, as a result, lacking sufficient annotated datasets or having to deal with inconsistent labels are open challenges. These challenges can be addressed under a learning paradigm called multimodal co-learning.This paper focuses on multimodal co-learning techniques for remote sensing data. We first review what data modalities are available in the remote sensing domain and the key benefits and challenges of combining multimodal data in the remote sensing context. We then review the remote sensing tasks that would benefit from multimodal processing including classification, segmentation, target detection, anomaly detection, and temporal change detection. We then dive deeper into technical details by reviewing more than 200 recent efforts in this area and provide a comprehensive taxonomy to systematically review state-of-the-art approaches in 4 key co-learning challenges including missing modalities, noisy modalities, limited modality annotations, and weakly-paired modalities. Based on these insights, we propose emerging research directions to inform potential future research in multimodal co-learning for remote sensing.

show abstract

Section: B Knowledge Distillationmentioning

confidence: 99%

Section: Remote Sensingmentioning

confidence: 99%

See 1 more Smart Citation

Multimodal Colearning Meets Remote Sensing: Taxonomy, State of the Art, and Future Works

Kieu,

Nguyen,

Nazib

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…A knowledge distillation framework is proposed in [46], which makes the output of the student and teacher models match. The discriminative modality distillation approach is introduced in [47], the teacher is trained on multimodal data, and then, the student model learns from the teacher model to improve the performance of the remote sensing image classifications. To address the problem of network overfitting due to noisy data, a novel noisy label distillation method (NLD) is proposed in [48].…”

Section: Knowledge Distillation and Sdmentioning

confidence: 99%

Embedded Self-Distillation in Compact Multibranch Ensemble Network for Remote Sensing Scene Classification

Zhao

Lyu

et al. 2022

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

show abstract

“…A knowledge distillation framework is proposed in [47], which makes the output of the student and teacher models match. Discriminative modality distillation approach is introduced in [48], the teacher is trained on multimodal data and then the student model learns from the teacher model to improve the performance of the RS image classifiction. To address the problem of network overfitting due to noisy data, a novel noisy label distillation method (NLD) is proposed in [49].…”

Section: Knowledge Distillation and Self-distillationmentioning

confidence: 99%

Embedded Self-Distillation in Compact Multi-Branch Ensemble Network for Remote Sensing Scene Classification

Zhao,

Ma,

Lyu

et al. 2021

Preprint

View full text Add to dashboard Cite

Remote sensing (RS) image scene classification task faces many challenges due to the interference from different characteristics of different geographical elements. To solve this problem, we propose a multi-branch ensemble network to enhance the feature representation ability by fusing features in final output logits and intermediate feature maps. However, simply adding branches will increase the complexity of models and decline the inference efficiency. On this issue, we embed selfdistillation (SD) method to transfer knowledge from ensemble network to main-branch in it. Through optimizing with SD, main-branch will have close performance as ensemble network. During inference, we can cut other branches to simplify the whole model. In this paper, we first design compact multibranch ensemble network, which can be trained in an endto-end manner. Then, we insert SD method on output logits and feature maps. Compared to previous methods, our proposed architecture (ESD-MBENet) performs strongly on classification accuracy with compact design. Extensive experiments are applied on three benchmark RS datasets AID, NWPU-RESISC45 and UC-Merced with three classic baseline models, VGG16, ResNet50 and DenseNet121. Results prove that our proposed ESD-MBENet can achieve better accuracy than previous state-of-the-art (SOTA) complex models. Moreover, abundant visualization analysis make our method more convincing and interpretable.

show abstract

An Adversarial Approach to Discriminative Modality Distillation for Remote Sensing Image Classification

Cited by 21 publications

References 37 publications

Multimodal Colearning Meets Remote Sensing: Taxonomy, State of the Art, and Future Works

Multimodal Colearning Meets Remote Sensing: Taxonomy, State of the Art, and Future Works

Embedded Self-Distillation in Compact Multibranch Ensemble Network for Remote Sensing Scene Classification

Embedded Self-Distillation in Compact Multi-Branch Ensemble Network for Remote Sensing Scene Classification

Contact Info

Product

Resources

About