All Grains, One Scheme (AGOS): Learning Multigrain Instance Representation for Aerial Scene Classification

Bi, Qi; Zhou, Beichen; Qin, Kun; Ye, Qinghao; Xia, Gui-Song

doi:10.1109/tgrs.2022.3201755

Cited by 20 publications

(10 citation statements)

References 87 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Medical image segmentation has been developed rapidly owing to the stronger representation from deep learning techniques (Bi et al 2022;Ji et al 2022;Li et al 2021a).…”

Section: Related Workmentioning

confidence: 99%

Learning Generalized Medical Image Segmentation from Decoupled Feature Queries

Bi,

Yi,

Zheng

et al. 2024

AAAI

View full text Add to dashboard Cite

Domain generalized medical image segmentation requires models to learn from multiple source domains and generalize well to arbitrary unseen target domain. Such a task is both technically challenging and clinically practical, due to the domain shift problem (i.e., images are collected from different hospitals and scanners). Existing methods focused on either learning shape-invariant representation or reaching consensus among the source domains. An ideal generalized representation is supposed to show similar pattern responses within the same channel for cross-domain images. However, to deal with the significant distribution discrepancy, the network tends to capture similar patterns by multiple channels, while different cross-domain patterns are also allowed to rest in the same channel. To address this issue, we propose to leverage channel-wise decoupled deep features as queries. With the aid of cross-attention mechanism, the long-range dependency between deep and shallow features can be fully mined via self-attention and then guides the learning of generalized representation. Besides, a relaxed deep whitening transformation is proposed to learn channel-wise decoupled features in a feasible way. The proposed decoupled fea- ture query (DFQ) scheme can be seamlessly integrate into the Transformer segmentation model in an end-to-end manner. Extensive experiments show its state-of-the-art performance, notably outperforming the runner-up by 1.31% and 1.98% with DSC metric on generalized fundus and prostate benchmarks, respectively. Source code is available at https://github.com/BiQiWHU/DFQ.

show abstract

“…Medical image segmentation has been developed rapidly owing to the stronger representation from deep learning techniques (Bi et al 2022;Ji et al 2022;Li et al 2021a).…”

Section: Related Workmentioning

confidence: 99%

Learning Generalized Medical Image Segmentation from Decoupled Feature Queries

Bi,

Yi,

Zheng

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Wang and Lan (2021) and Miao et al (2023) proposed similar single-CNN methods for fusing the deformed or decoupled features. Bi et al (2022) proposed a single-CNN method by fusing reformulated features through a so-called multigrain-perception module. Lv et al (2022) proposed a single-ViT method using a channel-attention module to fuse the channel and spatial features.…”

Section: Related Workmentioning

confidence: 99%

“…(2023) proposed similar single-CNN methods for fusing the deformed or decoupled features. Bi et al. (2022) proposed a single-CNN method by fusing reformulated features through a so-called multigrain-perception module.…”

Section: Related Workmentioning

confidence: 99%

MBC-Net: long-range enhanced feature fusion for classifying remote sensing images

Song

2023

IJICC

View full text Add to dashboard Cite

PurposeClassification of remote sensing images (RSI) is a challenging task in computer vision. Recently, researchers have proposed a variety of creative methods for automatic recognition of RSI, and feature fusion is a research hotspot for its great potential to boost performance. However, RSI has a unique imaging condition and cluttered scenes with complicated backgrounds. This larger difference from nature images has made the previous feature fusion methods present insignificant performance improvements.Design/methodology/approachThis work proposed a two-convolutional neural network (CNN) fusion method named main and branch CNN fusion network (MBC-Net) as an improved solution for classifying RSI. In detail, the MBC-Net employs an EfficientNet-B3 as its main CNN stream and an EfficientNet-B0 as a branch, named MC-B3 and BC-B0, respectively. In particular, MBC-Net includes a long-range derivation (LRD) module, which is specially designed to learn the dependence of different features. Meanwhile, MBC-Net also uses some unique ideas to tackle the problems coming from the two-CNN fusion and the inherent nature of RSI.FindingsExtensive experiments on three RSI sets prove that MBC-Net outperforms the other 38 state-of-the-art (STOA) methods published from 2020 to 2023, with a noticeable increase in overall accuracy (OA) values. MBC-Net not only presents a 0.7% increased OA value on the most confusing NWPU set but also has 62% fewer parameters compared to the leading approach that ranks first in the literature.Originality/valueMBC-Net is a more effective and efficient feature fusion approach compared to other STOA methods in the literature. Given the visualizations of grad class activation mapping (Grad-CAM), it reveals that MBC-Net can learn the long-range dependence of features that a single CNN cannot. Based on the tendency stochastic neighbor embedding (t-SNE) results, it demonstrates that the feature representation of MBC-Net is more effective than other methods. In addition, the ablation tests indicate that MBC-Net is effective and efficient for fusing features from two CNNs.

show abstract

“…LGRINet (Xu et al, 2022a(Xu et al, , 2022b(Xu et al, , 2022c(Xu et al, , 2022d 4 (Shen et al, 2022a) 3.8 95.32 (TR-60%) TST-Net (Chen et al, 2018) KD 1.0 80.00(TR-60%) ESD-MBENet (Zhao et al, 2022) 23.9 93.05 6 0.18 95.36 6 0.14 ET-GSNet (Xu et al, 2022a(Xu et al, , 2022b(Xu et al, , 2022c(Xu et al, , 2022d (Bi et al, 2022) Feature refining >12.5 93.04 6 0.35 94.91 6 0.17 MGS-Net (Guo et al, 2022) 244.2 91.92 6 0.12 94.33 6 0.08 GSCCTL-Net (Song and Yang, 2022) None 91.96 None ViT-Huge (Bazi et al, 2021) Single ViT 86 93.83 6 0.46 None ViT-AEv2 (Wang et al, 2023) 18.8 94.41 6 0.11 95.60 6 0.06 SC-ViT (Lv et al, 2022) 40.1 92.72 6 0.04 94.66 6 0.10 DFAGC-Net (Xu et al, 2022a(Xu et al, , 2022b(Xu et al, , 2022c(Xu et al, , 2022d Multiple models None None 89.29 6 0.28 GRMA-Net (Li et al, 2022) 54.1 93.67 6 0.21 95.32 6 0.28 ACNet (Tang et al, 2021) >276.6 91.09 6 0.13 92.42 6 0.16 T-CNN (Wang et al, 2022b) 15.9 90.25 6 0.14 93.05 6 0.12 GLDBS-Net (Xu et al, 2022a(Xu et al, , 2022b(Xu et al, , 2022c…”

Section: Efficient Knowledge Distillationmentioning

confidence: 99%

“…Similarly, Shi et al (2022) and Bai et al (2022) introduced their methods using variants of standard convolution structures. Moreover, Zhang et al (2022), Bi et al (2022), Guo et al (2022), Song and Yang (2022) and Miao et al (2023) also proposed CNN feature refining methods using Laplacian blocks, multigrain formulation, multi-granularity learning, scene clustering, or multi-granularity decoupling, respectively. Given that module insertion can weaken the pre-training effect, these studies should initially pre-train their modified deep models on large-scale data sets to ensure their training begins from a more advantageous point (Song and Zhou, 2023).…”

Section: Related Workmentioning

confidence: 99%

Efficient knowledge distillation for remote sensing image classification: a CNN-based approach

Song,

Wei,

Yong

2023

IJWIS

View full text Add to dashboard Cite

Purpose The paper aims to tackle the classification of Remote Sensing Images (RSIs), which presents a significant challenge for computer algorithms due to the inherent characteristics of clustered ground objects and noisy backgrounds. Recent research typically leverages larger volume models to achieve advanced performance. However, the operating environments of remote sensing commonly cannot provide unconstrained computational and storage resources. It requires lightweight algorithms with exceptional generalization capabilities. Design/methodology/approach This study introduces an efficient knowledge distillation (KD) method to build a lightweight yet precise convolutional neural network (CNN) classifier. This method also aims to substantially decrease the training time expenses commonly linked with traditional KD techniques. This approach entails extensive alterations to both the model training framework and the distillation process, each tailored to the unique characteristics of RSIs. In particular, this study establishes a robust ensemble teacher by independently training two CNN models using a customized, efficient training algorithm. Following this, this study modifies a KD loss function to mitigate the suppression of non-target category predictions, which are essential for capturing the inter- and intra-similarity of RSIs. Findings This study validated the student model, termed KD-enhanced network (KDE-Net), obtained through the KD process on three benchmark RSI data sets. The KDE-Net surpasses 42 other state-of-the-art methods in the literature published from 2020 to 2023. Compared to the top-ranked method’s performance on the challenging NWPU45 data set, KDE-Net demonstrated a noticeable 0.4% increase in overall accuracy with a significant 88% reduction in parameters. Meanwhile, this study’s reformed KD framework significantly enhances the knowledge transfer speed by at least three times. Originality/value This study illustrates that the logit-based KD technique can effectively develop lightweight CNN classifiers for RSI classification without substantial sacrifices in computation and storage costs. Compared to neural architecture search or other methods aiming to provide lightweight solutions, this study’s KDE-Net, based on the inherent characteristics of RSIs, is currently more efficient in constructing accurate yet lightweight classifiers for RSI classification.

show abstract

All Grains, One Scheme (AGOS): Learning Multigrain Instance Representation for Aerial Scene Classification

Cited by 20 publications

References 87 publications

Learning Generalized Medical Image Segmentation from Decoupled Feature Queries

Learning Generalized Medical Image Segmentation from Decoupled Feature Queries

MBC-Net: long-range enhanced feature fusion for classifying remote sensing images

Efficient knowledge distillation for remote sensing image classification: a CNN-based approach

Contact Info

Product

Resources

About