Multilayer Feature Fusion Network With Spatial Attention and Gated Mechanism for Remote Sensing Scene Classification

Meng, Qingyan; Zhao, Maofan; Zhang, Linlin; Shi, Wenxu; Su, Chen; Bruzzone, Lorenzo

doi:10.1109/lgrs.2022.3173473

Cited by 18 publications

(9 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second, researchers proposed different single-model methods by employing functional modules as auxiliary fusion tools. For example, Meng et al. (2022) proposed a single-CNN method by using two self-designed modules for multilayer feature fusion.…”

Section: Related Workmentioning

confidence: 99%

“…Second, researchers proposed different single-model methods by employing functional modules as auxiliary fusion tools. For example, Meng et al (2022) proposed a single-CNN method by using two self-designed modules for multilayer feature fusion. Tian et al (2021) and Wan et al (2021) proposed two different single-CNN methods by fusing multi-scale features, while the former employs a DenseNet and the latter uses a ResNeXt.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

MBC-Net: long-range enhanced feature fusion for classifying remote sensing images

Song

2023

IJICC

View full text Add to dashboard Cite

PurposeClassification of remote sensing images (RSI) is a challenging task in computer vision. Recently, researchers have proposed a variety of creative methods for automatic recognition of RSI, and feature fusion is a research hotspot for its great potential to boost performance. However, RSI has a unique imaging condition and cluttered scenes with complicated backgrounds. This larger difference from nature images has made the previous feature fusion methods present insignificant performance improvements.Design/methodology/approachThis work proposed a two-convolutional neural network (CNN) fusion method named main and branch CNN fusion network (MBC-Net) as an improved solution for classifying RSI. In detail, the MBC-Net employs an EfficientNet-B3 as its main CNN stream and an EfficientNet-B0 as a branch, named MC-B3 and BC-B0, respectively. In particular, MBC-Net includes a long-range derivation (LRD) module, which is specially designed to learn the dependence of different features. Meanwhile, MBC-Net also uses some unique ideas to tackle the problems coming from the two-CNN fusion and the inherent nature of RSI.FindingsExtensive experiments on three RSI sets prove that MBC-Net outperforms the other 38 state-of-the-art (STOA) methods published from 2020 to 2023, with a noticeable increase in overall accuracy (OA) values. MBC-Net not only presents a 0.7% increased OA value on the most confusing NWPU set but also has 62% fewer parameters compared to the leading approach that ranks first in the literature.Originality/valueMBC-Net is a more effective and efficient feature fusion approach compared to other STOA methods in the literature. Given the visualizations of grad class activation mapping (Grad-CAM), it reveals that MBC-Net can learn the long-range dependence of features that a single CNN cannot. Based on the tendency stochastic neighbor embedding (t-SNE) results, it demonstrates that the feature representation of MBC-Net is more effective than other methods. In addition, the ablation tests indicate that MBC-Net is effective and efficient for fusing features from two CNNs.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

MBC-Net: long-range enhanced feature fusion for classifying remote sensing images

Song

2023

IJICC

View full text Add to dashboard Cite

show abstract

“…Among these deep learning based methods, CNNs are the most commonly-utilized [2], [18]- [21], [44] as the convolutional filters are effective to extract multi-level features from the image. In the past two years, CNN based methods (e.g., DSENet [45], MS2AP [46], MSDFF [47], CADNet [48], LSENet [5], GBNet [49], MBLANet [50], MG-CAP [51], Contourlet CNN [52], STHP [53], SAGM [54], DARTS [55], LML [56], GCSANet [57]) still remain heated for aerial scene classification. On the other hand, recurrent neural network (RNN) based [25], auto-encoder based [58], [59] and generative adversarial network (GAN) based [60], [61] approaches have also been reported effective for aerial scene classification.…”

Section: A Aerial Scene Classificationmentioning

confidence: 99%

“…We compare the performance of our AGOS with three handcrafted features (PLSA, BOW, LDA) [17], [87], three typical CNN models (AlexNet, VGG, GoogLeNet) [17], [87], twentytwo latest CNN-based state-of-the-art approaches (MIDCNet [2], RANet [29], APNet [88], SPPNet [20], DCNN [28], TEXNet [89], MSCP [18], VGG+FV [21], DSENet [45], MS2AP [46], MSDFF [47], CADNet [48], LSENet [5], GBNet [49], MBLANet [50], MG-CAP [51], Contourlet CNN [52], STHP [53], SAGM [54], DARTS [55], LML [56], GCSANet [57]), one RNN-based approach (ARCNet [25]), two autoencoder based approaches (SGUFL [59], PARTLETS [58]) and two GAN-based approaches (MARTA [60], AGAN [61]) respectively. The performance under the backbone of ResNet-50, ResNet-101 and DenseNet-121 is all reported for fair evaluation as some latest methods [47], [48] use much deeper networks as backbone.…”

Section: Comparison With State-of-the-art Approachesmentioning

confidence: 99%

All Grains, One Scheme (AGOS): Learning Multigrain Instance Representation for Aerial Scene Classification

Zhou

Qin

et al. 2022

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

Aerial scene classification remains challenging as: 1) the size of key objects in determining the scene scheme varies greatly; 2) many objects irrelevant to the scene scheme are often flooded in the image. Hence, how to effectively perceive the region of interests (RoIs) from a variety of sizes and build more discriminative representation from such complicated object distribution is vital to understand an aerial scene. In this paper, we propose a novel all grains, one scheme (AGOS) framework to tackle these challenges. To the best of our knowledge, it is the first work to extend the classic multiple instance learning into multi-grain formulation. Specially, it consists of a multigrain perception module (MGP), a multi-branch multi-instance representation module (MBMIR) and a self-aligned semantic fusion (SSF) module. Firstly, our MGP preserves the differential dilated convolutional features from the backbone, which magnifies the discriminative information from multi-grains. Then, our MBMIR highlights the key instances in the multi-grain representation under the MIL formulation. Finally, our SSF allows our framework to learn the same scene scheme from multi-grain instance representations and fuses them, so that the entire framework is optimized as a whole. Notably, our AGOS is flexible and can be easily adapted to existing CNNs in a plug-andplay manner. Extensive experiments on UCM, AID and NWPU benchmarks demonstrate that our AGOS achieves a comparable performance against the state-of-the-art methods.

show abstract

“…Although CNNs can automatically extract abundant features from RS images, they are not good at mining strong discriminative information for complicated RS scenes with high intra-class variability, large inter-class similarity, and various objects with different scales, leading to limited performance. In human perception, “attention” plays a very important role in deciding “what” and “where” to focus, 43 thereby enhancing the representation of target objects. Motivated by it, we introduce the idea of “attention” into the architecture and propose a novel MAANet with a series of specially designed attention models for dealing with the above challenges.…”

Section: Introductionmentioning

confidence: 99%

Multi-attention aggregation network for remote sensing scene classification

Wang,

Li,

Shi

et al. 2023

J. Appl. Rem. Sens.

View full text Add to dashboard Cite

.Remote sensing (RS) scene classification is a highly challenging task because of the unique characteristics of RS scenes, such as high intra-class variability, large inter-class similarity, and various objects with different scales. Attention, interpreted as an important mechanism of the human visual system, can emphasize meaningful features of deep neural networks, which is beneficial for boosting the classification performance. Motivated by it, we present a multi-attention aggregation network (MAANet), which contains various specially designed attention models, for precise RS scene classification. First, a gated attention fluid coding structure is constructed for mining hierarchical gated attention features from RS images. Second, a progressive pyramid refinement architecture is designed to explore correlations of cross-layer attention features to learn enhanced multi-scale representations. Third, a two-stream attention aggregation structure, equipped with three different attention models, is developed to guide the generation of aggregated features. Finally, a scene label prediction module is proposed for scene label prediction. We conduct extensive experiments on three famous RS scene datasets, and the experimental results show that our MAANet outperforms a number of current representative state-of-the-art approaches for the RS scene classification task.

show abstract

Multilayer Feature Fusion Network With Spatial Attention and Gated Mechanism for Remote Sensing Scene Classification

Cited by 18 publications

References 24 publications

MBC-Net: long-range enhanced feature fusion for classifying remote sensing images

MBC-Net: long-range enhanced feature fusion for classifying remote sensing images

All Grains, One Scheme (AGOS): Learning Multigrain Instance Representation for Aerial Scene Classification

Multi-attention aggregation network for remote sensing scene classification

Contact Info

Product

Resources

About