A Multi-Scale Approach for Remote Sensing Scene Classification Based on Feature Maps Selection and Region Representation

Abstract

Scene classification is one of the bases for automatic remote sensing image interpretation. Recently, deep convolutional neural networks have presented promising performance in high-resolution remote sensing scene classification research. In general, most researchers directly use raw deep features extracted from the convolutional networks to classify scenes. However, this strategy only considers single scale features, which cannot describe both the local and global features of images. In fact, the dissimilarity of scene targets in the same category may result in convolutional features being unable to classify them into the same category. Besides, the similarity of the global features in different categories may also lead to failure of fully connected layer features to distinguish them. To address these issues, we propose a scene classification method based on multi-scale deep feature representation (MDFR), which mainly includes two contributions: (1) region-based features selection and representation; and (2) multi-scale features fusion. Initially, the proposed method filters the multi-scale deep features extracted from pre-trained convolutional networks. Subsequently, these features are fused via two efficient fusion methods. Our method utilizes the complementarity between local features and global features by effectively exploiting the features of different scales and discarding the redundant information in features. Experimental results on three benchmark high-resolution remote sensing image datasets indicate that the proposed method is comparable to some state-of-the-art algorithms.

Remote sensing scene classification based on high-order graph convolutional network

“…It has been proven that our method can learn more discriminative feature representation by combining attention CNN features and the relation-aware ability of high-order GCN. Compared with CaffeNet (Xia et al, 2017), VGG-VD -16 (Xia et al, 2017) and GoogLeNet (Xia et al, 2017) which only use the pre-trained CNN model, MCNN (Liu et al, 2017), MDFR (Zhang et al, 2019) and conv5-MSP5-FV (Zheng et al, 2019) consider the variation of the scene at different scales and thus have relatively better classification performance. It should be noted that the results of conv5-MSP5-FV are the best results for the UCM dataset (Zheng et al, 2019).…”

“…Liu et al (Liu et al, 2017) propose the multiscale CNN (MCNN) to alleviate the influence of the scale variation to the semantic objects. Zhang et al (Zhang et al, 2019) employ multi-scale deep feature representation (MDFR) for exploring the features of different scales. Zheng et al (Zheng et al, 2019) use the multiscale pooling (MSP) strategy to gain the multiscale invariant scene representation.…”

“…In this section, we conduct extensive experiments on UCM, RSSCN7 and AID datasets and compare the proposed method with CaffeNet (Cheng, Han et al, 2017a;Xia et al, 2017), VGG-VD-16 (Cheng, Han et al, 2017a;Xia et al, 2017), GoogLeNet (Cheng, Han et al, 2017a;Xia et al, 2017), BoCF (Cheng, Li et al, 2017b), Two-Stage Fusion (Liu et al, 2017), MCNN (Liu et al, 2017), MDFR (Zhang et al, 2019), conv5-MSP5-FV (Zheng et al, 2019) which are the representative remote sensing scene classification methods based on CNN in the past few years. Furthermore, considering the attention mechanism used in our method, the attention-based scene classification methods, e.g., APDC-Net (Bi et al, 2019), ADFF (Zhu et al, 2019), the method proposed by Fan et al (Fan et al, 2019), HoSA (He et al, 2019), Attention GANs (Yu et al, 2020) and SAFF (Cao et al, 2021), are also used for comparison.…”

“…More importantly, the highlevel semantics cannot be revealed by these methods. In the past few years, Convolutional Neural Network (CNN) has been proposed and widely used in remote sensing scene classification (Cheng, Han et al, 2017a;Cheng, Li et al, 2017b;Cheng et al, 2020Cheng et al, , 2018Lan et al, 2020;Liu et al, 2017Liu et al, , 2018Xia et al, 2017;Zhang et al, 2019;Zheng et al, 2019). The CNNbased methods generally use the hierarchical deep architecture of CNN to automatically learn high-level features of remote sensing images.…”

“…3) We conduct comprehensive experiments to evaluate the proposed method and compare it with the related methods (Bi et al, 2019;Cao et al, 2021;Cheng, Li et al, 2017b;Fan et al, 2019;He et al, 2019;Liu et al, 2017Liu et al, , 2018Xia et al, 2017;Yu et al, 2020;Zhang et al, 2019;Zheng et al, 2019;Zhu et al, 2019) on four public remote sensing image datasets (i.e., UCM, RSSCN7, AID and NWPU-RESISC45). The experimental results show that the semantics correlation embedding is conducive to enhancing the representational ability of CNN features and especially the relation-aware features yielded by H-GCN is promising to improve the remote sensing scene classification performance.…”