Two-stream feature aggregation deep neural network for scene classification of remote sensing images

Xu, Kejie; Hong, Huixiao; Deng, Peifang; Shi, Guangyao

doi:10.1016/j.ins.2020.06.011

Cited by 41 publications

(16 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Training-to-test set ratio 20% 50% PLSA(SIFT) [10] 56.24±0.58 63.07±1.77 BoVW(SIFT) [10] 62.49±0.53 68.37±0.40 AlexNet [10] 86.86±0.47 89.53±0.31 VGGNet-16 [10] 86.59±0.29 89.64±0.36 GoogLeNet [10] 83.44±0.40 86.39±0.55 CaffeNet [10] 86.86±0.47 89.53±0.31 TEX-Net with VGG [41] 87.32±0.37 90.00±0.33 D-CNN with AlexNet [13] 85.62±0.10 94.47±0.12 SPP with AlexNet [19] 87.44±0.45 91.45±0.38 Two-Stream Fusion [26] 92.32±0.41 94.58±0.25 Fusion by addition [20] --91.87±0.36 MIDC-Net [66] 88.51±0.41 92.95±0.17 Gated attention [64] 87.63±0.44 92.01±0.21 DFAGCN [44] --94.88±0.22 TFADNN [67] 93.21±0.32 95.04±0.16 Inception-v3-CapsNet [34] 93.79±0.13 96.32±0.12 Backbone (Xception) [47] 86.12±0.28 90.14±0.52 CSDS (ours) 94.29±0.35 96.70±0.14…”

Section: Methodsmentioning

confidence: 99%

CSDS: End-to-End Aerial Scenes Classification With Depthwise Separable Convolution and an Attention Mechanism

Wang

Yuan

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Compared with natural scenes, aerial scenes are usually composed of numerous objects densely distributed within the aerial view, and thus more key local semantic features are needed to describe them. However, when existing CNNs are used for remote sensing image classification, they typically focus on the global semantic features of the image, and especially for deep models, shallow and intermediate features are easily lost. This paper proposes a channel-spatial attention mechanism based on a depthwise separable convolution (CSDS) network for aerial scene classification to solve these challenges. First, we construct a depthwise separable convolution (DS-Conv) and pyramid residual connection architecture.DS-Conv extracts features from each channel and merges them, effectively reducing the number of necessary calculations, and the pyramid residual connections connect the features from multiple layers and create associations. Then, the channel-spatial attention algorithm causes the model to obtain more effective features in the channel and spatial domains. Finally, an improved cross-entropy loss function is used to reduce the impact of similar categories on backpropagation. Comparative experiments on three public data sets show that the CSDS network can achieve results comparable to those of other state-of-the-art methods. In addition, visualization of feature extraction results by the Grad-CAM algorithm and ablation experiments for each module reflect the powerful feature learning and representation capabilities of the proposed CSDS network.

show abstract

Section: Methodsmentioning

confidence: 99%

CSDS: End-to-End Aerial Scenes Classification With Depthwise Separable Convolution and an Attention Mechanism

Wang

Yuan

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Dong et al [16] fused the deep CNN features with the GIST features before the classification by LSTM. Xu et al [17] proposed a two-stream feature fusion method, where one stream provided features using a pretrained CNN while the other output the multi-scale unsupervised MNBOVW features. Shi et al [19] proposed a network with several groups, each of which had two branches for feature extraction and fusion respectively.…”

Section: The Feature-level Fusion-based Methodsmentioning

confidence: 99%

“…Freeway Basketball_Court Golf_Course Ship Thermal_Power_Station Tennis_Court Island In recent years, different strategies of feature-level fusion have already been investigated [16][17][18][19][20], which could produce more discriminative features and better performance than the methods that only utilize deep features. Nevertheless, such feature-level fusion still has shortcomings.…”

Section: Airplanementioning

confidence: 99%

“…In this section, we briefly review the related work of remote sensing scene classification. The existing methods with CNN could be divided into three categories: the conventional CNN features-based methods [12,24], the feature-level fusion-based methods [16][17][18][19][20][25][26][27][28][29][30][31][32][33][34] , and the decision-level fusion-based methods [35,36].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Decision-Level Fusion with a Pluginable Importance Factor Generator for Remote Sensing Image Scene Classification

et al. 2021

View full text Add to dashboard Cite

Remote sensing image scene classification acts as an important task in remote sensing image applications, which benefits from the pleasing performance brought by deep convolution neural networks (CNNs). When applying deep models in this task, the challenges are, on one hand, that the targets with highly different scales may exist in the image simultaneously and the small targets could be lost in the deep feature maps of CNNs; and on the other hand, the remote sensing image data exhibits the properties of high inter-class similarity and high intra-class variance. Both factors could limit the performance of the deep models, which motivates us to develop an adaptive decision-level information fusion framework that can incorporate with any CNN backbones. Specifically, given a CNN backbone that predicts multiple classification scores based on the feature maps of different layers, we develop a pluginable importance factor generator that aims at predicting a factor for each score. The factors measure how confident the scores in different layers are with respect to the final output. Formally, the final score is obtained by a class-wise and weighted summation based on the scores and the corresponding factors. To reduce the co-adaptation effect among the scores of different layers, we propose a stochastic decision-level fusion training strategy that enables each classification score to randomly participate in the decision-level fusion. Experiments on four popular datasets including the UC Merced Land-Use dataset, the RSSCN 7 dataset, the AID dataset, and the NWPU-RESISC 45 dataset demonstrate the superiority of the proposed method over other state-of-the-art methods.

show abstract

“…With the growing amount of data, there is a practical need for a faster and more accurate automated approach to extract their semantic content information and to identify and classify land use and land cover (LULC) types in those images. RS image scene classification [5][6][7] is one crucial way to help alleviate the problem mentioned above since it automatically assigns semantic labels to an RS image scene and has been widely studied due to its vital contributions in land resources planning [8], disaster monitoring [9], urban planning [10], object detection [11], and many other RS applications [12][13][14][15].…”

Section: Introductionmentioning

confidence: 99%

A Convolutional Neural Network Based on Grouping Structure for Scene Classification

Zhang

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

Convolutional neural network (CNN) is capable of automatically extracting image features and has been widely used in remote sensing image classifications. Feature extraction is an important and difficult problem in current research. In this paper, data augmentation for avoiding over fitting was attempted to enrich features of samples to improve the performance of a newly proposed convolutional neural network with UC-Merced and RSI-CB datasets for remotely sensed scene classifications. A multiple grouped convolutional neural network (MGCNN) for self-learning that is capable of promoting the efficiency of CNN was proposed, and the method of grouping multiple convolutional layers capable of being applied elsewhere as a plug-in model was developed. Meanwhile, a hyper-parameter C in MGCNN is introduced to probe into the influence of different grouping strategies for feature extraction. Experiments on the two selected datasets, the RSI-CB dataset and UC-Merced dataset, were carried out to verify the effectiveness of this newly proposed convolutional neural network, the accuracy obtained by MGCNN was 2% higher than the ResNet-50. An algorithm of attention mechanism was thus adopted and incorporated into grouping processes and a multiple grouped attention convolutional neural network (MGCNN-A) was therefore constructed to enhance the generalization capability of MGCNN. The additional experiments indicate that the incorporation of the attention mechanism to MGCNN slightly improved the accuracy of scene classification, but the robustness of the proposed network was enhanced considerably in remote sensing image classifications.

show abstract

Two-stream feature aggregation deep neural network for scene classification of remote sensing images

Cited by 41 publications

References 46 publications

CSDS: End-to-End Aerial Scenes Classification With Depthwise Separable Convolution and an Attention Mechanism

CSDS: End-to-End Aerial Scenes Classification With Depthwise Separable Convolution and an Attention Mechanism

Decision-Level Fusion with a Pluginable Importance Factor Generator for Remote Sensing Image Scene Classification

A Convolutional Neural Network Based on Grouping Structure for Scene Classification

Contact Info

Product

Resources

About