Multilayer Feature Fusion Network for Scene Classification in Remote Sensing

Xu, Kejie; Hong, Huixiao; Li, Yuan; Shi, Guangyao

doi:10.1109/lgrs.2019.2960026

Cited by 77 publications

(26 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among the typical deep architectures, CNN provides a strong ability of feature extraction and yield significant performance improvement on scene classification. There have already been several attempts to use deep CNN features for classifying the remote sensing images [3][4][5]21,[42][43][44][45][46][47]. Wang et al [26] employed CaffeNet with the soft-max layer for scene classification.…”

Section: The Deep Learning-based Methodsmentioning

confidence: 99%

“…At the same time, this advantage posts the requirement of scene understanding which is to label the images with semantic tags based on the image content and then facilitate the automatic analysis of remote sensing images. Targeting at this, scene classification [1][2][3][4][5] has already been a popular research topic and has witnessed successful deployment in related applications.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Dual-Model Architecture with Grouping-Attention-Fusion for Remote Sensing Scene Classification

et al. 2021

View full text Add to dashboard Cite

Remote sensing images contain complex backgrounds and multi-scale objects, which pose a challenging task for scene classification. The performance is highly dependent on the capacity of the scene representation as well as the discriminability of the classifier. Although multiple models possess better properties than a single model on these aspects, the fusion strategy for these models is a key component to maximize the final accuracy. In this paper, we construct a novel dual-model architecture with a grouping-attention-fusion strategy to improve the performance of scene classification. Specifically, the model employs two different convolutional neural networks (CNNs) for feature extraction, where the grouping-attention-fusion strategy is used to fuse the features of the CNNs in a fine and multi-scale manner. In this way, the resultant feature representation of the scene is enhanced. Moreover, to address the issue of similar appearances between different scenes, we develop a loss function which encourages small intra-class diversities and large inter-class distances. Extensive experiments are conducted on four scene classification datasets include the UCM land-use dataset, the WHU-RS19 dataset, the AID dataset, and the OPTIMAL-31 dataset. The experimental results demonstrate the superiority of the proposed method in comparison with the state-of-the-arts.

show abstract

Section: The Deep Learning-based Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Dual-Model Architecture with Grouping-Attention-Fusion for Remote Sensing Scene Classification

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Freeway Basketball_Court Golf_Course Ship Thermal_Power_Station Tennis_Court Island In recent years, different strategies of feature-level fusion have already been investigated [16][17][18][19][20], which could produce more discriminative features and better performance than the methods that only utilize deep features. Nevertheless, such feature-level fusion still has shortcomings.…”

Section: Airplanementioning

confidence: 99%

Decision-Level Fusion with a Pluginable Importance Factor Generator for Remote Sensing Image Scene Classification

et al. 2021

View full text Add to dashboard Cite

Remote sensing image scene classification acts as an important task in remote sensing image applications, which benefits from the pleasing performance brought by deep convolution neural networks (CNNs). When applying deep models in this task, the challenges are, on one hand, that the targets with highly different scales may exist in the image simultaneously and the small targets could be lost in the deep feature maps of CNNs; and on the other hand, the remote sensing image data exhibits the properties of high inter-class similarity and high intra-class variance. Both factors could limit the performance of the deep models, which motivates us to develop an adaptive decision-level information fusion framework that can incorporate with any CNN backbones. Specifically, given a CNN backbone that predicts multiple classification scores based on the feature maps of different layers, we develop a pluginable importance factor generator that aims at predicting a factor for each score. The factors measure how confident the scores in different layers are with respect to the final output. Formally, the final score is obtained by a class-wise and weighted summation based on the scores and the corresponding factors. To reduce the co-adaptation effect among the scores of different layers, we propose a stochastic decision-level fusion training strategy that enables each classification score to randomly participate in the decision-level fusion. Experiments on four popular datasets including the UC Merced Land-Use dataset, the RSSCN 7 dataset, the AID dataset, and the NWPU-RESISC 45 dataset demonstrate the superiority of the proposed method over other state-of-the-art methods.

show abstract

“…In [67] convolutional features at multi-levels were integrated to classify electroencephalograph (EEG) images. In [68], a multi-layer convolutional feature fusion network was proposed by combining CNN features to classify high spatial resolution images. To integrate the multi-layer convolutional features, they used the VGG16 [60] model pre-trained on ImageNet [61] data.…”

Section: Feature Fusion In Cnnmentioning

confidence: 99%

Cursive Character Recognition in Natural Scene Images Using a Multilevel Convolutional Neural Network Fusion

2020

View full text Add to dashboard Cite

The accuracy of current natural scene text recognition algorithms is limited by the poor performance of character recognition methods for these images. The complex backgrounds, variations in the writing, text size, orientations, low resolution and multi-language text make recognition of text in natural images a complex and challenging task. Conventional machine learning and deep learning-based methods have been developed that have achieved satisfactory results, but character recognition for cursive text such as Arabic and Urdu scripts in natural images is still an open research problem. The characters in the cursive text are connected and are difficult to segment for recognition. Variations in the shape of a character due to its different positions within a word make the recognition task more challenging than non-cursive text. Optical character recognition (OCR) techniques proposed for Arabic and Urdu scanned documents perform very poorly when applied to character recognition in natural images. In this paper, we propose a multiscale feature aggregation (MSFA) and a multi-level feature fusion (MLFF) network architecture to recognize isolated Urdu characters in natural images. The network first aggregates multi-scale features of the convolutional layers by up-sampling and addition operations and then combines them with the high-level features. Finally, the outputs of the MSFA and MLFF networks are fused together to create more robust and powerful features. A comprehensive dataset of segmented Urdu characters is developed for the evaluation of the proposed network models. Synthetic text on the patches of images with real natural scene backgrounds is generated to increase the samples of infrequently used characters. The proposed model is evaluated on the Chars74K and ICDAR03 datasets. To validate the proposed model on the new Urdu character image dataset, we compare its performance with the histogram of oriented gradients (HoG) method. The experimental results show that the aggregation of multi-scale and multilevel features and their fusion is more effective, and outperforms other methods on the Urdu character image and Chars74K datasets. INDEX TERMS Cursive text recognition, natural scene Urdu character recognition, multi-scale feature aggregation, multi-level feature fusion, convolutional neural network (CNN)

show abstract

Multilayer Feature Fusion Network for Scene Classification in Remote Sensing

Cited by 77 publications

References 32 publications

A Dual-Model Architecture with Grouping-Attention-Fusion for Remote Sensing Scene Classification

A Dual-Model Architecture with Grouping-Attention-Fusion for Remote Sensing Scene Classification

Decision-Level Fusion with a Pluginable Importance Factor Generator for Remote Sensing Image Scene Classification

Cursive Character Recognition in Natural Scene Images Using a Multilevel Convolutional Neural Network Fusion

Contact Info

Product

Resources

About