Multi-Label Remote Sensing Image Scene Classification by Combining a Convolutional Neural Network and a Graph Neural Network

Li, Yansheng; Chen, Ruixian; Zhang, Yongjun; Zhang, Mi; Chen, Ling

doi:10.3390/rs12234003

Cited by 64 publications

(43 citation statements)

References 57 publications

(67 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another direction is to embed the knowledge into the classifier. Some researchers have already used a GNN (Graph Neural Network) model to classify remote sensing imagery [ 39 , 40 ]. These studies have proved that the spatial relationship introduced by GCN boosts the performance and robustness of the classification model.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Remote Sensing Image Classification with a Graph-Based Pre-Trained Neighborhood Spatial Relationship

Guan

Huang

Yang

et al. 2021

Sensors

View full text Add to dashboard Cite

Previous knowledge of the possible spatial relationships between land cover types is one factor that makes remote sensing image classification “smarter”. In recent years, knowledge graphs, which are based on a graph data structure, have been studied in the community of remote sensing for their ability to build extensible relationships between geographic entities. This paper implements a classification scheme considering the neighborhood relationship of land cover by extracting information from a graph. First, a graph representing the spatial relationships of land cover types was built based on an existing land cover map. Empirical probability distributions of the spatial relationships were then extracted using this graph. Second, an image was classified based on an object-based fuzzy classifier. Finally, the membership of objects and the attributes of their neighborhood objects were joined to decide the final classes. Two experiments were implemented. Overall accuracy of the two experiments increased by 5.2% and 0.6%, showing that this method has the ability to correct misclassified patches using the spatial relationship between geo-entities. However, two issues must be considered when applying spatial relationships to image classification. The first is the “siphonic effect” produced by neighborhood patches. Second, the use of global spatial relationships derived from a pre-trained graph loses local spatial relationship in-formation to some degree.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Li et al proposed a scene classification scheme that first extracted scene features then segmented the feature map to construct a graph. Finally, the graph was classified by a graph attention network (GAT) [ 40 ]. Ma et al proposed a sum of minimum distance parameter to determine graph adjacency relationships.…”

Section: Introductionmentioning

confidence: 99%

Remote Sensing Image Classification with a Graph-Based Pre-Trained Neighborhood Spatial Relationship

Guan

Huang

Yang

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…In the research of remote sensing image analysis, as hyperspectral images usually have large homogeneous regions, some research started to use graph neural networks (GNNs) to achieve remote sensing images analysis [45][46][47][48]. For example, Reference [45] investigated the use of graph convolutional networks (GCNs) in order to characterize spatial arrangement features for land use classification and [46] proposed a novel deep learning-based MLRSSC framework by combining graph neural network (GNN) and convolutional neural network (CNN) to mine the spatio-topological relationships of the scene graph. To further improve the detection accuracy, Reference [49] proposed a novel anomaly detection method based on texture feature extraction and a graph dictionary-based low rank decomposition (LRD).…”

Section: Remote Sensing With Gnnmentioning

confidence: 99%

Knowledge and Spatial Pyramid Distance-Based Gated Graph Attention Network for Remote Sensing Semantic Segmentation

Cui

Meng

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

The pixel-based semantic segmentation methods take pixels as recognitions units, and are restricted by the limited range of receptive fields, so they cannot carry richer and higher-level semantics. These reduce the accuracy of remote sensing (RS) semantic segmentation to a certain extent. Comparing with the pixel-based methods, the graph neural networks (GNNs) usually use objects as input nodes, so they not only have relatively small computational complexity, but also can carry richer semantic information. However, the traditional GNNs are more rely on the context information of the individual samples and lack geographic prior knowledge that reflects the overall situation of the research area. Therefore, these methods may be disturbed by the confusion of “different objects with the same spectrum” or “violating the first law of geography” in some areas. To address the above problems, we propose a remote sensing semantic segmentation model called knowledge and spatial pyramid distance-based gated graph attention network (KSPGAT), which is based on prior knowledge, spatial pyramid distance and a graph attention network (GAT) with gating mechanism. The model first uses superpixels (geographical objects) to form the nodes of a graph neural network and then uses a novel spatial pyramid distance recognition algorithm to recognize the spatial relationships. Finally, based on the integration of feature similarity and the spatial relationships of geographic objects, a multi-source attention mechanism and gating mechanism are designed to control the process of node aggregation, as a result, the high-level semantics, spatial relationships and prior knowledge can be introduced into a remote sensing semantic segmentation network. The experimental results show that our model improves the overall accuracy by 4.43% compared with the U-Net Network, and 3.80% compared with the baseline GAT network.

show abstract

“…Some representative deep learning techniques include convolutional neural network (CNN) [9,10], recurrent neural network (RNN) [11,12], generative adversarial network [13,14] and convolutional auto-encoder [15,16]. Recently, a series of deep learning-based classification frameworks have been widely used in the field of remote sensing [17][18][19][20][21][22]. Combining the deep CNN with multiple feature learning, a joint feature map for HSI classification was generated, which makes the developed method have high classification performance on test datasets [18].…”

Section: Introductionmentioning

confidence: 99%

Hyperspectral Image Classification Based on Superpixel Pooling Convolutional Neural Network with Transfer Learning

et al. 2021

View full text Add to dashboard Cite

Deep learning-based hyperspectral image (HSI) classification has attracted more and more attention because of its excellent classification ability. Generally, the outstanding performance of these methods mainly depends on a large number of labeled samples. Therefore, it still remains an ongoing challenge how to integrate spatial structure information into these frameworks to classify the HSI with limited training samples. In this study, an effective spectral-spatial HSI classification scheme is proposed based on superpixel pooling convolutional neural network with transfer learning (SP-CNN). The suggested method includes three stages. The first part consists of convolution and pooling operation, which is a down-sampling process to extract the main spectral features of an HSI. The second part is composed of up-sampling and superpixel (homogeneous regions with adaptive shape and size) pooling to explore the spatial structure information of an HSI. Finally, the hyperspectral data with each superpixel as a basic input rather than a pixel are fed to fully connected neural network. In this method, the spectral and spatial information is effectively fused by using superpixel pooling technique. The use of popular transfer learning technology in the proposed classification framework significantly improves the training efficiency of SP-CNN. To evaluate the effectiveness of the SP-CNN, extensive experiments were conducted on three common real HSI datasets acquired from different sensors. With 30 labeled pixels per class, the overall classification accuracy provided by this method on three benchmarks all exceeded 93%, which was at least 4.55% higher than that of several state-of-the-art approaches. Experimental and comparative results prove that the proposed algorithm can effectively classify the HSI with limited training labels.

show abstract

Multi-Label Remote Sensing Image Scene Classification by Combining a Convolutional Neural Network and a Graph Neural Network

Cited by 64 publications

References 57 publications

Remote Sensing Image Classification with a Graph-Based Pre-Trained Neighborhood Spatial Relationship

Remote Sensing Image Classification with a Graph-Based Pre-Trained Neighborhood Spatial Relationship

Knowledge and Spatial Pyramid Distance-Based Gated Graph Attention Network for Remote Sensing Semantic Segmentation

Hyperspectral Image Classification Based on Superpixel Pooling Convolutional Neural Network with Transfer Learning

Contact Info

Product

Resources

About