Joint Correlation Alignment-Based Graph Neural Network for Domain Adaptation of Multitemporal Hyperspectral Remote Sensing Images

Abstract

In this paper, we propose a novel deep domain adaptation method based on graph neural network (GNN) for multitemporal hyperspectral remote sensing images. In GNN, graphs are constructed for source and target data, respectively. Then the graphs are utilized in each hidden layer to obtain features. GNN operates on graph structure and the relations between data samples can be exploited. It aggregates features and propagate information through graph nodes. Thus, the extracted features have an improved smoothness in each spectral neighborhood which is beneficial to classification. Furthermore, the domain-wise correlation alignment and classwise correlation alignment are jointly embedded in GNN network to achieve a joint distribution adaptation performance. By introducing the joint correlation alignment strategy in GNN, the extracted features can not only be aligned between domains but also have a superior discriminability in each domain. This domain adaptation network is named as joint correlation alignment based graph neural network (JCGNN). Experiments using multitemporal Hyperion and NCALM datasets demonstrate the effectiveness of the proposed method. Index Terms-Hyperspectral remote sensing, graph neural network, domain adaptation, classification Recently, deep learning has been applied to domain adaptation because of its excellent feature representation ability. Compared with shallow domain adaptation approaches, end-toend deep domain adaptation approaches transfer more effective knowledge by embedding adaptation modules in the network architecture. Maximum Mean Discrepancy (MMD) is a popular distribution alignment strategy, which is utilized in many deep domain adaptation methods. Tzeng et al. [17] proposed the deep domain confusion method, which firstly regularizes the single adaptation layer of deep neural network using linear-kernel MMD. Similar to MMD, correlation alignment could also measure the distribution distance between domains, and it attempts to align the second-order statistics of the source and target features. Sun et al. [18] extended the correlation alignment to deep architectures and proposed the correlation alignment for deep domain adaptation (D-CORAL) approach, so that a nonlinear transformation that aligns the correlations of features between the two domains is obtained. Besides,

On the Optimization of Regression-Based Spectral Reconstruction

“…Using a hyperspectral camera [ 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 ], we can record scene radiance at high spectral and spatial resolution. This technique has been widely used in machine vision applications such as remote sensing [ 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 ], medical imaging [ 28 , 29 , 30 , 31 ], food processing [ 32 , 33 , 34 , 35 , 36 , 37 ], and anomaly detection [ 38 , 39 , 40 , 41 , 42 , 43 , 44 ], as well as in the spectral characterization domain, including the calibration of color devices (e.g., cameras [ 45 ] and scanners [ 46 ]), scene relighting [ 47 , 48 ], and art conservation and archiving [ 49 , 50 , 51 ]. While useful, hyperspectral cameras are usually much more expensive than the RGB cameras.…”

Deep learning for processing and analysis of remote sensing big data: a technical review

“…Recent pixelwise LULC classification using semantic segmentation networks has achieved great advances (Tong et al, 2020;Zhang et al, 2020aZhang et al, , 2019a. In particular, to further explore spatial and graph topological information, GNNs have been employed in LULC classification tasks on hyperspectral images (Mou et al, 2020;Qin et al, 2018;Wan, Gong, & Zhong et al, 2019;Wan et al, 2020;Wang, Ma, Chen, & Du, 2021), very high-resolution satellite images (Cui et al, 2021;Khan, Chaudhuri, Banerjee, & Chaudhuri, 2019;Liu, Kampffmeyer, & Jenssen et al, 2020b;Ouyang & Li, 2021), and time-series images (Censi et al, 2021).…”

Edge-Inferring Graph Neural Network With Dynamic Task-Guided Self-Diagnosis for Few-Shot Hyperspectral Image Classification

“…In [33], GNN was first presented to build neural networks on graphs. After that, GNN-based models [34], [35], [36] have been increasingly popular in various areas and showed great potential ability. In HSIC, many methods applied GNN to achieve promising performance.…”