Attention Multisource Fusion-Based Deep Few-Shot Learning for Hyperspectral Image Classification

Abstract

Recently, deep learning-based methods outperform others in hyperspectral image (HSI) classification. However, the deep learning methods require sufficient labeled samples to improve performance, which is unfeasible in practice. The training labels are usually limited in HSIs that need to be classified (namely target domain), while other available labels in multisource HSIs (namely source domain) are not utilized effectively. To mitigate these issues, an attention multisource fusion method of few-shot learning (AMF-FSL) is proposed for small-sized HSI classification. AMF-FSL is an implementation of few-shot learning (FSL) in the meta-learning field, which can transfer the learned ability of classification from multiple source data to target data. The process of learning to classify in AMF-FSL is not restricted by the traditional requirement of the same distribution between the source and target domains, which can learn from the source domain and apply it to a different distribution in the target domain. Moreover, the multisource domain adaption in AMF-FSL has the capacity of extracting features from fused homogeneous and heterogeneous data in the source domain, which can improve the generalization of the classification model in the cross domains. Specifically, the multisource domain adaption contains three modules, namely the target-based class alignment, domain attention assignment, and multisource data fusion, which are responsible for aligning the class space, paying band-level attention, and merging the distributions of homogeneous and heterogeneous data in the source domain. The experimental results demonstrate the effectiveness of the multisource domain adaption and the superiority of AMF-FSL over other state-of-theart methods in small-sized HSI classification.

Domain Adaptation in Remote Sensing Image Classification: A Survey

“…multiple-kernel learning [75]- [78], ELM-based [79]- [83], MDAF and MBCF [84], open set DA [85], EasyTL [86], BHC [87], DASVM [88], MRC [89], AL-based [90]- [97] Deep DA Discrepancy-based DAN [98], JAN [99], MRAN [100], DSAN [101], DeepCORAL [102], DNN with class centroid alignment [103], TCANet [104], class-wise distribution alignment based deep DA [105], DDA-Net [106], TDDA [107], TSTnet [108], GNN [109], AMF-FSL [110], MSCN [111], AMRAN [112], DJDANs [113] Adversarial-based GAN [114], [115], adversarial CNN [116], MADA [117], DAAN [118], MCD [119], DWL [120], GAN with VAE-based generator [121], [122], content-wise alignment [123], class reconstruction driven adversarial [124], class-wise adversarial [125], ADADL [126], DABAN [127], UDAD [123], deep metric learning [128], DCFSL …”

“…Wang et al proposed a graph neural network (GNN) DA method for multitemporal HSIs [109], which incorporated the domain-wise and class-wise CORAL into the GNN network to align the joint distributions of domains. Liang et al proposed an attention multisource fusion-based deep few-shot learning (AMF-FSL) method for small-sized HSI classification [110], which contains three modules, namely the target-based class alignment, domain attention assignment, and multisource data fusion. It can transfer the learned ability of classification from multiple source data to target data.…”

Multi-DS Strategy for Source Camera Identification in Few-Shot Sample Data Sets

“…Tan et al [17] constructed multiple prototype sets using the ensemble projection (EP) method and used richer features to solve the problem of insufficient known labels in the few-shot sample problem. Liang et al [18] used the attention multisource fusion few-shot learning method (AMF-FSL) to transfer the classification ability of few-shot learning from multisource data to target data, which improved the generalization ability of the classification model in cross-domain. Sameer and Naskar [19] used the deep Siamese network method to enhance the training space by forming paired samples from the same camera model and different camera models and obtained a better model of camera source identification problem.…”

Spectral-Spatial Domain Attention Network for Hyperspectral Image Few-Shot Classification

“…Few-shot learning (FSL) is an important implementation method of meta-learning [39][40][41], which can transfer the extracted classification knowledge from source domain to target domain. In recent years, a growing number of FSL methods have been proposed for HSI classification [42][43][44][45]. Bing Liu et al [42] proposed a deep FSL method to solve the small sample size problem of HSI classification via learning the metric space from the training set.…”