MIMA: MAPPER-Induced Manifold Alignment for Semi-Supervised Fusion of Optical Image and Polarimetric SAR Data

Abstract

This is the pre-acceptance version, to read the final version please go to IEEE Transactions on Geoscience and Remote Sensing on IEEE Xplore. Multi-modal data fusion has recently been shown promise in classification tasks in remote sensing. Optical data and radar data, two important yet intrinsically different data sources, are attracting more and more attention for potential data fusion. It is already widely known that, a machine learning based methodology often yields excellent performance. However, the methodology relies on a large training set, which is very expensive to achieve in remote sensing. The semi-supervised manifold alignment (SSMA), a multi-modal data fusion algorithm, has been designed to amplify the impact of an existing training set by linking labeled data to unlabeled data via unsupervised techniques. In this paper, we explore the potential of SSMA in fusing optical data and polarimetric SAR data, which are multi-sensory data sources. Furthermore, we propose a MAPPER-induced manifold alignment (MIMA) for semisupervised fusion of multi-sensory data sources. Our proposed method unites SSMA with MAPPER, which is developed from the emerging topological data analysis (TDA) field. To our best knowledge, this is the first time that SSMA has been applied on fusing optical data and SAR data, and also the first time that TDA has been applied in remote sensing. The conventional SSMA derives a topological structure using k-nearest-neighbor (kNN), while MIMA employs MAPPER, which considers the field knowledge and derives a novel topological structure through the spectral clustering in a data-driven fashion. Experiment results on data fusion with respect to land cover land use classification and local climate zone classification suggest superior performance of MIMA.Index Terms-Hyperspectral image, MAPPER, multi-modal data fusion, multi-sensory data fusion, multispectral image, Pol-SAR, semi-supervised manifold alignment (SSMA), topological data analysis (TDA).

A Comparative Review of Manifold Learning Techniques for Hyperspectral and Polarimetric SAR Image Fusion

“…Meanwhile, the theoretical impact of different learning strategies to the fusion result is discussed. Then, the following up sub-sections recall the principles of the four selected state-of-art manifold fusion techniques, namely LPP [52], GGF [48], MA [36,44], and MIMA [53]. Pseudo-codes of these four algorithms are listed in the Appendixes A-D, which provides the technical details.Finally, the data sets and the experiment settings are introduced in detail.…”

“…Those manifolds can be aligned in a latent space. Representative algorithms are the manifold alignment (MA) [36] and the MAPPER-Induced manifold alignment (MIMA) [53]. The other essential research question of manifold-based fusion is: how should the manifold be extracted?…”

“…This paper investigates the performance of manifold learning technique on the fusion of hyperspectral and PolSAR data, based on four state-of-art algorithms, locality preservation projection (LPP) [52], generalized graph fusion (GGF) [48], manifold alignment (MA) [36,44], and MAPPER-induced manifold alignment (MIMA) [53]. We implemented 16 variants of the four algorithms which involve the above-mentioned two alignment approaches and the three manifold learning strategies.…”

“…MIMA is designed to fuse optical and PolSAR data for the purpose of LULC classification [53]. It follows the framework of MA [36,44] yet introduces a novel constraint term which originates from a recent field of topological data analysis (TDA).…”

A Robust Dynamic Classifier Selection Approach for Hyperspectral Images with Imprecise Label Information

“…It is generally agreed upon that incorporating spatial context together with spectral information leads to better classification results than using spectral information alone [ 16 , 17 , 18 , 19 , 20 ]. Further improvements in the classification accuracy can be obtained by combining multiple data sources, e.g., by augmenting HSI data with Light Detection and Ranging (LiDAR) data [ 21 , 22 , 23 ], Synthetic Aperture Radar (SAR) data [ 24 , 25 ] and/or high-resolution colour images [ 26 , 27 , 28 ]. Fusion of these multiple data sources is typically accomplished at feature level [ 29 , 30 , 31 ], or at decision level [ 26 , 32 , 33 ].…”