An Efficient Hyperspectral Image Retrieval Method: Deep Spectral-Spatial Feature Extraction with DCGAN and Dimensionality Reduction Using t-SNE-Based NM Hashing

Abstract

Abstract:Hyperspectral images are one of the most important fundamental and strategic information resources, imaging the same ground object with hundreds of spectral bands varying from the ultraviolet to the microwave. With the emergence of huge volumes of high-resolution hyperspectral images produced by all sorts of imaging sensors, processing and analysis of these images requires effective retrieval techniques. How to ensure retrieval accuracy and efficiency is a challenging task in the field of hyperspectral image retrieval. In this paper, an efficient hyperspectral image retrieval method is proposed. In principle, our method includes the following steps: (1) in order to make powerful representations for hyperspectral images, deep spectral-spatial features are extracted with the Deep Convolutional Generative Adversarial Networks (DCGAN) model; (2) considering the higher dimensionality of deep spectral-spatial features, t-Distributed Stochastic Neighbor Embedding-based Nonlinear Manifold (t-SNE-based NM) hashing is utilized to make dimensionality reduction by learning compact binary codes embedded on the intrinsic manifolds of deep spectral-spatial features for balancing between learning efficiency and retrieval accuracy; and (3) multi-index hashing in Hamming space is measured to find similar hyperspectral images. Five comparative experiments are conducted to verify the effectiveness of deep spectral-spatial features, dimensionality reduction of t-SNE-based NM hashing, and similarity measurement of multi-index hashing. The experimental results using NASA datasets show that our hyperspectral image retrieval method can achieve comparable and superior performance with less computational time.

Low Dimensional Discriminative Representation of Fully Connected Layer Features Using Extended LargeVis Method for High-Resolution Remote Sensing Image Retrieval

“…Among them, SNE, t -SNE and LargeVis are designed for the visualization of large-scale high-dimensional data. Zhang et al [ 10 ] utilized a t -SNE-based nonlinear manifold hashing algorithm to make dimensionality reduction by learning compact binary codes embedded on the intrinsic manifolds of deep spectral-spatial features for balancing between learning efficiency and retrieval accuracy.…”

A Survey on Deep Learning-Driven Remote Sensing Image Scene Understanding: Scene Classification, Scene Retrieval and Scene-Guided Object Detection

“…The approach has surprisingly better retrieval performance in that the mean average precision (MAP) values exceed those of KSH by about 45%, which can train all of the data that is available. As Zhang et al [79] brought the hashing method to hyperspectral RS scenes, deep features which are spectral-spatial and produced by the Deep Convolutional Generative Adversarial Networks (DCGAN) model were then Nonlinear Manifold [80] (NM) hashed to reduce their dimensionality. Multi-index hashing was also employed to search hyperspectral images.…”

Generative Adversarial Network for Medical Images (MI-GAN)

“…The generator achieves this by minimizing equation 1, which is our objective function. Practically, by using the approximation scheme as in [29], this can be done by minimizing −log (D γ (G θ (y, z))), which is a simpler form than original log (1 − D γ (G θ (y, z))). Overall generator loss can be defined as:…”