Index and Retrieve Multimedia Data: Cross-Modal Hashing by Learning Subspace Relation

Liu, Luchen; Yang, Yang; Hu, Mengqiu; Xu, Xing; Shen, Fumin; Xie, Ning; Huang, Zi

doi:10.1007/978-3-319-91458-9_37

Cited by 9 publications

(7 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our proposed DCMHGMS method is compared with eight state-of-the-art cross-modal hashing methods: SePH [10], DCH [11], FSH [7], CCQ [4], SRLCH [17], SCRATCH [36], SMFH [19], DCMH [23]. FSH is an unsupervised method, and other seven methods are supervised ones.…”

Section: B Baseline Methodsmentioning

confidence: 99%

“…Semantics-Preserving Hashing (SePH) [10] transforms semantic affinities of training data into a probability distribution and approximates it with tobe-learnt hash codes in hamming space via minimizing the Kullback-Leibler divergence. Subspace Relation Learning for Cross modal Hashing (SRLCH) [17] method exploits correlation information with semantic label and preserves the nonlinear structure. Supervised Matrix Factorization Hashing for Cross-Modal Retrieval (SMFH) [19] is a supervised cross modal hashing method based on collective matrix factorization, which considers both the label consistency across different modalities and the local geometric consistency in each modality.Fast Discrete Cross-Modal Hashing (FDCH) [15] is a method with regressing from semantic labels to take advantage of supervised labels to improve retrieval performance, which enhances the discriminative capability of hash codes.…”

Section: Related Workmentioning

confidence: 99%

“…Cross modal hashing methods [4]- [17], [19], [22]- [24], [36], [37], [41] project high dimensional instances to a common hamming space, where the heterogeneous data can get a unified representation and be measured uniformly. Actually, the heterogeneous images and texts have different specifics, that is different feature representation ways, one of them adopts the rotation invariant feature, the other adopts the word bag feature.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep Semantic Cross Modal Hashing Based on Graph Similarity of Modal-Specific

2021

IEEE Access

View full text Add to dashboard Cite

With the advantages of low storage cost and fast query speed, cross modal hashing has attracted increasing attention recently. However, most existing cross-modal hashing methods adopt the same measurement metric when processing data of different modalities or cannot explore heterogeneous correlation across different modalities well, which will result in information loss and heterogeneous correlation cannot be solved. In this paper, we propose a Deep semantic Cross Modal Hashing based on Graph similarity of Modal-Specific (DCMHGMS) method, which not only considers the inter-modal similarity but also designs two graphs to characterize the intra-modal similarity of modal-specific models. First, we use a weighted measurement metric of Euclidean distance and cosine distance to measure the inter-modal similarity between image and text, which can solve the heterogeneous correlation problem. Next, for image graph, we build the intra-modal similarity with Euclidean distance function. Then, for text graph, we build the intra-modal similarity with cosine distance function. Paying attention to the specifics of each modality can improve the retrieval accuracy, thus solving the problem of information loss. Moreover, the semantic information embedding, quantization loss, and bit balanced constraint are considered in this model. Experimental results on two datasets show the effectiveness of our proposed DCMHGMS method.INDEX TERMS Deep semantic embedding, Graph similarity of modal-specific, Cross modal retrieval

show abstract

Section: B Baseline Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Semantic Cross Modal Hashing Based on Graph Similarity of Modal-Specific

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…However, one common limitation of them is that they relax the discrete constraints within the optimization resulting in suboptimal binary codes. Therefore, a number of cross-modal hashing methods based on discrete optimization are studied [7], [8], [21]. In [22], Locally Linear Embedding is used to extract the manifold information as similarity matrix for learning unified hash codes where the binary codes are learned directly without relaxation.…”

Section: Related Workmentioning

confidence: 99%

“…It is worth noting that the above methods generate binary codes by relaxing the discrete constraints, which leads to a large quantization error. To address this issue, many studies propose to learn binary codes with discrete optimization [7], [20], [21]. For instance, Discrete cross-modal hashing (DCH) [5] was proposed where binary codes are directly learned without relaxation, and label information is used to enhance the discrimination of binary codes through linear classifiers.…”

Section: Introductionmentioning

confidence: 99%

MESH: A Flexible Manifold-Embedded Semantic Hashing for Cross-Modal Retrieval

et al. 2020

View full text Add to dashboard Cite

Hashing based methods for cross-modal retrieval has been widely explored in recent years. However, most of them mainly focus on the preservation of neighborhood relationship and label consistency, while ignore the proximity of neighbors and proximity of classes, which degrades the discrimination of hash codes. And most of them learn hash codes and hashing functions simultaneously, which limits the flexibility of algorithms. To address these issues, in this paper, we propose a two-step cross-modal retrieval method named Manifold-Embedded Semantic Hashing (MESH). It exploits Local Linear Embedding to model the neighborhood proximity and uses class semantic embeddings to consider the proximity of classes. By so doing, MESH can not only extract the manifold structure in different modalities, but also can embed the class semantic information into hash codes to further improve the discrimination of learned hash codes. Moreover, the two-step scheme makes MESH flexible to various hashing functions. Extensive experimental results on three datasets show that MESH is superior to 10 state-of-the-art cross-modal hashing methods. Moreover, MESH also demonstrates superiority on deep features compared with the deep cross-modal hashing method.

show abstract

Semantic embedding based online cross-modal hashing method

Zhang,

Li,

Zheng

2024

Sci Rep

View full text Add to dashboard Cite

Hashing has been extensively utilized in cross-modal retrieval due to its high efficiency in handling large-scale, high-dimensional data. However, most existing cross-modal hashing methods operate as offline learning models, which learn hash codes in a batch-based manner and prove to be inefficient for streaming data. Recently, several online cross-modal hashing methods have been proposed to address the streaming data scenario. Nevertheless, these methods fail to fully leverage the semantic information and accurately optimize hashing in a discrete fashion. As a result, both the accuracy and efficiency of online cross-modal hashing methods are not ideal. To address these issues, this paper introduces the Semantic Embedding-based Online Cross-modal Hashing (SEOCH) method, which integrates semantic information exploitation and online learning into a unified framework. To exploit the semantic information, we map the semantic labels to a latent semantic space and construct a semantic similarity matrix to preserve the similarity between new data and existing data in the Hamming space. Moreover, we employ a discrete optimization strategy to enhance the efficiency of cross-modal retrieval for online hashing. Through extensive experiments on two publicly available multi-label datasets, we demonstrate the superiority of the SEOCH method.

show abstract

Index and Retrieve Multimedia Data: Cross-Modal Hashing by Learning Subspace Relation

Cited by 9 publications

References 28 publications

Deep Semantic Cross Modal Hashing Based on Graph Similarity of Modal-Specific

Deep Semantic Cross Modal Hashing Based on Graph Similarity of Modal-Specific

MESH: A Flexible Manifold-Embedded Semantic Hashing for Cross-Modal Retrieval

Semantic embedding based online cross-modal hashing method

Contact Info

Product

Resources

About