Deep Semantic Cross Modal Hashing Based on Graph Similarity of Modal-Specific

Abstract

With the advantages of low storage cost and fast query speed, cross modal hashing has attracted increasing attention recently. However, most existing cross-modal hashing methods adopt the same measurement metric when processing data of different modalities or cannot explore heterogeneous correlation across different modalities well, which will result in information loss and heterogeneous correlation cannot be solved. In this paper, we propose a Deep semantic Cross Modal Hashing based on Graph similarity of Modal-Specific (DCMHGMS) method, which not only considers the inter-modal similarity but also designs two graphs to characterize the intra-modal similarity of modal-specific models. First, we use a weighted measurement metric of Euclidean distance and cosine distance to measure the inter-modal similarity between image and text, which can solve the heterogeneous correlation problem. Next, for image graph, we build the intra-modal similarity with Euclidean distance function. Then, for text graph, we build the intra-modal similarity with cosine distance function. Paying attention to the specifics of each modality can improve the retrieval accuracy, thus solving the problem of information loss. Moreover, the semantic information embedding, quantization loss, and bit balanced constraint are considered in this model. Experimental results on two datasets show the effectiveness of our proposed DCMHGMS method.INDEX TERMS Deep semantic embedding, Graph similarity of modal-specific, Cross modal retrieval

Cross-modal retrieval based on multi-dimensional feature fusion hashing

“…Self-supervised adversarial Hashing (SSAH) [35] introduces adversarial loss through the construction of a label network to shorten the distance between image and text distribution, which brings a better retrieval effect. Using cosine distance and Euclidean distance, the same measurement index can accurately reflect the similarity between different modal data in Deep Semantic Cross-Modal Hashing Based on Graph Similarity of Modal-Specific (DCMHGMS) [36]. The distance between similar data can be reduced by constructing ranking alignment loss to unearth the semantic structure between different modal data in Deep Rank Cross-modal Hashing (DRCH) [37,38].…”