Deep Semantic-Alignment Hashing for Unsupervised Cross-Modal Retrieval

Yang, Dejie; Wu, Dayan; Zhang, Wanqian; Zhang, Haisu; Li, Bo; Wang, Weiping

doi:10.1145/3372278.3390673

Cited by 65 publications

(39 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some representative methods were selected for comparison to verify the effectiveness of the proposed KDCMH method, including nine unsupervised methods and two supervised methods. CMSSH [1] and SCM [30] are supervised methods, while CVH [9], PDH [23], CMFH [24],CCQ [25], UGACH [19], DJSRH [34], UKD-SS [38], JDSH [17] and DSAH [35] are unsupervised methods. In the original work, JDSH and D-JSRH used Recall=50 when calculating the MAP, whereas the other methods returned all the query results.…”

Section: Experiments Resultsmentioning

confidence: 99%

Unsupervised Deep Cross-Modal Hashing by Knowledge Distillation for Large-scale Cross-modal Retrieval

Wang

2021

Proceedings of the 2021 International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

Cross-modal hashing (CMH) maps heterogeneous multiple modality data into compact binary code to achieve fast and flexible retrieval across different modalities, especially in large-scale retrieval. As the data don't need a lot of manual annotation, unsupervised cross-modal hashing has a wider application prospect than supervised method. However, the existing unsupervised methods are difficult to achieve satisfactory performance due to the lack of credible supervisory information. To solve this problem, inspired by knowledge distillation, we propose a novel unsupervised Knowledge Distillation Cross-Modal Hashing method (KDCMH), which can use similarity information distilled from unsupervised method to guide supervised method. Specifically, firstly, the teacher model adopted an unsupervised distribution-based similarity hashing method, which can construct a modal fusion similarity matrix.Secondly, under the supervision of teacher model distillation information, student model can generate more discriminative hash codes. In two public datasets NUS-WIDE and MIRFLICKR-25K, extensive experiments have proved the significant improvement of KDCMH on several representative unsupervised cross-modal hashing methods. CCS CONCEPTS• Information systems → Multimedia and multimodal retrieval; • Computing methodologies → Neural networks.

show abstract

Section: Experiments Resultsmentioning

confidence: 99%

Unsupervised Deep Cross-Modal Hashing by Knowledge Distillation for Large-scale Cross-modal Retrieval

Wang

2021

Proceedings of the 2021 International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

show abstract

“…Unsupervised Learning. Compared with supervised methods, unsupervised hashing techniques [2,5,15,18,38,43] have more extensive practical application since they do not rely on annotation training samples. Instead, unsupervised methods model the latent semantic correlations among different modalities to generate hash codes.…”

Section: Related Work 21 Cross-modal Hashingmentioning

confidence: 99%

“…As stated in existing literatures [4,28,35,40], the major roadblocks of cross-modal retrieval are two folds, namely (1) cross-modal heterogeneity that hinders the similarity measurement between samples from different modalities, and (2) semantic gap existing between multi-modal data and human understanding, which obstructs the cross-modal semantic alignment. According to the type of representation, cross-modal retrieval techniques can be grouped into two categories, i.e., the real value-based methods [38,48] and binary value-based (hashing) approaches [15,18,43]. As the higher efficiency of retrieval and lower cost of storage, more attention has been paid to cross-modal hashing for big data applications.…”

Section: Introductionmentioning

confidence: 99%

“…However, in order to ensure effective model training, the supervised methods have to require a large amount of labeled data, which cannot easily obtained from practical applications. To break this limitation, another type of learning technique, namely unsupervised cross-modal hashing methods [2,5,15,18,24,38,43] have been studied by more and more scholars, which is aiming to mine out latent semantic correlation between different modalities from large number of unlabeled samples. Due to this advantage, unsupervised hashing method can be easily extended to real-world scenarios.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

M2guda

Zhang

Zhi

Zhu

et al. 2021

Proceedings of the 2021 International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

Cross-modal hashing is a critical but very challenging task that is to retrieve similar samples of one modality via queries of other modalities. To improve the unsupervised cross-modal hashing, domain adaptation techniques can be used to support unsupervised hashing learning by transferring semantic knowledge from labeled source domain to unlabeled target domain. However, there are two problems that cannot be ignored: (1) most of domain adaptation based researches mainly focused on unimodal hashing or cross-modal real value-based retrieval but the study for cross-modal hashing is limited; (2) most existing studies only consider one or two consistency constraints during the domain adaptation learning. To this end, this paper propose a novel end-to-end framework to realize unsupervised domain adaptation for cross-modal hashing. This method, dubbed M 2 GUDA, including four different consistency constraints: structure consistency, domain consistency, semantic consistency and modality consistency for domain adaptation learning. Besides, to enhance the structure consistency learning, we develop a multimetrics graph modeling method to capture structure information comprehensively. Extensive experiments are performed on three common used benchmarks to evaluate the effectivity of our method. The results show that our method outperforms several state-of-theart cross-modal hashing methods. CCS CONCEPTS• Information systems → Multimedia and multimodal retrieval.

show abstract

“…Recently, with the great success of deep learning in the field of representation learning, many deep hashing methods [13,15,40,43] have been proposed. For example, Deep Semantic-Alignment Hashing (DSAH) [44] is an unsupervised hashing method, which explores the similarity information of different modalities and proposes a semantic-alignment loss to learn the hash codes. Unsupervised Deep Cross-modal Hashing with Virtual Label Regression (UDCH-VLR) [45] proposes a novel unified learning framework to jointly perform deep hash function training, virtual label learning, and regression.…”

Section: Deep Hashingmentioning

confidence: 99%

Discrete Semantics-Guided Asymmetric Hashing for Large-Scale Multimedia Retrieval

et al. 2021

View full text Add to dashboard Cite

Cross-modal hashing technology is a key technology for real-time retrieval of large-scale multimedia data in real-world applications. Although the existing cross-modal hashing methods have achieved impressive accomplishment, there are still some limitations: (1) some cross-modal hashing methods do not make full consider the rich semantic information and noise information in labels, resulting in a large semantic gap, and (2) some cross-modal hashing methods adopt the relaxation-based or discrete cyclic coordinate descent algorithm to solve the discrete constraint problem, resulting in a large quantization error or time consumption. Therefore, in order to solve these limitations, in this paper, we propose a novel method, named Discrete Semantics-Guided Asymmetric Hashing (DSAH). Specifically, our proposed DSAH leverages both label information and similarity matrix to enhance the semantic information of the learned hash codes, and the ℓ2,1 norm is used to increase the sparsity of matrix to solve the problem of the inevitable noise and subjective factors in labels. Meanwhile, an asymmetric hash learning scheme is proposed to efficiently perform hash learning. In addition, a discrete optimization algorithm is proposed to fast solve the hash code directly and discretely. During the optimization process, the hash code learning and the hash function learning interact, i.e., the learned hash codes can guide the learning process of the hash function and the hash function can also guide the hash code generation simultaneously. Extensive experiments performed on two benchmark datasets highlight the superiority of DSAH over several state-of-the-art methods.

show abstract

Deep Semantic-Alignment Hashing for Unsupervised Cross-Modal Retrieval

Cited by 65 publications

References 54 publications

Unsupervised Deep Cross-Modal Hashing by Knowledge Distillation for Large-scale Cross-modal Retrieval

Unsupervised Deep Cross-Modal Hashing by Knowledge Distillation for Large-scale Cross-modal Retrieval

M2guda

Discrete Semantics-Guided Asymmetric Hashing for Large-Scale Multimedia Retrieval

Contact Info

Product

Resources

About