Bit-aware Semantic Transformer Hashing for Multi-modal Retrieval

Tan, Wentao; Zhu, Lei; Guan, Weili; Li, Jingjing; Cheng, Zhiyong

doi:10.1145/3477495.3531947

Cited by 14 publications

(1 citation statement)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, due to the massive growth of multimedia data (Qin et al 2023b;Qin, Pu, and Wu 2023;He et al 2022), continuous-value methods suffer from high storage costs and computation time. To solve this issue, cross-modal hashing (CMH) methods (Tan et al 2022;Yang et al 2023b) have been proposed to achieve efficient performance. The essential key of CMH is to map multimodal data into discriminative binary codes while eliminating the cross-modal gap.…”

Section: Introductionmentioning

confidence: 99%

Dual Self-Paced Cross-Modal Hashing

Sun,

Dai,

Ren

et al. 2024

AAAI

View full text Add to dashboard Cite

Cross-modal hashing~(CMH) is an efficient technique to retrieve relevant data across different modalities, such as images, texts, and videos, which has attracted more and more attention due to its low storage cost and fast query speed. Although existing CMH methods achieve remarkable processes, almost all of them treat all samples of varying difficulty levels without discrimination, thus leaving them vulnerable to noise or outliers. Based on this observation, we reveal and study dual difficulty levels implied in cross-modal hashing learning, \ie instance-level and feature-level difficulty. To address this problem, we propose a novel Dual Self-Paced Cross-Modal Hashing (DSCMH) that mimics human cognitive learning to learn hashing from ``easy'' to ``hard'' in both instance and feature levels, thereby embracing robustness against noise/outliers. Specifically, our DSCMH assigns weights to each instance and feature to measure their difficulty or reliability, and then uses these weights to automatically filter out the noisy and irrelevant data points in the original space. By gradually increasing the weights during training, our method can focus on more instances and features from ``easy'' to ``hard'' in training, thus mitigating the adverse effects of noise or outliers. Extensive experiments are conducted on three widely-used benchmark datasets to demonstrate the effectiveness and robustness of the proposed DSCMH over 12 state-of-the-art CMH methods.

show abstract