Label Embedding Online Hashing for Cross-Modal Retrieval

Abstract

Supervised cross-modal hashing has gained a lot of attention recently. However, most existing methods learn binary codes or hash functions in a batch-based scheme, which is inefficient in an online scenario, i.e., data points come in a streaming fashion. Online hashing is a promising solution; however, there still exist several challenges, e.g., how to effectively exploit semantic information, how to discretely solve the binary optimization problem, how to efficiently update hash codes and hash functions. To address these issues, in this paper, we propose a novel supervised online cross-modal hashing method, i.e., Label EMbedding ONline hashing, LEMON for short. It builds a label embedding framework including label similarity preserving and label reconstructing, which may generate discriminative binary codes and reduce the computational complexity. Furthermore, it not only preserves the pairwise similarity of incoming data, but also establishes a connection between newly coming data and existing data by the inner product minimization on a block similarity matrix. In the light of this, it can exploit more similarity information and make the optimization less sensitive to incoming data, leading to effective binary codes. In addition, we design a discrete optimization algorithm to solve the binary optimization problem without relaxation. Therefore, the quantization error can be reduced. Moreover, its computational complexity is only relevant to the size of incoming data, making it very efficient and scalable to large-scale datasets. Extensive experimental results on three benchmark datasets demonstrate that LEMON outperforms some state-of-the-art offline and online cross-modal hashing methods in terms of accuracy and efficiency. CCS CONCEPTS • Computing methodologies → Learning paradigms; • Information systems → Multimedia and multimodal retrieval.

Angular Quantization Online Hashing for Image Retrieval

“…Mingbao Lin et.al [42] proposes Fast Class-wise Updating for Online Hashing (FCOH), class-based update method is brought up to decompose binary code learning, and a semirelaxation strategy is adopted in the optimization process, which can well solve the burden of a large number of training batches. Label EMbedding ONline hashing (LEMON) [43] unifies the label similarity and label semantic embedding into a unified framework, and uses a two-step learning method to efficiently update the hash function and binary code, aiming to generate a binary code with strong resolution and effectively reduce the quantization error. Flexible Online Multimodal Hashing (FOMH) [44] proposes to deal with the multimodal image retrieval problem in the form of streaming data, using the multi-modal data weighting method, combined with asymmetric semantic supervision, to generate a binary code with strong versatility between modalities.…”

Online Enhanced Semantic Hashing: Towards Effective and Efficient Retrieval for Streaming Multi-Modal Data

“…Such strategy may lose the information of old data. The state-of-the-art online cross-modal hashing (Wang, Luo, and Xu 2020) learns from the above four parts and is endowed with impressive results. However, it is impossible to handle class incremental problem because the incorrect dimensions for matrix multiplication error will happen when performing ⃗ P (t)T P (t) operation if new classes come.…”

“…To overcome the limitation, many efforts have been devoted to online hashing. Similarly, we could roughly classify the literature into online uni-modal hashing (Huang, Yang, and Zheng 2013;Cakir and Sclaroff 2015;Chen, King, and Lyu 2017;Weng and Zhu 2020;Tian, Ng, and Wang 2019;Chen et al 2021a), online cross-modal hashing (Xie, Shen, and Zhu 2016;Qi, Wang, and Li 2017;Wang, Luo, and Xu 2020;Yi et al 2021;Zhan et al 2022), and online multi-modal hashing (Xie et al 2017;Lu et al 2019a).…”

Graph Convolutional Multi-modal Hashing for Flexible Multimedia Retrieval

“…Recent developments in massive multimedia data [11,30,43] have heightened the need for multi-modal hashing technology [19,44], which can support large-scale multimedia retrieval with its extremely low storage cost and high retrieval efficiency. Different from uni-modal hashing [5,18,22] which trains and searches data from a single source, and cross-modal hashing [1,29,37] which explores a shared subspace for two heterogeneous modalities and achieves mutual retrieval across them, multi-modal hashing [23,28,35,41] is a real-world application that data are collected from diverse sources or represented by heterogeneous features from different modalities [39]. It focuses on developing collaborative relationships of multiple modalities and supporting multimedia retrieval task.…”