A probabilistic model for multimodal hash function learning

Abstract

In recent years, both hashing-based similarity search and multimodal similarity search have aroused much research interest in the data mining and other communities. While hashing-based similarity search seeks to address the scalability issue, multimodal similarity search deals with applications in which data of multiple modalities are available. In this paper, our goal is to address both issues simultaneously. We propose a probabilistic model, called multimodal latent binary embedding (MLBE), to learn hash functions from multimodal data automatically. MLBE regards the binary latent factors as hash codes in a common Hamming space. Given data from multiple modalities, we devise an efficient algorithm for the learning of binary latent factors which corresponds to hash function learning. Experimental validation of MLBE has been conducted using both synthetic data and two realistic data sets. Experimental results show that MLBE compares favorably with two state-of-theart models.

Binary Set Embedding for Cross-Modal Retrieval

“…In our experiments, since few works focus on the local feature representation-based hashing scheme for cross-modal retrieval, we can only systematically compare the proposed BSE method with six prevailing global hashing methods for cross-modal retrieval tasks: CVH [11], MLBE [22], IMH [15], CMSSH [21], CHMIS [14], CMFH [16], and QCH [39]. For fair comparison, all the methods are implemented on the same SIFT features and word vectors in the image and text domains, respectively.…”

“…With extended SpH [18], Kumar and Udupa [11] proposed cross-view hashing (CVH) to generate binary codes for each modality via canonical correlation analysis (CCA). Multimodal latent binary embedding (MLBE) [22] is another cross-modal hashing method considering both the intermodal and intramodal similarity via a probabilistic model. To learn the hash function with good generalization, co-regularized hashing [13] was proposed to project data far from zero.…”

Effective multi-modal retrieval based on stacked auto-encoders

“…Other recent works include CMSSH [1], MLBE [25] and LSCMR [12]. CMSSH uses a boosting method to learn the projection function for each dimension of the latent space.…”

“…There has been a long stream of research on multi-modal retrieval [27,1,15,9,25,12,26,20]. These works share a similar query processing strategy which consists of two major steps.…”

“…We observe that most existing works, such as CVH [9], IMH [20], MLBE [25], CMSSH [1], and LSCMR [12], require a substantial amount of prior knowledge about the training data to learn effective mapping functions. Preparing prior knowledge in terms of large training dataset is labor-intensive, and due to manual intervention, the prepared knowledge may not be comprehensive in capturing the regularities (e.g., distribution or similarity relationships) of data.…”