Cross‐modal retrieval based on deep regularized hashing constraints

Abstract

Cross‐modal retrieval has attracted great attention due to the increasing demand for tremendous amounts of multimodal data in recent years. These retrievals could either be text‐to‐image or image‐to‐text. To address the problem of inappropriate information included between images and texts, we propose two cross‐modal recovery techniques established on a dual‐branch neural network defined on a common subspace and the hashing learning method. First, a cross‐modal recovery technique established on a multilabel information deep ranking model (MIDRM) is provided. In this method, we introduce a triplet‐loss function into the dual‐branch neural network model. This function takes advantage of the semantic information of the bimodal components, focusing on not only the similarities between similar images and text features but also the distances between dissimilar images and texts. Second, we establish a new cross‐modal hashing technique said to be the deep regularized hashing constraint (DRHC). In this method, the regularized function is used to replace the binary constraint, and the discrete value is constrained to a certain numerical range so that the network can achieve end‐to‐end training. Overall, the time complexity is greatly improved, and the occupied storage space is also greatly reduced. Different experiments on our proposed MIDRM and DRHC models demonstrate their superior performance to those of the state‐of‐the‐art methods on two widely used data sets. The experimental results show that our approach also increases the mean average precision of cross‐modal recovery.

A watermarking scheme for dual‐color images based on URV decomposition and image correction

“…As digital image is vivid, it is widely used and becomes one of the important means for human to express information 16 . However, it brings convenience to us as well as security risks 17 .…”

“…As digital image is vivid, it is widely used and becomes one of the important means for human to express information. 16 However, it brings convenience to us as well as security risks. 17 Therefore, how to properly protect the copyright of digital images and ensure the rights of authors and legitimate users is also a hot issue.…”

Foreground object structure transfer for unsupervised domain adaptation

“…In the past few decades, the number of Internet devices including computers, smart phones, digital cameras, and sensors, has increased rapidly, and people have entered the era of big data. The management of a large number of networked devices, [1][2][3] the safe transmission of information 4,5 and multimode data processing 6 are very arduous tasks. The performance of traditional methods is often unsatisfactory.…”

Contrastive hashing with vision transformer for image retrieval

“…Images, as the most commonly used modality data, have become the epitome of providing extremely rich information for our daily life. With the rapid accumulation of visual data on websites, effective and efficient image retrieval techniques have become an urgent demand for searching valuable information from large‐scale data sets 1,2 Many information retrieval techniques have been explored with the approximate nearest neighborhood 3,4 . Among them, hashing has become one of the most prominent techniques in the information retrieval field thanks to its distinctly fast computation and low storage 5 .…”

“…1 and G 2 indicate global views, and L 1 indicates local views. Then, we conduct two groups of experiments based on different sizes of augmented view settings on the MIRFlickr data set.…”