Efficient Query-based Black-box Attack against Cross-modal Hashing Retrieval

Abstract

Deep cross-modal hashing retrieval models inherit the vulnerability of deep neural networks. They are vulnerable to adversarial attacks, especially for the form of subtle perturbations to the inputs. Although many adversarial attack methods have been proposed to handle the robustness of hashing retrieval models, they still suffer from two problems: 1) Most of them are based on the white-box settings, which is usually unrealistic in practical application. 2) Iterative optimization for the generation of adversarial examples in them results in heavy computation. To address these problems, we propose an Efficient Query-based Black-Box Attack (EQB 2 A) against deep cross-modal hashing retrieval, which can efficiently generate adversarial examples for the black-box attack. Specifically, by sending a few query requests to the attacked retrieval system, the cross-modal retrieval model stealing is performed based on the neighbor relationship between the retrieved results and the query, thus obtaining the knockoffs to substitute the attacked system. A multi-modal knockoffs-driven adversarial generation is proposed to achieve efficient adversarial example generation. While the entire network training converges, EQB 2 A can efficiently generate adversarial examples by forward-propagation with only given benign images. Experiments show that EQB 2 A achieves superior attacking performance under the black-box setting.

Once and for All: Universal Transferable Adversarial Perturbation against Deep Hashing-Based Facial Image Retrieval

“…(Hu et al 2021) propose AdvHash, the first targeted adversarial patch applicable to specific classes. Recently, some researchers are even paying new focuses on attacking cross-modal retrieval (Li et al 2021;Zhang et al 2023;Zhu et al 2023;Wang et al 2023). However, no existing method can simultaneously consider universality and transferability, which is the goal of our proposed UTAP.…”

“…Prior investigations have encompassed untargeted attacks (Yang et al 2018;Xiao, Wang, and Gao 2020), targeted attacks (Bai et al 2020;Hu et al 2021;Lu et al 2021), and transferable attacks (Xiao and Wang 2021) on DH models, demonstrating promising outcomes through adversarial perturbations and patches. Recent studies have even made advancements in attacking cross-modal retrieval (Li et al 2021;Zhang et al 2023;Zhu et al 2023;Wang et al 2023). These insights inspire us to design adversarial examples to prevent the exposure of privacy held within the database, as shown in Fig.…”

Deep Self-Supervised Hashing With Fine-Grained Similarity Mining for Cross-Modal Retrieval

“…In particular, due to the advantages of retrieval cost and retrieval speed, hash-based cross-modal similarity retrieval has recently received increasingly widespread interest. The goal of crossmodal hashing is to project high-dimensional raw features from different modalities to fixed-length discrete codes while maintaining correct semantic similarity relations [4], [5]. With the superior performance in similarity preservation, hash learning has been widely applied in various domains such as information retrieval [6], [7], recommender systems [8], and other applications.…”