Bullseye Polytope: A Scalable Clean-Label Poisoning Attack with Improved Transferability

Aghakhani, Hojjat; Meng, Dongyu; Wang, Yuxiang; Kruegel, Christopher; Vigna, Giovanni

doi:10.48550/arxiv.2005.00191

Cited by 6 publications

(19 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Backdoor attacks. As a variant of poisoning attacks, a backdoor attack aims to mislead the model on some specific target inputs [1,24,34,76,97], or inject trigger patterns [12,30,52,70,83,86], without affecting model performance on clean test data. Backdoor attacks have a similar formulation as poisoning attacks, except that S val in Eq.…”

Section: Attacking Strategiesmentioning

confidence: 99%

Accumulative Poisoning Attacks on Real-time Data

Pang¹,

Yang²,

Dong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Collecting training data from untrusted sources exposes machine learning services to poisoning adversaries, who maliciously manipulate training data to degrade the model accuracy. When trained on offline datasets, poisoning adversaries have to inject the poisoned data in advance before training, and the order of feeding these poisoned batches into the model is stochastic. In contrast, practical systems are more usually trained/fine-tuned on sequentially captured real-time data, in which case poisoning adversaries could dynamically poison each data batch according to the current model state. In this paper, we focus on the real-time settings and propose a new attacking strategy, which affiliates an accumulative phase with poisoning attacks to secretly (i.e., without affecting accuracy) magnify the destructive effect of a (poisoned) trigger batch. By mimicking online learning and federated learning on CIFAR-10, we show that the model accuracy will significantly drop by a single update step on the trigger batch after the accumulative phase. Our work validates that a well-designed but straightforward attacking strategy can dramatically amplify the poisoning effects, with no need to explore complex techniques. * Equal contribution Preprint. Under review.

show abstract

Section: Attacking Strategiesmentioning

confidence: 99%

Accumulative Poisoning Attacks on Real-time Data

Pang¹,

Yang²,

Dong³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…So-called clean-label poisoning attacks have been recently proposed against image classification systems for the first time [1,27,38], wherein the adversary has no control over the labeling process. While the poison samples are perturbed to achieve the system's misbehavior with regards to specific target inputs, such perturbations are small enough to justify the original labels in the human perception.…”

Section: Introductionmentioning

confidence: 99%

“…To mitigate this limitation, we adopt the Bullseye Polytope attack [1] and extend it as follows: at the beginning of each round of the attack, the surrogate networks are trained on the current version of the (poisoned) dataset. Then, the poisons are changed to achieve the desired heuristics with regards to the refreshed surrogate models.…”

Section: Introductionmentioning

confidence: 99%

“…• End-to-end learning. Unlike previous clean-label poisoning attacks [1,27,38], in our threat model, we assume the victim system is trained on the poisoned data from scratch.…”

Section: Introductionmentioning

confidence: 99%

“…To support further research in this area, we release the source code of all experiments as well as the poison samples generated by VENOMAVE at https://github.com/9yte/ VenoMave. 1…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

VenoMave: Targeted Poisoning Against Speech Recognition

Aghakhani¹,

Schönherr²,

Eisenhofer³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

In the past few years, we observed a wide adoption of practical systems that use Automatic Speech Recognition (ASR) systems to improve human-machine interaction. Modern ASR systems are based on neural networks and prior research demonstrated that these systems are susceptible to adversarial examples, i.e., malicious audio inputs that lead to misclassification by the victim's network during the system's run time. The research question if ASR systems are also vulnerable to data poisoning attacks is still unanswered. In such an attack, a manipulation happens during the training phase of the neural network: an adversary injects malicious inputs into the training set such that the neural network's integrity and performance are compromised.In this paper, we present the first data poisoning attack in the audio domain, called VENOMAVE. Prior work in the image domain demonstrated several types of data poisoning attacks, but they cannot be applied to the audio domain. The main challenge is that we need to attack a time series of inputs. To enforce a targeted misclassification in an ASR system, we need to carefully generate a specific sequence of disturbed inputs for the target utterance, which will eventually be decoded to the desired sequence of words. More specifically, the adversarial goal is to produce a series of misclassification tasks and in each of them, we need to poison the system to misrecognize each frame of the target file. To demonstrate the practical feasibility of our attack, we evaluate VENOMAVE on an ASR system that detects sequences of digits from 0 to 9. When poisoning only 0.94% of the dataset on average, we achieve an attack success rate of 83.33%. We conclude that data poisoning attacks against ASR systems represent a real threat that needs to be considered.

show abstract

Adversarial Deep Learning: A Survey on Adversarial Attacks and Defense Mechanisms on Image Classification

et al. 2022

View full text Add to dashboard Cite

The popularity of adapting deep neural networks (DNNs) in solving hard problems has increased substantially. Specifically, in the field of computer vision, DNNs are becoming a core element in developing many image and video classification and recognition applications. However, DNNs are vulnerable to adversarial attacks, in which, given a well-trained image classification model, a malicious input can be crafted by adding mere perturbations to misclassify the image. This phenomena raise many security concerns in utilizing DNNs in critical life applications which attracts the attention of academic and industry researchers. As a result, multiple studies have proposed discussing novel attacks that can compromise the integrity of state-of-the-art image classification neural networks. The raise of these attacks urges the research community to explore countermeasure methods to mitigate these attacks and increase the reliability of adapting DDNs in different major applications. Hence, various defense strategies have been proposed to protect DNNs against adversarial attacks. In this paper, we thoroughly review the most recent and state-of-the-art adversarial attack methods by providing an in-depth analysis and explanation of the working process of these attacks. In our review, we focus on explaining the mathematical concepts and terminologies of the adversarial attacks, which provide a comprehensive and solid survey to the research community. Additionally, we provide a comprehensive review of the most recent defense mechanisms and discuss their effectiveness in defending DNNs against adversarial attacks. Finally, we highlight the current challenges and open issues in this field as well as future research directions.

show abstract

Bullseye Polytope: A Scalable Clean-Label Poisoning Attack with Improved Transferability

Cited by 6 publications

References 0 publications

Accumulative Poisoning Attacks on Real-time Data

Accumulative Poisoning Attacks on Real-time Data

VenoMave: Targeted Poisoning Against Speech Recognition

Adversarial Deep Learning: A Survey on Adversarial Attacks and Defense Mechanisms on Image Classification

Contact Info

Product

Resources

About