Preventing Unauthorized Use of Proprietary Data: Poisoning for Secure Dataset Release

Fowl, Liam; Chiang, Ping-yeh; Goldblum, Micah; Geiping, Jonas; Bansal, Ankit; Czaja, Wojciech; Goldstein, Tom

doi:10.48550/arxiv.2103.02683

Cited by 5 publications

(14 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Early availability attacks focused on simple settings like logistic regression, and support vector machines [Biggio et al, 2012, Muñoz-González et al, 2017. Recently, heuristics have been leveraged to perform availability attacks on deep networks [Feng et al, 2019, Fowl et al, 2021. In contrast to availability attacks, integrity attacks focus on causing a victim model to misclassify a select set of targets.…”

Section: Reinforcement Learningmentioning

confidence: 99%

Execute Order 66: Targeted Data Poisoning for Reinforcement Learning

Foley¹,

Fowl²,

Goldstein³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Data poisoning for reinforcement learning has historically focused on general performance degradation, and targeted attacks have been successful via perturbations that involve control of the victim's policy and rewards. We introduce an insidious poisoning attack for reinforcement learning which causes agent misbehavior only at specific target states -all while minimally modifying a small fraction of training observations without assuming any control over policy or reward. We accomplish this by adapting a recent technique, gradient alignment, to reinforcement learning. We test our method and demonstrate success in two Atari games of varying difficulty.

show abstract

Section: Reinforcement Learningmentioning

confidence: 99%

Execute Order 66: Targeted Data Poisoning for Reinforcement Learning

Foley¹,

Fowl²,

Goldstein³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In general, data poisoning attacks perturb training data to intentionally cause some malfunctions of the target model [Biggio and Roli, 2018, Goldblum et al, 2020, Schwarzschild et al, 2021. A common class of poisoning attacks aim to cause test-time error on some given samples [Koh and Liang, 2017, Muñoz-González et al, 2017, Chen et al, 2017, Koh et al, 2018, Shafahi et al, 2018 or on all unseen samples [Biggio et al, 2012, Feng et al, 2019, Liu and Shroff, 2019, Shen et al, 2019, Huang et al, 2021, Yuan and Wu, 2021, Fowl et al, 2021a. The latter attacks are also known as indiscriminate poisoning attacks as they do not have specific target examples [Barreno et al, 2010].…”

Section: Related Workmentioning

confidence: 99%

“…The perturbations are restricted to be small and within in a set ∆. Directly solving Equation ( 1) is intractable for deep neural networks and recent works have designed multiple approximate solutions [Feng et al, 2019, Fowl et al, 2021a, Yuan and Wu, 2021. Feng et al [2019] use multiple rounds of optimization to generate perturbations.…”

Section: The Alternating Optimization Approachmentioning

confidence: 99%

“…For instance, a private company has collected more than three billion face images to build commercial face recognition models without acquiring any user consents [Hill, 2020]. To address those concerns, many data poisoning attacks have been proposed to prevent data from being learned by unauthorized deep models [Feng et al, 2019, Shen et al, 2019, Shan et al, 2020, Huang et al, 2021, Cherepanova et al, 2021, Yuan and Wu, 2021, Fowl et al, 2021a. They add imperceptible perturbations to the training data so that the model accuracy on unseen data is arbitrarily bad.…”

Section: Introductionmentioning

confidence: 99%

“…Roughly speaking, there are three methods available to construct the indiscriminate poisoning attack. The first method formulates the perturbations as the solution of a bi-level optimization problem [Biggio et al, 2012, Feng et al, 2019, Fowl et al, 2021a, Yuan and Wu, 2021. The bi-level optimization problem requires models trained on perturbed data to have the maximum loss on unseen data.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Availability Attacks Create Shortcuts

Yu,

Zhang,

Chen

et al. 2021

Preprint

View full text Add to dashboard Cite

Indiscriminate data poisoning attacks, which add imperceptible perturbations to training data to maximize the test error of trained models, have become a trendy topic because they are thought to be capable of preventing unauthorized use of data. In this work, we investigate why these perturbations work in principle. We find that the perturbations of advanced poisoning attacks are almost linear separable when assigned with the target labels of the corresponding samples, which hence can work as shortcuts for the learning objective. This important population property has not been unveiled before. Moreover, we further verify that linear separability is indeed the workhorse for poisoning attacks. We synthesize linear separable data as perturbations and show that such synthetic perturbations are as powerful as the deliberately crafted attacks. Our finding suggests that the shortcut learning problem is more serious than previously believed as deep learning heavily relies on shortcuts even if they are of an imperceptible scale and mixed together with the normal features. This finding also suggests that pre-trained feature extractors would disable these poisoning attacks effectively.

show abstract

Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning

et al. 2023

View full text Add to dashboard Cite

The success of machine learning is fueled by the increasing availability of computing power and large training datasets. The training data is used to learn new models or update existing ones, assuming that it is sufficiently representative of the data that will be encountered at test time. This assumption is challenged by the threat of poisoning, an attack that manipulates the training data to compromise the model’s performance at test time. Although poisoning has been acknowledged as a relevant threat in industry applications, and a variety of different attacks and defenses have been proposed so far, a complete systematization and critical review of the field is still missing. In this survey, we provide a comprehensive systematization of poisoning attacks and defenses in machine learning, reviewing more than 100 papers published in the field in the last 15 years. We start by categorizing the current threat models and attacks, and then organize existing defenses accordingly. While we focus mostly on computer-vision applications, we argue that our systematization also encompasses state-of-the-art attacks and defenses for other data modalities. Finally, we discuss existing resources for research in poisoning, and shed light on the current limitations and open research questions in this research field.

show abstract

Preventing Unauthorized Use of Proprietary Data: Poisoning for Secure Dataset Release

Cited by 5 publications

References 25 publications

Execute Order 66: Targeted Data Poisoning for Reinforcement Learning

Execute Order 66: Targeted Data Poisoning for Reinforcement Learning

Availability Attacks Create Shortcuts

Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning

Contact Info

Product

Resources

About