Subpopulation Data Poisoning Attacks

Jagielski, Matthew; Severi, Giorgio; Harger, Niklas Pousette; Oprea, Alina

doi:10.1145/3460120.3485368

Cited by 41 publications

(29 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the subpopulation attack experiments, we use the Adult dataset [3]. This dataset was used for evaluation by the first subpopulation attack paper [7] and we directly use Jagielski et al's preprocessed version of the dataset. We conduct experiments on linear SVM model and compare our model-targeted poisoning attack in Algorithm 1 to the state-of-the-art KKT attack [9].…”

Section: Methodsmentioning

confidence: 99%

“…Subpopulations. We identify the subpopulations for the Adult dataset using k-means clustering techniques (ClusterMatch [7]) to obtain different clusters (k = 20 in our case). For each cluster, we select instances with label "<=50K" to form the subpopulation (indicating all instances in the subpopulation are in low income group).…”

Section: Methodsmentioning

confidence: 99%

“…The clean-label attack assumes adversaries can only perturb the features of the data, but the label is given by an oracle labeler [8,21,29,6]. In label-flipping attack, adversaries are only allowed to change the labels [1,26,27,7]. These restricted attacks are much weaker than the poisoning attacks without restrictions [9,5].…”

Section: Related Workmentioning

confidence: 99%

“…The attacker objective is typically one of two extremes: indiscriminate attacks, where the adversary's goal is simply to decrease the overall accuracy of the model [2,26,18,24,9]; and instance-targeted attacks, where the goal is to produce a classifier that misclassifies a particular known input [21,29,8]. Recently, Jagielski et al introduced a more realistic attacker objective known as a subpopulation attack, where the goal is to increase the error rate or obtain a particular output for a defined subpopulation of the data distribution [7]. Attacker objectives for realistic attacks are diverse and designing a unified and effective attack strategy for different attacker objectives is hard.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Model-Targeted Poisoning Attacks with Provable Convergence

Suya,

Mahloujifar,

Suri

et al. 2020

Preprint

View full text Add to dashboard Cite

Machine learning systems that rely on training data collected from untrusted sources are vulnerable to poisoning attacks, in which adversaries controlling some of the collected data are able to induce a corrupted model. In this paper, we consider poisoning attacks where there is an adversary who has a particular target classifier in mind and hopes to induce a classifier close to that target by adding as few poisoning points as possible. We propose an efficient poisoning attack based on online convex optimization. Unlike previous model-targeted poisoning attacks, our attack comes with provable convergence to any achievable target classifier. The distance from the induced classifier to the target classifier is inversely proportional to the square root of the number of poisoning points. We also provide a certified lower bound on the minimum number of poisoning points needed to achieve a given target classifier. We report on experiments showing our attack has performance that is similar to or better than the state-of-the-art attacks in terms of attack success rate and distance to the target model, while providing the advantages of provable convergence, and the efficiency benefits associated with being an online attack that can determine near-optimal poisoning points incrementally. * Equal contribution.Preprint. Under review.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Model-Targeted Poisoning Attacks with Provable Convergence

Suya,

Mahloujifar,

Suri

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…To increase stealthiness, some techniques exploit properties of image classification: TaCT [64] uses triggers that only work for a given class and WaNet [50] uses image warping as a trigger such that the trigger is imperceptible to humans. A recent subpopulation attack [35] does not use triggers, but instead supplies poisoned data targeting a specific "subpopulation" within the dataset. After training, the poisoned classifier will exclusively misclassify the target subpopulation.…”

Section: Attacker Controlled Trainingmentioning

confidence: 99%

Jigsaw Puzzle: Selective Backdoor Attack to Subvert Malware Classifiers

Yang¹,

Chen²,

Cortellazzi³

et al. 2022

Preprint

View full text Add to dashboard Cite

Malware classifiers are subject to training-time exploitation due to the need to regularly retrain using samples collected from the wild. Recent work has demonstrated the feasibility of backdoor attacks against malware classifiers, and yet the stealthiness of such attacks is not well understood. In this paper, we investigate this phenomenon under the clean-label setting (i.e., attackers do not have complete control over the training or labeling process). Empirically, we show that existing backdoor attacks in malware classifiers are still detectable by recent defenses such as MNTD. To improve stealthiness, we propose a new attack, Jigsaw Puzzle (JP), based on the key observation that malware authors have little to no incentive to protect any other authors' malware but their own. As such, Jigsaw Puzzle learns a trigger to complement the latent patterns of the malware author's samples, and activates the backdoor only when the trigger and the latent pattern are pieced together in a sample. We further focus on realizable triggers in the problem space (e.g., software code) using bytecode gadgets broadly harvested from benign software. Our evaluation confirms that Jigsaw Puzzle is effective as a backdoor, remains stealthy against state-of-the-art defenses, and is a threat in realistic settings that depart from reasoning about feature-space only attacks. We conclude by exploring promising approaches to improve backdoor defenses.

show abstract