Subpopulation Data Poisoning Attacks

Jagielski, Matthew; Severi, Giorgio; Harger, Niklas Pousette; Oprea, Alina

doi:10.48550/arxiv.2006.14026

Cited by 3 publications

(7 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conventionally, a poisoning attack is to degrade the model overall inference accuracy for clean samples of its primary task [44]. Poisoning attack is also often called availability attack [45], [46] in a sense that such attack results in lower accuracy of the model, akin to a denial of service attack. In contrast, though a backdoor attack can be realized through data poisoning, a backdoor attack retains the inference accuracy for benign samples of its primary task and only misbehaves in the presence of the secret trigger stealthily.…”

Section: Data Poisoning Attackmentioning

confidence: 99%

See 1 more Smart Citation

Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review

Gao,

Doan,

Zhang

et al. 2020

Preprint

View full text Add to dashboard Cite

Backdoor attacks insert hidden associations or triggers to the deep learning models to override correct inference such as classification and make the system perform maliciously according to the attacker-chosen target while behaving normally in the absence of the trigger. As a new and rapidly evolving realistic attack, it could result in dire consequences, especially considering that the backdoor attack surfaces are broad. In 2019, the U.S. Army Research Office started soliciting countermeasures and launching TrojAI project, the National Institute of Standards and Technology has initialized a corresponding online competition accordingly.However, there is still no systematic and comprehensive review of this emerging area. Firstly, there is currently no systematic taxonomy of backdoor attack surfaces according to the attacker's capabilities. In this context, attacks are diverse and not combed. Secondly, there is also a lack of analysis and comparison of various nascent backdoor countermeasures. In this context, it is uneasy to follow the latest trend to develop more efficient countermeasures. Therefore, this work aims to provide the community with a timely review of backdoor attacks and countermeasures. According to the attacker's capability and affected stage of the machine learning pipeline, the attack surfaces are recognized to be wide and then formalized into six categorizations: code poisoning, outsourcing, pretrained, data collection, collaborative learning and post-deployment. Accordingly, attacks under each categorization are combed. The countermeasures are categorized into four general classes: blind backdoor removal, offline backdoor inspection, online backdoor inspection, and post backdoor removal. Accordingly, we review countermeasures and compare and analyze their advantages and disadvantages. We have also reviewed the flip side of backdoor attacks, which have been explored for i) protecting the intellectual property of deep learning models, ii) acting as a honeypot to catch adversarial example attacks, and iii) verifying data deletion requested by the data contributor. Overall, the research on the defense side is far behind the attack side, and there is no single defense that can prevent all types of backdoor attacks. In some This version might be updated.

show abstract

Section: Data Poisoning Attackmentioning

confidence: 99%

“…4) Data Collection: Data collection is usually error-prone and susceptible to untrusted sources [45]. If a user collects training data from multiple sources, then data poisoning attacks become a more realistic threat.…”

Section: Introductionmentioning

confidence: 99%

Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review

Gao,

Doan,

Zhang

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…To perform the attack, the adversary only requires access to the labels of the training dataset, however, to optimize the attack, it is often assumed that the adversary has access to the learner's loss function. To fully optimize this attack, the adversary would need either the learning model's parameters and read access to the samples in the training dataset or an auxiliary dataset that follows the same distribution as the training dataset [82].…”

Section: Targeted Attacksmentioning

confidence: 99%

“…To date, there does not seem to be any effective defenses to fight against targeted data poisoning attacks. According to [82], some data poisoning attacks are even impossible to defend against. Our proposed defensive strategies utilize deep learning techniques that may pave the way to the development of more generic defensive mechanisms for various targeted attacks.…”

Section: Targeted Attacksmentioning

confidence: 99%

See 1 more Smart Citation

Defeat data poisoning attacks on facial recognition applications

Cole¹

View full text Add to dashboard Cite

In the modern era, facial photos are used for a wide array of applications, from logging into a smartphone to bragging about a weekend getaway. With the vast amount of use cases for facial images, adversaries will attack these applications for profit. This dissertation focuses on two major applications of facial photos: facial authentication and deepfakes. Facial authentication has become increasingly popular on personal devices. Due to the ease of use, it has great potential to be widely deployed for web-service authentication in the near future in which people can easily log on to online accounts from different devices without memorizing lengthy passwords. However, the growing number of attacks targeting machine learning, especially Deep Neural Networks (DNN), which is commonly used for facial recognition, imposes big challenges on the successful roll-out of such web-service facial authentication. We demonstrate a new data poisoning attack, called replacement data poisoning, which does not require the adversary to have any knowledge of the server-side and simply needs a handful of malicious photo injections to enable an attacker to impersonate the victim in existing facial authentication systems. We then propose a novel defensive approach called DEFEAT that leverages deep learning techniques to automatically detect such attacks. Our experiments using real-world datasets achieve a detection accuracy of over 90 percent. Deepfakes target specific individuals to cause shame or misinformation. With the spread of fake news, deepfakes have become incredibly prevalent in recent years. With deepfakes, an adversary could have photographic or even video-graphic \proof" of someone, such as a politician, committing a devious act or saying untrue words. Our deepfake work consists of two parts. First, we propose a label ipping data poisoning attack targeting deepfake detectors. With over a 99 percent poison success rate in most cases, this attack demonstrates the devastating effects a data poisoning attack can have on deepfake detectors and how important a need to defend against this assault is. Our second contribution revolves around defending deepfake detectors from such an attack. We propose several defense strategies, most notably a convolutional neural network (CNN) based strategy to detect poisoned images. Our CNN-based approach achieves a greater than 98 percent poison detection rate while keeping the number of false positives to a minimum with a precision rate of over 99 percent in most cases.

show abstract

Algorithmic fairness in the real world: challenges and considerations

Ghosh

View full text Add to dashboard Cite

Happy moments and sad moments. Many, many cups of hot chocolate and late nights. The work I've done, the papers I've written, and the people and organizations I've collaborated with have all helped to get me through this process, and I am grateful for every experience that I have had on this journey.Foremost among those to whom I owe my thanks is my advisor, Christo Wilson. His guidance, support, and mentorship have given me the tools necessary to achieve most of the things I had listed in my Ph.D. statement of purpose. I've also been privileged enough to work with amazing individuals in the industry. Aalok, Mary, Lea, and Josh at Fiddler; Kristian, Tomo, and Rumman at Twitter; Your guidance, support and friendship have helped me jump start my career in the field, and I count myself incredibly lucky to have met and collaborated with you all. These acknowledgments would be incomplete without mentioning my friends and lab-mates in academia, both within and outside Northeastern. In no particular order, Ritam, Rajarshi, Geno,

show abstract

Subpopulation Data Poisoning Attacks

Cited by 3 publications

References 6 publications

Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review

Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review

Defeat data poisoning attacks on facial recognition applications

Algorithmic fairness in the real world: challenges and considerations

Contact Info

Product

Resources

About