Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning

Despite the impressive performances reported by deep neural networks in different application domains, they remain largely vulnerable to adversarial examples, i.e., input samples that are carefully perturbed to cause misclassification at test time. In this work, we propose a deep neural rejection mechanism to detect adversarial examples, based on the idea of rejecting samples that exhibit anomalous feature representations at different network layers. With respect to competing approaches, our method does not require generating adversarial examples at training time, and it is less computationally demanding. To properly evaluate our method, we define an adaptive white-box attack that is aware of the defense mechanism and aims to bypass it. Under this worst-case setting, we empirically show that our approach outperforms previously-proposed methods that detect adversarial examples by only analyzing the feature representation provided by the output network layer.

show abstract

“…Another research direction may be that of testing our defense against training-time poisoning attacks [2], [5], [36]- [38].…”

Section: Discussionmentioning

confidence: 99%

Deep neural rejection against adversarial examples

Sotgiu

Demontis

Melis

et al. 2020

EURASIP J. on Info. Security

Self Cite

View full text Add to dashboard Cite

show abstract

“…Other than adversarial examples, we could also leverage data poisoning attacks [65][66][67][68][69][70][71][72] to defend against inference attacks. Specifically, an attacker needs to train an ML classifier in inference attacks.…”

Section: Data Poisoning Attacks Based Defensesmentioning

confidence: 99%

Defending Against Machine Learning Based Inference Attacks via Adversarial Examples: Opportunities and Challenges

Jia

Gong

2020

Adaptive Autonomous Secure Cyber Systems

View full text Add to dashboard Cite

As machine learning (ML) becomes more and more powerful and easily accessible, attackers increasingly leverage ML to perform automated large-scale inference attacks in various domains. In such an ML-equipped inference attack, an attacker has access to some data (called public data) of an individual, a software, or a system; and the attacker uses an ML classifier to automatically infer their private data. Inference attacks pose severe privacy and security threats to individuals and systems. Inference attacks are successful because private data are statistically correlated with public data, and ML classifiers can capture such statistical correlations. In this chapter, we discuss the opportunities and challenges of defending against ML-equipped inference attacks via adversarial examples. Our key observation is that attackers rely on ML classifiers in inference attacks. The adversarial machine learning community has demonstrated that ML classifiers have various vulnerabilities. Therefore, we can turn the vulnerabilities of ML into defenses against inference attacks. For example, ML classifiers are vulnerable to adversarial examples, which add carefully crafted noise to normal examples such that an ML classifier makes predictions for the examples as we desire. To defend against inference attacks, we can add carefully crafted noise into the public data to turn them into adversarial examples, such that attackers' classifiers make incorrect predictions for the private data. However, existing methods to construct adversarial examples are insufficient because they did not consider the unique challenges and requirements for the crafted noise at defending against inference attacks. In this chapter, we take defending against inference attacks in online social networks as an example to illustrate the opportunities and challenges.

show abstract

“…Helen does not prevent a malicious party from choosing a bad dataset for the coopetitive computation (e.g., in an attempt to alter the computation result). In particular, Helen does not prevent poisoning attacks [48,19]. MPC protocols generally do not protect against bad inputs because there is no way to ensure that a party provides true data.…”

Section: Threat Modelmentioning

confidence: 99%

Helen: Maliciously Secure Coopetitive Learning for Linear Models

Zheng

Popa

Gonzalez

et al. 2019

2019 IEEE Symposium on Security and Privacy (SP)

111

View full text Add to dashboard Cite

Many organizations wish to collaboratively train machine learning models on their combined datasets for a common benefit (e.g., better medical research, or fraud detection). However, they often cannot share their plaintext datasets due to privacy concerns and/or business competition. In this paper, we design and build Helen, a system that allows multiple parties to train a linear model without revealing their data, a setting we call coopetitive learning. Compared to prior secure training systems, Helen protects against a much stronger adversary who is malicious and can compromise m − 1 out of m parties. Our evaluation shows that Helen can achieve up to five orders of magnitude of performance improvement when compared to training using an existing state-of-the-art secure multi-party computation framework.

show abstract

Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning

Cited by 598 publications

References 34 publications

Deep neural rejection against adversarial examples

Deep neural rejection against adversarial examples

Defending Against Machine Learning Based Inference Attacks via Adversarial Examples: Opportunities and Challenges

Helen: Maliciously Secure Coopetitive Learning for Linear Models

Contact Info

Product

Resources

About