Evasion Attacks against Machine Learning at Test Time

Biggio, Battista; Corona, Igino; Maiorca, Davide; Nelson, Brent D.; Šrndić, Nedim; Laskov, Pavel; Giacinto, Giorgio; Roli, Fabio

doi:10.1007/978-3-642-40994-3_25

Cited by 1,451 publications

(1,505 citation statements)

References 14 publications

Supporting

Mentioning

1,402

Contrasting

Unclassified

Order By: Relevance

“…Two main attack scenarios are often considered in the field of adversarial machine learning, i.e., evasion and poisoning [4], [7], [57], [3], [2], [31], [6], [5]. In an evasion attack, the attacker manipulates malicious samples at test time to have them misclassified as legitimate by a trained classifier, without having influence over the training data.…”

Section: Attack Model and Scenariosmentioning

confidence: 99%

Who Are You? A Statistical Approach to Measuring User Authenticity

Jain

Freeman²,

Biggio

et al. 2016

Proceedings 2016 Network and Distributed System Security Symposium

Self Cite

136

View full text Add to dashboard Cite

Abstract-Passwords are used for user authentication by almost every Internet service today, despite a number of wellknown weaknesses. Numerous attempts to replace passwords have failed, in part because changing users' behavior has proven to be difficult. One approach to strengthening password-based authentication without changing user experience is to classify login attempts into normal and suspicious activity based on a number of parameters such as source IP, geo-location, browser configuration, and time of day. For the suspicious attempts, the service can then require additional verification, e.g., by an additional phone-based authentication step. Systems working along these principles have been deployed by a number of Internet services but have never been studied publicly. In this work, we perform the first public evaluation of a classification system for user authentication. In particular:(i) We develop a statistical framework for identifying suspicious login attempts. (ii) We develop a fully functional prototype implementation that can be evaluated efficiently on large datasets. (iii) We validate our system on a sample of real-life login data from LinkedIn as well as simulated attacks, and demonstrate that a majority of attacks can be prevented by imposing additional verification steps on only a small fraction of users. (iv) We provide a systematic study of possible attackers against such a system, including attackers targeting the classifier itself.

show abstract

Section: Attack Model and Scenariosmentioning

confidence: 99%

Who Are You? A Statistical Approach to Measuring User Authenticity

Jain

Freeman²,

Biggio

et al. 2016

Proceedings 2016 Network and Distributed System Security Symposium

Self Cite

136

View full text Add to dashboard Cite

show abstract

“…In previous work (Biggio et al, 2013), the success rate was about the same, even if they made a deeper analysis of it using various settings of their SVM.…”

Section: An Illustration Of the Results Of This Attack Is Shown Onmentioning

confidence: 90%

“…Several attacks have been proposed (e.g. (Ateniese et al, 2013;Biggio et al, 2013)). In (Biggio et al, 2013), the authors propose to evade SVM and Neural Network classifiers using gradientdescent algorithms.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Malware Detection in PDF Files using Machine Learning

Cuan

Damien

Delaplace

et al. 2018

Proceedings of the 15th International Joint Conference on E-Business and Telecommunications

View full text Add to dashboard Cite

Abstract:We present how we used machine learning techniques to detect malicious behaviours in PDF files. At this aim, we first set up a SVM (Support Machine Vector) classifier that was able to detect 99.7% of malware. However, this classifier was easy to lure with malicious PDF files, which we forged to make them look like clean ones. For instance, we implemented a gradient-descent attack to evade this SVM. This attack was almost 100% successful. Next, we provided counter-measures to this attack: a more elaborated features selection and the use of a threshold allowed us to stop up to 99.99% of this attack. Finally, using adversarial learning techniques, we were able to prevent gradient-descent attacks by iteratively feeding the SVM with malicious forged PDF files. We found that after 3 iterations, every gradient-descent forged PDF file were detected, completely preventing the attack.

show abstract

“…In this framework, given an attacker's cost function, the goal is to find a lowest attacker-cost instance that the classifier labels as negative (i.e., it passes through). Various other papers in the literature have used such a model [7], [28], [29]. Specifically, we consider how many tries does it take an adversary to compromise the security of a classifier.…”

Section: Attack Modelmentioning

confidence: 99%

K-means++ vs. Behavioral Biometrics: One Loop to Rule Them All

Negi¹,

Sharma²,

Jain³

et al. 2018

Proceedings 2018 Network and Distributed System Security Symposium

View full text Add to dashboard Cite

Abstract-Behavioral biometrics, a field that studies patterns in an individual's unique behavior, has been researched actively as a means of authentication for decades. Recently, it has even been adopted in many real world scenarios. In this paper, we study keystroke dynamics, the most researched of such behavioral biometrics, from the perspective of an adversary. We designed two adversarial agents with a standard accuracy convenience tradeoff: Targeted K-means++, which is an expensive, but extremely effective adversarial agent, and Indiscriminate K-means++, which is slightly less powerful, but adds no overhead cost to the attacker. With Targeted K-means++ we could compromise the security of 40-70% of users within ten tries. In contrast, with Indiscriminate K-means++, the security of 30-50% of users was compromised. Therefore, we conclude that while keystroke dynamics has potential, it is not ready for security critical applications yet. Future keystroke dynamics research should use such adversaries to benchmark the performance of the detection algorithms, and design better algorithms to foil these. Finally, we show that the K-means++ adversarial agent generalizes well to even other types of behavioral biometrics data by applying it on a dataset of touchscreen swipes.

show abstract

Evasion Attacks against Machine Learning at Test Time

Cited by 1,451 publications

References 14 publications

Who Are You? A Statistical Approach to Measuring User Authenticity

Who Are You? A Statistical Approach to Measuring User Authenticity

Malware Detection in PDF Files using Machine Learning

K-means++ vs. Behavioral Biometrics: One Loop to Rule Them All

Contact Info

Product

Resources

About