FENCE: Feasible Evasion Attacks on Neural Networks in Constrained Environments

Abstract

As advances in Deep Neural Networks (DNNs) demonstrate unprecedented levels of performance in many critical applications, their vulnerability to attacks is still an open question. We consider evasion attacks at testing time against Deep Learning in constrained environments, in which dependencies between features need to be satisfied. These situations may arise naturally in tabular data or may be the result of feature engineering in specific application domains, such as threat detection in cyber security. We propose a general iterative gradient-based framework called FENCE for crafting evasion attacks that take into consideration the specifics of constrained domains and application requirements. We apply it against Feed-Forward Neural Networks trained for two cyber security applications: network traffic botnet classification and malicious domain classification, to generate feasible adversarial examples. We extensively evaluate the success rate and performance of our attacks, compare their improvement over several baselines, and analyze factors that impact the attack success rate, including the optimization objective and the data imbalance. We show that with minimal effort (e.g., generating 12 additional network connections), an attacker can change the model’s prediction from the Malicious class to Benign and evade the classifier. We show that models trained on datasets with higher imbalance are more vulnerable to our FENCE attacks. Finally, we demonstrate the potential of performing adversarial training in constrained domains to increase the model resilience against these evasion attacks.

Learning Location From Shared Elevation Profiles in Fitness Apps: A Privacy Perspective

“…First, we performed evaluations on the raw data. As the multi-class classification models require a fixed input shape and an arbitrary elevation profile does not have a specific length, we divided the elevation profiles into equallength (32) chunks and use the raw data to train and test the models. For all datasets and threat models, we used a slightly modified version of the soft voting ensemble method while testing with raw data.…”

“…in the image domain is a valid perturbation candidate. We emphasize, however, that this issue is not particular to this problem space we address in this paper, but applicable to a range of problems in general, such as software [29]- [31] and network domains [32], where the feature representation used for implementing the machine learning algorithm transforms the input by upholding a dependency among the features, which is not the case in the original image modality used in computer vision applications [33].…”

Towards resilient cyber networks against adversarial attacks

“…In the fourth problem, we propose algorithms for designing evasion attacks against ML classifiers to test the robustness of DNN models and understand possible countermeasures [36].…”

“…• Robustness of ML security measures: We design ML techniques to predict malicious or benign behavior in the networks and study their robustness by developing the framework for mounting evasion attacks against ML models feasible in the cybersecurity constrained environments [36].…”

“…In this chapter, I introduce a novel, a general framework for mounting evasion attacks against deep learning models in constrained application domains presented in [36]. The framework is named FENCE (Feasible Evasion Attacks on Neural Networks in Constrained Environments).…”