Explainability and Adversarial Robustness for RNNs

Abstract

Recurrent Neural Networks (RNNs) yield attractive properties for constructing Intrusion Detection Systems (IDSs) for network data. With the rise of ubiquitous Machine Learning (ML) systems, malicious actors have been catching up quickly to find new ways to exploit ML vulnerabilities for profit. Recently developed adversarial ML techniques focus on computer vision and their applicability to network traffic is not straightforward: Network packets expose fewer features than an image, are sequential and impose several constraints on their features.We show that despite these completely different characteristics, adversarial samples can be generated reliably for RNNs. To understand a classifier's potential for misclassification, we extend existing explainability techniques and propose new ones, suitable particularly for sequential data. Applying them shows that already the first packets of a communication flow are of crucial importance and are likely to be targeted by attackers. Feature importance methods show that even relatively unimportant features can be effectively abused to generate adversarial samples. We thus introduce the concept of feature sensitivity which quantifies how much potential a feature has to cause misclassification.Since traditional evaluation metrics such as accuracy are not sufficient for quantifying the adversarial threat, we propose the Adversarial Robustness Score (ARS) for comparing IDSs and show that an adversarial training procedure can significantly and successfully reduce the attack surface.

Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges

“…According to them, gradient similarity shows the influence of training data on test samples, and behaves differently for genuine and adversarial input samples, enabling the detection of various adversarial attacks with high accuracy. Some other interesting works are relying on explainable ML techniques to guard against adversarial attacks [25], [218], [220], [221]. However, the explanations/information regarding the working mechanism of ML algorithms revealed by explainability methods could also be utilized to generate more effective adversarial attacks on the algorithms [57].…”

Subverting Network Intrusion Detection: Crafting Adversarial Examples Accounting for Domain-Specific Constraints

“…Papernot et al [16] have proposed the adversarial distance, which measures distances between different labels using gradient information to indicate the risk of misclassification between classes. Meanwhile, in the NIDS domain, Hartl et al [8] have developed the Adversarial Risk Score (ARS), which is a distance-based robustness score for classifiers against adversarial examples. In their work, they use a Recurrent Neural Network (RNN) for classification, and investigate the feature sensitivity of their classifier.…”

“…due to complementary security measures). Moreover, weights can be assigned empirically using a feature sensitivity analysis [8].…”