‘Security Theater’: On the Vulnerability of Classifiers to Exploratory Attacks

Abstract

The increasing scale and sophistication of cyber-attacks has led to the adoption of machine learning based classification techniques, at the core of cybersecurity systems. These techniques promise scale and accuracy, which traditional rule/signature based methods cannot. However, classifiers operating in adversarial domains are vulnerable to evasion attacks by an adversary, who is capable of learning the behavior of the system by employing intelligently crafted probes. Classification accuracy in such domains provides a false sense of security, as detection can easily be evaded by carefully perturbing the input samples. In this paper, a generic data driven framework is presented, to analyze the vulnerability of classification systems to black box probing based attacks. The framework uses an exploration-exploitation based strategy, to understand an adversary's point of view of the attack-defense cycle. The adversary assumes a black box model of the defender's classifier and can launch indiscriminate attacks on it, without information of the defender's model type, training data or the domain of application. Experimental evaluation on 10 real world datasets demonstrates that even models having high perceived accuracy (>90%), by a defender, can be effectively circumvented with a high evasion rate (>95%, on average). The detailed attack algorithms, adversarial model and empirical evaluation, serve as a background for developing secure machine learning based systems.

A Review of Deep Learning Security and Privacy Defensive Techniques

“…e researchers [81,83,84] proposed different techniques to detect adversarial examples in the input and to create different benign and adversarial examples. As we mentioned earlier, the target of the attacker is to add more noise to formulate [47] Changes the discriminant results Resource consuming [50][51][52] Misclassifies positive sample Integrity attack [53] False negative passes through the system Easily detected [54][55][56] Availability attack [57] False positive results in blocking records Time and resource consuming [58][59][60] Privacy violation attack [61] Easily exploit the training dataset Its performance is not reliable as it based on iterations [62][63][64] Targeted attack [65] Misclassified to any arbitrary class It does not provide assurance about the generated samples [66][67][68] Indiscriminate attack [69] Good trade-off Perturbation is high [70,71] Highly efficient 10 Mobile Information Systems effective adversarial examples. According to [83], it is not easy to detect such adaptive attacks, and some detection techniques effectively work while some ineffective.…”

A dynamic‐adversarial mining approach to the security of machine learning

“…Machine learning systems deployed in the real world are vulnerable to exploratory attacks, which aim to degrade the learned model over time. Exploratory attacks are launched by an adversary by first learning about the characteristics of the system through carefully crafted probes and then morphing suspicious samples to cause evasion at test time (Biggio et al, ; Biggio, Fumera, & Roli, , ; Papernot, McDaniel, & Goodfellow, ; Sethi et al, ; Tramèr et al, ). Such attacks are commonplace and difficult to avoid, as they rely on the same black box access to the systems that a benign user is entitled to.…”

“…By measuring adversarial cost in terms of the number of features which need evasion, the efficacy of feature‐bagged models was demonstrated (Lowd & Meek, ), as robust models by design will require a majority of the features to be modified for a successful evasion. This reasoning is not valid for the case of an indiscriminate exploratory attack, where an adversary is interested in generating any sample which evades C without any predefined set of attack samples (Sethi et al, ). In this setting, the impact of feature‐bagged ensemble is similar to that of a simple model as seen from the evasion probability and adversarial certainty computations of Table .…”

“…One such black box, data-driven and algorithmic attack strategy, was proposed by Sethi et al (2017). The work by Sethi et al (2017) proposed the anchor points (AP) attack strategy, which uses an exploration-exploitation framework to perform evasion on classifiers, as shown in Figure 4. In these attacks, the adversary starts with exploration of the deployed classifier model (green) by using crafted probing samples.…”

“…However, the increased exuberance in the application of machine learning has led to the overlooking of its security vulnerabilities. Recent works on adversarial machine learning (Papernot, McDaniel, Jha, et al, , ; Sethi, Kantardzic, & Ryu, ; Tramèr, Zhang, Juels, Reiter, & Ristenpart, ) have demonstrated that deployed classification systems are vulnerable to adversarial perturbations at test time, which cause the models to degrade over time. Such attacks on the integrity of machine learning systems are termed as exploratory attacks (Barreno, Nelson, Sears, Joseph, & Tygar, ), as they are caused at test time by an adversary learning and evading the characteristics of the deployed classifier model.…”