A dynamic‐adversarial mining approach to the security of machine learning

Abstract

Operating in a dynamic real‐world environment requires a forward thinking and adversarial aware design for classifiers beyond fitting the model to the training data. In such scenarios, it is necessary to make classifiers such that they are: (a) harder to evade, (b) easier to detect changes in the data distribution over time, and (c) be able to retrain and recover from model degradation. While most works in the security of machine learning have concentrated on the evasion resistance problem (a), there is little work in the areas of reacting to attacks (b) and (c). Additionally, while streaming data research concentrates on the ability to react to changes to the data distribution, they often take an adversarial agnostic view of the security problem. This makes them vulnerable to adversarial activity, which is aimed toward evading the concept drift detection mechanism itself. In this paper, we analyze the security of machine learning from a dynamic and adversarial aware perspective. The existing techniques of restrictive one‐class classifier models, complex learning‐based ensemble models, and randomization‐based ensemble models are shown to be myopic as they approach security as a static task. These methodologies are ill suited for a dynamic environment, as they leak excessive information to an adversary who can subsequently launch attacks which are indistinguishable from the benign data. Based on empirical vulnerability analysis against a sophisticated adversary, a novel feature importance hiding approach for classifier design is proposed. The proposed design ensures that future attacks on classifiers can be detected and recovered from. The proposed work provides motivation, by serving as a blueprint, for future work in the area of dynamic‐adversarial mining, which combines lessons learned from streaming data mining, adversarial learning, and cybersecurity. This article is categorized under: Technologies > Machine Learning Technologies > Classification Fundamental Concepts of Data and Knowledge > Motivation and Emergence of Data Mining

No Free Lunch Theorem for concept drift detection in streaming data classification: A review

“…For example, a new type of credit card fraud can appear that tries to circumvent existing fraud detection set in place. These changes in data streams, named concept drift (Pinage, dos Santos, & da Gama, 2016;Sethi, Kantardzic, Lyu, & Chen, 2018), often affect the underlying data distribution and reduce the performance of existing classifiers.…”

A Survey of Attacks Against Twitter Spam Detectors in an Adversarial Environment

“…[36] suggested that even if an adversary knows the importance of features used by the deployed model, he or she will not be able to evade detection without knowing the ML algorithm used. However, this cannot stop determined adversaries from trying every way possible to accomplish their goals [23]. Furthermore, as stated in Ref.…”

“…Consequently, considering the robustness of selected features and applying the disinformation method when designing a spam detector could help reduce the effect of adversaries' activities. However, this cannot stop determined adversaries from trying every way possible to accomplish their goals [23]. Furthermore, as stated in Ref.…”

A Survey of Attacks Against Twitter Spam Detectors in an Adversarial Environment

“…However, this cannot stop determined adversaries from trying every way possible to accomplish their goals [46]. Furthermore, as stated in [2], relying on obscurity in an adversarial environment is not a good security practice, as one should always overestimate rather than underestimate the adversary's capabilities.…”

“…The traditional assumption of stationarity of data distribution in ML is that the dataset used for training a classifier (such as SVM or RF) and the testing data (the future data that will be classified) have a similar underlying distribution. This assumption is violated in the adversarial environment, as adversaries are able to manipulate data either during training or before testing [46,2].…”