Is data clustering in adversarial settings secure?

Abstract

Clustering algorithms have been increasingly adopted in security applications to spot dangerous or illicit activities. However, they have not been originally devised to deal with deliberate attack attempts that may aim to subvert the clustering process itself. Whether clustering can be safely adopted in such settings remains thus questionable. In this work we propose a general framework that allows one to identify potential attacks against clustering algorithms, and to evaluate their impact, by making specific assumptions on the adversary's goal, knowledge of the attacked system, and capabilities of manipulating the input data. We show that an attacker may significantly poison the whole clustering process by adding a relatively small percentage of attack samples to the input data, and that some attack samples may be obfuscated to be hidden within some existing clusters. We present a case study on single-linkage hierarchical clustering, and report experiments on clustering of malware samples and handwritten digits

Detecting poisoning attacks on hierarchical malware classification systems

“…It has to be remarked that ∆DH t is not the kind of tool one hopes to employ as part of a proper informationtheoretic approach to the problem of quantifying differences between hierarchies: its sign is not constant, and there is much interaction among the summands in (15) and (16). It would be much more appropriate to construct and employ a proper notion of conditional entropy.…”

“…4, 14, 15. Given the central role of HC in cyber-security it is critical to understand and design around the vulnerabilities of hierarchical clustering methods. An interesting set of articles by Biggio et al 4,16,17 highlights a major vulnerability in HC: sensitivity to poisoning attacks. Biggio et al 4 emphasizes the centrality of clustering of malware families in the identification of common characteristics and the design of suitable countermeasures.…”

“…4 Their analysis provides convincing poisoning schemes against other existing algorithms. 16,17 In this paper, we test the hypothesis that, by employing natural entropy based diversity measures developed here, one could counter poisoning attacks against the SLHC method using a fairly simple reactive control mechanism based on allowing only small variations in the above measures for each time step. The idea is that large variations of the measure may only occur as a result of very coarse-grained alterations in the underlying hierarchy.…”

Pattern Recognition Systems under Attack

“…Capability: since poisoning samples do not need to preserve any malicious functionality, the attacker can add any number of them to the initial dataset, with the only constraint on their feature values given by feature normalization. The optimal attack strategy can be therefore formulated as We refer the reader to [23] for details on the (approximate) solution of this optimization problem. We considered three different greedy heuristics tailored to single-linkage hierarchical clustering, named Bridge (Best), Bridge (Hard), and Bridge (Soft): poisoning samples are added one at a time, until |A| points are added.…”

Fortifying Botnet Classification based on Venn-abers Prediction

“…And over 70% of the advanced malware created today uses one or more evasion techniques to avoid detection 2 . Attackers collect the knowledge of machine learning based detection approach and develop new evasion techniques, such as new communication channel [4][5][6][7][8], mimicry attack [9,10], gradient descent attack [9,10], poison attack [11], and so on.…”