Extending Detection with Privileged Information via Generalized Distillation

Abstract

Detection systems based on machine learning models are essential tools for system and enterprise defense. These systems construct models of attacks (or non-attacks) from past observations (i.e., features) using a training algorithm. After that, the detection systems use that model for detection at run-time. In this way, the detection system recognizes when the environmental state becomes-at least probabilistically-dangerous. A limitation of this traditional model of detection is that model training is limited to features available at run-time. However, many features are either too expensive to collect in real-time or only available after the fact. In traditional detection, such features are ignored for the purpose of detection. In this paper, we consider an alternative detection model learning approach, generalized distillation, that trains models using privileged informationfeatures available at training time but not at run-time-to improve the accuracy of detection systems. We use a deep neural network to implement generalized distillation for the training of detection models and making predictions. Our empirical study shows that detection with privileged information via generalized distillation increases precision and recall in systems of user face authentication, fast-flux bot detection, and malware classification over systems with no privileged information.

Deep Learning for Cyber Security Applications: A Comprehensive Survey

“…Dataset [179] Alexa & Private [180] Genome, Contagio, VirusShare, Drebin [181] Drebin [182] BIG 2015 [184] Private [183] Malicious Behavior Windows Audit Logs [185] VirusShare, Citadel, APT1 [186] Spambase [187] Wild dataset & BIG 2015 [188] GTSRB [189] Enron Spam Dataset [190] Mozilla Common Voice Dataset [191] NSL-KDD [192] AICS'2019 challenge dataset [193] Car-Hacking Dataset from HCRL [194] ADFA-LD [195] CIDDS-001 [196] KDDCup-99 [197] Ember [198] AmritaDGA [199] CycleGAN & StarGAN service (DDoS) attacks. Recently, a DL based architecture called generative adversarial network (GAN) was trained to generate adversarial domain names to intentionally bypass a DL based detector [179].…”

“…However, they fail to detect attacks that are less aggressive such as label flipping. Another study employed generalized distillation learning approach to train the DL based detection model using privileged features [187]. Attacks against computer vision based modules of autonomous vehicles in real-world applications were studied by using adversarial examples to misclassify advertisements and innocuous signs with a success rate of 95% [188].…”

A Comprehensive Tutorial and Survey of Applications of Deep Learning for Cyber Security

“…It can be observed that the proposed approach is effective but fails to detect attacks that are less aggressive like label flipping. In [388], the authors employs generalized distillation learning approach to train the DL based detection model using privileged features which are available at the time of training. They have shown that the proposed method leads to better accuracy when compared to systems with no privileged information.…”

Real-time malware process detection and automated process killing

“…"Privileged" data is data that is expensive or slow to generate and so will not be available during runtime. We use the finding of Celik and McDaniel's work [24] to train a Random Forest on a short snapshot of behaviour but benefiting from a recurrent neural network which has been trained by observing a longer window of behavioural data.…”

“…Instead of the full window of data the Random Forest will just see the most recent captured data (see Figures 1 and 2). This may seem counter-intuitive but Celik and McDaniel [24] found that in a number of security applications (including classifying malware into families) distillation could be used to train models with partial information when compared with the full models from which they are distilled. Instead of values between 0 and 1, the Random Forest will be trained to mimic the behaviour of the malware with the adjusted decision threshold to mitigate the chances of false positives.…”