Continuous Learning for Android Malware Detection

Abstract

Machine learning methods can detect Android malware with very high accuracy. However, these classifiers have an Achilles heel, concept drift: they rapidly become out of date and ineffective, due to the evolution of malware apps and benign apps. Our research finds that, after training an Android malware classifier on one year's worth of data, the F1 score quickly dropped from 0.99 to 0.76 after 6 months of deployment on new test samples.In this paper, we propose new methods to combat the concept drift problem of Android malware classifiers. Since machine learning technique needs to be continuously deployed, we use active learning: we select new samples for analysts to label, and then add the labeled samples to the training set to retrain the classifier. Our key idea is, similarity-based uncertainty is more robust against concept drift. Therefore, we combine contrastive learning with active learning. We propose a new hierarchical contrastive learning scheme, and a new sample selection technique to continuously train the Android malware classifier. Our evaluation shows that this leads to significant improvements, compared to previously published methods for active learning. Our approach reduces the false negative rate from 16% (for the best baseline) to 10%, while maintaining the same false positive rate (0.6%). Also, our approach maintains more consistent performance across a seven-year time period than past methods.

Drift Forensics of Malware Classifiers

“…Our current framework relies on the availability of ground truth labels for individual families every month. This is similar to Chen et al [13] where they assume a monthly labeling budget for active learning. Although this is an appropriate assumption for a post-hoc forensics framework, ground truth labels are scarce in real-world deployments.…”

“…In our settings, drift was mostly caused by a malware family (although which one or whether more were present is irrelevant to the problem at hand). This is either because of the actual evolution of malware behaviors, imprecise abstractions and representations of the datasets, or a combination thereof, which calls for further research on drift detection [11,13,25,35,47] and programs' representations.…”

“…Follow-up research thus proposed various approaches to alleviate the impact of concept drift on the classifier's performance, including methods for drift detection [e.g., 11,22,23] and online learning [13,33,35,45]. While these methods compensate for the performance drop to some extent, none of them aims to analyze the root causes of the underlying drift.…”

Addressing Behavioral Drift in Ransomware Early Detection Through Weighted Generative Adversarial Networks

“…In addition to the considered metrics, a comparison of the False Positive Rate and Detection rate for the proposed work is also used to represent the efficacy of the proposed technique. A False Positive Rate (FPR) also known as Fall-Out is the measurement of accuracy and calculated with equation (17). False Positive Rate is defined as a ratio of false positive with a sum of false positive and true negative.…”

“…The behavioral drift is defined in terms of the development of new attack variants with changing behavior over time [15,16]. Behavioral drift is the basis for zero-day attacks [17], by changing the attack attributes (which are also called features) [6,14,18]. For developing variants, changing distribution of features changes the significance of features accordingly.…”