Machine Learning for Malware Detection: Beyond Accuracy Rates

Abstract

Today's world is supported by connected, electronic systems, thus ensuring their secure operation is essential to our daily lives. A major threat to system's security is malware infections, which cause financial and image losses to corporate and end-users, thus motivating the development of malware detectors. In this scenario, Machine Learning (ML) has been demonstrated to be a powerful technique to develop classifiers able to distinguish malware from goodware samples. However, many ML research work on malware detection focus only on the final detection accuracy rate and overlook other important aspects of classifier's implementation and evaluation, such as feature extraction and parameter selection. In this paper, we shed light to these aspects to highlight the challenges and drawbacks of ML-based malware classifiers development. We trained 25 distinct classification models and applied them to 2,800 real x86, Linux ELF malware binaries. Our results shows that: (i) dynamic features outperforms static features when the same classifiers are considered; (ii) Discrete-bounded features present smaller accuracy variance over time in comparison to continuous features, at the cost of some time-localized accuracy loss; (iii) Datasets presenting distinct characteristics (e.g., temporal changes) impose generalization challenges to ML models; and (iv) Feature analysis can be used as feedback information for malware detection and infection prevention. We expect that our work could help other researchers when developing their ML-based malware classification solutions.

Machine Learning (In) Security: A Stream of Problems

“…Therefore, the selection of the attribute extraction procedure should consider its effect on outcome. Galante et al [54] shows the effect of attribute extraction procedures over three distinct pair of models to detect malware (SVM with RBF kernel, Multi-layer Perceptron, and Random Forest). The three models consider the same feature, but extracting them dynamically and statically, respectively.…”