An Adaptive Behavioral-Based Incremental Batch Learning Malware Variants Detection Model Using Concept Drift Detection and Sequential Deep Learning

Abstract

Malware variants are the major emerging threats that face cybersecurity due to the potential damage to computer systems. Many solutions have been proposed for detecting malware variants. However, accurate detection is challenging due to the constantly evolving nature of the malware variants that cause concept drift. Existing malware detection solutions assume that the mapping learned from historical malware features will be valid for new and future malware. The relationship between input features and the class label has been considered stationary, which doesn't hold for the ever-evolving nature of malware variants. Malware features change dynamically due to code obfuscations, mutations, and the modification made by malware authors to change the features' distribution and thus evade the detection rendering the detection model obsolete and ineffective. This study presents an Adaptive behavioral-based Incremental Batch Learning Malware Variants Detection model using concept drift detection and sequential deep learning (AIBL-MVD) to accommodate the new malware variants. Malware behavior were extracted using dynamic analysis by running the malware files in a sandbox environment and collecting their Application Programming Interface (API) traces. According to the malware first-time appearance, the malware samples were sorted to capture the malware variants' change characteristics. The base classifier was then trained based on a subset of historical malware samples using a sequential deep learning model. The new malware samples were mixed with a subset of old data and gradually introduced to the learning model in an adaptive batch size incremental learning manner to address the catastrophic forgetting dilemma of the incremental learning. The statistical process control technique has been used to detect the concept drift as indication for incrementally updating the model as well as reducing the frequency of model updates. Results from extensive experiments show that the proposed model is superior in terms of detection rate and the efficiency compared with the static model, periodic retraining approaches, and the fixed batch size incremental learning approach. The model maintains an average of 99.41% detection accuracy of new and variants malware with a low updating frequency of 1.35 times per month.

Evaluation of Machine Learning Algorithms for Malware Detection

“…After generating a malware and cleanware dataset, it is put to use in testing. We analysed malware and placed it into different groups using a number of supervised machine learning methods, such as kNN, DT, RF, AdaBoost, SGD, extra trees and the Gaussian NB Classifier [ 22 , 25 ].…”

“…To examine Ye’s (2017) material would be an enormous task. All of these things make it difficult to develop a malware detection system that uses machine learning in real time [ 22 ].…”

“…Included were more than 198,063 separate data points gathered from a wide variety of sources. There were 373 rows in the data and 531 columns as shown in Table 1 [22][23][24][25]. column that indicates whether or not the file is harmful.…”

A Weighted Minimum Redundancy Maximum Relevance Technique for Ransomware Early Detection in Industrial IoT

“…Like static data, dynamic data can be introspected, and malicious patterns can be extracted [45]. Due to its efficacy for countering the sophisticated ransomware families that employ polymorphic techniques to deceive detection, the dynamic analysis gained popularity in the research community [23,41,42,[46][47][48][49][50]. During the dynamic analysis, several types of data are collected, including API calls, PE contents, and file systems; memory; CPU and I/O statistics [2].…”

ACMFNN: Design of an augmented convolutional model for intelligent cross-domain malware localization via forensic neural networks

“…Similar methods that utilize Learning-Based Generative Model for PDF files [16], Concept Drift Detection with Sequential Deep Learning (CDS SDL) for batch malwares [17], Fuzzified Features with Boosted Fuzzy Random Forest (FBRF) [18], and RNN for visualization of malwares [19] are discussed by researchers. These models aim at improving inter-class feature variance via rigorous analysis of extracted features in order to identify malwarespecific models that can be deployed for on-field use cases.…”