Polymorphic malware detection using sequence classification methods and ensembles

Abstract

Identifying malicious software executables is made difficult by the constant adaptations introduced by miscreants in order to evade detection by antivirus software. Such changes are akin to mutations in biological sequences. Recently, high-throughput methods for gene sequence classification have been developed by the bioinformatics and computational biology communities. In this paper, we apply methods designed for gene sequencing to detect malware in a manner robust to attacker adaptations. Whereas most gene classification tools are optimized for and restricted to an alphabet of four letters (nucleic acids), we have selected the Strand gene sequence classifier for malware classification. Strand's design can easily accommodate unstructured data with any alphabet, including source code or compiled machine code. To demonstrate that gene sequence classification tools are suitable for classifying malware, we apply Strand to approximately 500 GB of malware data provided by the Kaggle Microsoft Malware Classification Challenge (BIG 2015) used for predicting nine classes of polymorphic malware. Experiments show that, with minimal adaptation, the method achieves accuracy levels well above 95% requiring only a fraction of the training times used by the winning team's method.

BagMinHash - Minwise Hashing Algorithm for Weighted Sets

“…Algorithm 1 is a straightforward implementation of signature de nition (9). First, the signature components are initialized with in nity.…”

“…In the oncoming sections we will describe methods to make the calculation of the new signature much more e cient. e signature components de ned by (9) are all nonnegative real numbers. However, as mentioned in Section 1.1, integer values with a prede ned number of bits are o en more preferable.…”

“…Poisson processes have some nice properties. Points can be generated in ascending order, because distances between Algorithm 1: Straightforward implementation of signature de nition (9).…”

Locality sensitive hashing for the edit distance

“…Even though the Jaccard and minHash sketches are regularly used as a measure of the k-mer content similarity in computational biology software, the weighted Jaccard similarity has been heavily studied and used in other contexts, such as large database document classification and retrieval (e.g., Manasse et al, 2010;Shrivastava, 2016;Wu et al, 2017), near duplicate image detection (Chum et al, 2008), duplicate news story detection (Alonso et al, 2013), source code deduplication (Markovtsev and Kant, 2017), time series indexing (Luo and Shrivastava, 2017), hierarchical topic extraction (Gollapudi and Panigrahy, 2006), or malware classifcation (Drew et al, 2017) and detection (Raff and Nicholas, 2017).…”

Detection of Advanced Malware by Machine Learning Techniques

“…Recently some authors worked on malware dataset released for kaggle dataset [14]. In the year 2016, Ahmadi et al Drew et al [16] used The Super Threaded Reference Free Alignment-Free Nsequence Decoder (STRAND) classifier to perform classification of polymorphic malware. In their approach, they presented ASM sequence model and obtained accuracy greater than 98.59 % using 10-fold cross-validation.…”