Detecting Malware with Information Complexity

Abstract

Malware concealment is the predominant strategy for malware propagation. Black hats create variants of malware based on polymorphism and metamorphism. Malware variants, by definition, share some information. Although the concealment strategy alters this information, there are still patterns on the software. Given a zoo of labelled malware and benign-ware, we ask whether a suspect program is more similar to our malware or to our benign-ware. Normalized Compression Distance (NCD) is a generic metric that measures the shared information content of two strings. This measure opens a new front in the malware arms race, one where the countermeasures promise to be more costly for malware writers, who must now obfuscate patterns as strings qua strings, without reference to execution, in their variants. Our approach classifies disk-resident malware with 97.4% accuracy and a false positive rate of 3%. We demonstrate that its accuracy can be improved by combining NCD with the compressibility rates of executables using decision forests, paving the way for future improvements. We demonstrate that malware reported within a narrow time frame of a few days is more homogeneous than malware reported over two years, but that our method still classifies the latter with 95.2% accuracy and a 5% false positive rate. Due to its use of compression, the time and computation cost of our method is nontrivial. We show that simple approximation techniques can improve its running time by up to 63%. We compare our results to the results of applying the 59 anti-malware programs used on the VirusTotal website to our malware. Our approach outperforms each one used alone and matches that of all of them used collectively.

Malware Detection Using 1-Dimensional Convolutional Neural Networks

“…There have been techniques in the literature that utilize compression-based distance metrics to analyze malicious pieces of code. The work from [1] is based on the notion of normalized information distance (NID) and employ this as a distinguishing feature for training the classifier on.…”

“…For this work, we tried the implementation of another benchmark detector based on the information theoretic approach for malware detection, similar to the ones used by [1]. We found the computational limitation of such an approach during the attempted implementation.…”

“…Malware, or malicious software, is one of the primary threats in digital security. Large resources, both labour and monetary, are invested in anti-malware technologies to tackle this problem [1]. As the efficiency of the anti-malware technologies has improved, so has the sophistication, potency and domain of malware and their impact.…”

A Survey on Using Kolmogorov Complexity in Cybersecurity

“…The scalability to mount an attack on multiple institutions at once is not expensive compared to the reward. This leads to blackmail to companies/users, disclosure of files or credit card data in the web, or encryption of databases and files [ 9 , 12 , 33 , 34 , 35 , 36 , 37 ]. Identity/Authentication: With the exponential growth of wearables devices, users and devices will need to manage a new authentication mechanism to pair and interact with enhanced features, gathering information from the ECG (electrocardiogram) or similar, for example [ 38 , 39 , 40 , 41 , 42 ].…”

Nonnegative matrix factorization and metamorphic malware detection

“…However, their experimental results shown no better than 77% accuracy. Another compression based technique combined with machine learning by Alshahwan et al [3] where a random forest classifier was built using Normalized Compression Distance (NCD) and compressibility ratio features on binary files to detect malware.…”