Malware Pattern Scanning Schemes Secure Against Black-box Analysis

Abstract

As a general rule, copycats produce most of malware variants from an original malware strain. For this purpose, they widely perform black-box analyses of commercial scanners aiming at extracting malware detection patterns. In this paper, we first study the malware detection pattern extraction problem from a complexity point of view and provide the results of a wide-scale study of commercial scanners' black-box analysis. These results clearly show that most of the tested commercial products fail to thwart black-box analysis. Such weaknesses therefore urge copycats to produce even more malware variants. Then, we present a new model of malware detection pattern based on Boolean functions and identify some properties that a reliable detection pattern should have. Lastly, we describe a combinatorial, probabilistic malware pattern scanning scheme that, on the one hand, highly limits black-box analysis and on the other hand can only be bypassed in the case where there is collusion between a number of copycats. This scheme can incidentally provide some useful technical information to malware crime investigators, thus allowing a faster identification of copycats.

Pinpointing Malicious Activities through Network and System-Level Malware Execution Behavior

“…Their work presents a strong mathematical basis to define different types of behavior and is an extension of a previous work that was only able to handle sequences of bytes [8]. The limiting factor is that their new approach requires the malware's source code, which is sometimes difficult to obtain.…”

Auto-Sign: an automatic signature generator for high-speed malware filtering devices

“…Filiol [26] address the problem that commercially available anti-viruses are not resistant against black-box analysis. He suggested generating multiple sub-signatures that are randomly selected from a longer signature.…”

“…Various techniques have been proposed to derive malware signatures automatically, including among others: vulnerability-based signatures [1]; payload-based signatures ( [7]; [19]); content sifting [17]; semantic-aware signatures [21]; The Amd algorithm [2]; Honeypot-based signatures ([8]; [15]; [18]), and polymorphic content-based signatures [14] [26]. These studies examine code by matching and analyzing the distribution of string patterns in communication packets; classifying unsuccessful connections; and modeling invariant code structures.…”

Unknown malcode detection and the imbalance problem

“…Additionally, it was found that above 15% of the files in the KaZaA network contained malicious code 2 . Thus, we assume that the percentage of malicious files in real life is about or less than 10%, but we also consider other possible percentages.…”