A semantics-based approach to malware detection

Abstract

Malware detection is a crucial aspect of software security. Current malware detectors work by checking for signatures , which attempt to capture the syntactic characteristics of the machine-level byte sequence of the malware. This reliance on a syntactic approach makes current detectors vulnerable to code obfuscations, increasingly used by malware writers, that alter the syntactic properties of the malware byte sequence without significantly affecting their execution behavior. This paper takes the position that the key to malware identification lies in their semantics. It proposes a semantics-based framework for reasoning about malware detectors and proving properties such as soundness and completeness of these detectors. Our approach uses a trace semantics to characterize the behavior of malware as well as that of the program being checked for infection, and uses abstract interpretation to “hide” irrelevant aspects of these behaviors. As a concrete application of our approach, we show that (1) standard signature matching detection schemes are generally sound but not complete, (2) the semantics-aware malware detector proposed by Christodorescu et al. is complete with respect to a number of common obfuscations used by malware writers and (3) the malware detection scheme proposed by Kinder et al. and based on standard model-checking techniques is sound in general and complete on some, but not all, obfuscations handled by the semantics-aware malware detector.

Modelling Metamorphism by Abstract Interpretation

“…In [13] the authors use trace semantics to characterize the behaviours of both the malware and the potentially infected program, and use abstract interpretation to "hide" their irrelevant behaviours. A program is infected by a malware if their behaviours are indistinguishable up to a certain abstraction, which corresponds to some obfuscations.…”

“…The key for identifying metamorphic malware lies, instead, in a deeper understanding of their semantics. Still a major drawback of existing semantics-based methods (e.g., see [13,19]) relies upon the a priori knowledge of the obfuscations used to implement the metamorphic engine. Because of this, it is always possible for any expert malware writer to develop alternative metamorphic strategies, even by simple modification of existing ones, able to foil any given detection scheme.…”

Partial Evaluation for Java Malware Detection

“…Program transformation has been proposed for deobfuscating binary programs [5], by unpacking and removing superfluous jumps and junk, again with the aim of improving AV scanning. This suggest that partial evaluation also has a role in binary analysis, where the aim is to make malware detection more semantic [7].…”

SA3: Automatic Semantic Aware Attribution Analysis of Remote Exploits

“…Christodorescu et al [30] present a dependency-graph based approach to mining the malicious behaviors present in a known malware that are not present in a set of benign programs, which can be used by malware detectors to detect malware variants. Also, Christodorescu et al [31] use a trace semantics to characterize the behaviors of malware as well as the program being checked for infection, and use abstract interpretation to "hide" irrelevant aspects of these behaviors for malware detection/classification. The motivation of our work is very similar to these works, but ours is specific to exploit code category analysis, and more importantly, we present a novel approach for attribution analysis which combines the semantic analysis with statistical analysis.…”