Invalidating Analysis Knowledge for Code Virtualization Protection Through Partition Diversity

Abstract

To protect programs from unauthorized analysis, virtualize the code based on Virtual Machine (VM) technologies is emerging as a feasible method for accomplishing code obfuscation. However, in some State-of-the-art VM-based protection approaches, the set of virtual instructions and bytecode interpreters are fixed across the whole programs. This means an experienced attacker could extract the mapping information between virtual instructions and native code from programs, and use this knowledge to uncover the mapping relationships in similar protecting applications. To address this problem, we present CoDiver (Code Virtualization Protection with Diversity), a novel VM-based code obfuscation system in this paper. The main idea of our approach is to obfuscate the mapping between the opcodes of bytecode instructions and their semantics. To achieve this goal, we first turn every protected code region into multiple parts by partition proceeding, randomize the mapping of opcodes and their semantics of each part. By this way, we could translate the bytecode instruction into different native code in different sections of the obfuscated code. This method could increase the diversity of program behavior significantly. As a result, it will be useless to learn the mapping relationship between bytecode and native code of some other programs, then migrate it into a new program. We build a prototype of CoDiver and tested it on a set of real-world applications. Experimental results show that as compared with two state-of-the-art VM-based code obfuscation approaches, our approach is more effective and could provide stronger protection with comparable runtime overhead and code size. INDEX TERMS Instruction set randomization, reverse engineering, virtualized obfuscation.

Semantics-aware obfuscation scheme prediction for binary

“…According to effectiveness and efficiency, obfuscating transformations are classified with four categories, including layout obfuscation, data obfuscation, control obfuscation, and preventive obfuscation. So far, the research on code obfuscation algorithms has been very mature, and various algorithms have emerged endlessly [12], [29], [54], [13], [41], [56]. Table I shows eight typical code obfuscation algorithms, most of the existing algorithms are derived from them.…”

“…The second question, from a reverse engineering perspective, is: What kinds of obfuscation algorithm are employed on the target program? It is worth mentioning that new obfuscation algorithms [12], [29], [54], [13], [41], [56] and tools [48], [46], [50], [49], [25], [14] have emerged in an endless stream in recent years. Obviously, it is very ambitious for security analysts to understand the characteristics of each code obfuscation and quickly analyze which obfuscation algorithms the target program adopts.…”