Fine-grained static detection of obfuscation transforms using ensemble-learning and semantic reasoning

Abstract

e ability to e ciently detect the so ware protections used is at a prime to facilitate the selection and application of adequate deobfuscation techniques. We present a novel approach that combines semantic reasoning techniques with ensemble learning classi cation for the purpose of providing a static detection framework for obfuscation transformations. By contrast to existing work, we provide a methodology that can detect multiple layers of obfuscation, without depending on knowledge of the underlying functionality of the training-set used. We also extend our work to detect constructions of obfuscation transformations, thus providing a ne-grained methodology. To that end, we provide several studies for the best practices of the use of machine learning techniques for a scalable and e cient model. According to our experimental results and evaluations on obfuscators such as Tigress and OLLVM, our models have up to 91% accuracy on state-of-the-art obfuscation transformations. Our overall accuracies for their constructions are up to 100%.

Input-Output Example-Guided Data Deobfuscation on Binary

“…Even though obfuscation can hide semantics very well, there can still be some hints left. Existing obfuscation detection work has high accuracy [13,[33][34][35][36][37]. us, we are motivated to employ a classifier to detect which basic block is obfuscated.…”

“…But they do not discuss locating the obfuscated code snippet. Tofighi et al [33] present a fine-grained detection framework of obfuscation transformations and constructions. Compared with this work, the same thing is that both of us consider the locating of obfuscated code snippets.…”

DFSGraph: Data Flow Semantic Model for Intermediate Representation Programs Based on Graph Network

“…In traditional binary code analysis methods, most researchers use symbolic execution to combat code deobfuscation. For instance, Tofighi-Shirazi et al [1] proposed DoSE, which can improve the deobfuscation technique based on dynamic symbolic execution by statically eliminating obfuscation transformations, and remove two-way opaque constructs by semantic equivalence. While Xu et al [2] adopted the multi-granularity symbolic execution method to simplify the trace snippets for partially virtualized binary code, and achieved encouraging experimental results at that time.…”