Revisiting Binary Code Similarity Analysis Using Interpretable Feature Engineering and Lessons Learned

Abstract

Binary code similarity analysis (BCSA) is widely used for diverse security applications, including plagiarism detection, software license violation detection, and vulnerability discovery. Despite the surging research interest in BCSA, it is significantly challenging to perform new research in this field for several reasons. First, most existing approaches focus only on the end results, namely, increasing the success rate of BCSA, by adopting uninterpretable machine learning. Moreover, they utilize their own benchmark, sharing neither the source code nor the entire dataset. Finally, researchers often use different terminologies or even use the same technique without citing the previous literature properly, which makes it difficult to reproduce or extend previous work. To address these problems, we take a step back from the mainstream and contemplate fundamental research questions for BCSA. Why does a certain technique or a certain feature show better results than the others? Specifically, we conduct the first systematic study on the basic features used in BCSA by leveraging interpretable feature engineering on a large-scale benchmark. Our study reveals various useful insights on BCSA. For example, we show that a simple interpretable model with a few basic features can achieve a comparable result to that of recent deep learning-based approaches. Furthermore, we show that the way we compile binaries or the correctness of underlying binary analysis tools can significantly affect the performance of BCSA. Lastly, we make all our source code and benchmark public and suggest future directions in this field to help further research.

Detecting Obfuscated Function Clones in Binaries using Machine Learning

“…A review of related approaches has shown that binary similarity has profited from machine learning and that usage of machine learning is viable for deobfuscation tasks. Out of recent state-of-the-art function clone detection approaches, only some approaches deal with obfuscation [47], [13], [30], [34]. None of these approaches have tried to apply binary code similarity algorithms to counter obfuscation by virtualization.…”

“…Research dealing with virtualization in the context of function clone detection is scarce. A recent survey [34] highlights the need for covering interprocedural virtualization obfuscators like Themida [57], or VMProtect [61], as the obfuscations applied by Hikari [64] do not appear to be more complex than cross-optimization binary similarity. OFCI does not solve interprocedural obfuscation either, but is intended as a stepping stone to expand research in this area.…”

“…ASM2VEC [13], SAFE [39], GEMINI [63] and BLEX [15], they compare results with different metrics, because with the exception of SAFE, none of the other approaches provides the full source-code or trained model. While there is recent work trying to tackle this issue [34]…”

Multi-feature based Function Embedding Network for Binary Code Similarity

“…Second, the function signature of a program is an important feature for representing the function semantics and able to improve the performance of binary code similarity detection [22], however, previous works didn't design methods to extract the information in function signatures.…”

The Inconvenient Truths of Ground Truth for Binary Analysis

“…Li et al [10] grab intermediate information from the compiler to generate their disassembly ground truth, in an attempt to validate ground truth. Another useful paper is by Kim et al [8] that discusses a benchmark for binary code similarity analysis.…”