Pop Quiz! Can a Large Language Model Help With Reverse Engineering?

Abstract

Large language models (such as OpenAI's Codex) have demonstrated impressive zero-shot multi-task capabilities in the software domain, including code explanation. In this work, we examine if this ability can be used to help with reverse engineering. Specifically, we investigate prompting Codex to identify the purpose, capabilities, and important variable names or values from code, even when the code is produced through decompilation. Alongside an examination of the model's responses in answering open-ended questions, we devise a true/false quiz framework to characterize the performance of the language model. We present an extensive quantitative analysis of the measured performance of the language model on a set of program purpose identification and information extraction tasks: of the 136,260 questions we posed, it answered 72,754 correctly. A key takeaway is that while promising, LLMs are not yet ready for zero-shot reverse engineering.

Efficient Ransomware Detection via Portable Executable File Image Analysis By LLaMA-7b

“…These methods enable the monitoring of ransomware in a real-time context, shedding light on the behavioral dynamics of ransomware during its active phases [25,26]. Alongside these developments, there has been a growing recognition of the potential that cloud-based solutions hold for ransomware detection [27]. By harnessing the power of distributed computing, these solutions offer a scalable and effective means of identifying and mitigating ransomware threats [28,29].…”

“…Such advancements could potentially lead to more sophisticated detection algorithms that are adept at navigating the complexities of modern ransomware, thereby enhancing overall cybersecurity resilience [9,25]. The integration of large language models in this sphere reflects a proactive approach in adapting to the sophisticated nature of modern cyber threats, offering a beacon of innovation in the ongoing efforts to safeguard digital ecosystems [26,27,45].…”

LLaMa Assisted Reverse Engineering of Modern Ransomware: A Comparative Analysis with Early Crypto-Ransomware

“…The integration of the LLaMa-12B model into the process of analyzing disassembled ransomware code has been a transformative step, markedly accelerating the identification of critical features within these malicious programs [41,42]. The capacity of LLaMa-12B to swiftly parse and interpret the complexities inherent in ransomware code has been instrumental in revealing sophisticated techniques employed by these digital threats [43,44].…”

Automatic Generation of Programming Exercises and Code Explanations with Large Language Models

“…Pearce et al Pearce et al (2021) analyzed the performance of Codex and similar models for repairing source code containing security flaws and found that through providing a carefully constructed prompt for the model, they were able to patch security issues in programs in some cases. Another study by Pearce et al Pearce et al (2022) analyzed the possibility of utilizing Codex for reverse engineering. In their study, they provided Codex decompiled code and prompted Codex to explain the purpose of the code.…”