Network-level polymorphic shellcode detection using emulation

Abstract

Significant progress has been made in recent years towards preventing code injection attacks at the network level. However, as state-of-the-art attack detection technology becomes more prevalent, attackers are likely to evolve, employing techniques such as polymorphism and metamorphism to defeat these defenses. A major outstanding question in security research and engineering is thus whether we can proactively develop the tools needed to contain advanced polymorphic and metamorphic attacks. While recent results have been promising, most of the existing proposals can be defeated using only minor enhancements to the attack vector. In fact, some publicly-available polymorphic shellcode engines are currently one step ahead of the most advanced publicly-documented network-level detectors. In this paper, we present a heuristic detection method that scans network traffic streams for the presence of previously unknown polymorphic shellcode. In contrast to previous work, our approach relies on a NIDSembedded CPU emulator that executes every potential instruction sequence in the inspected traffic, aiming to identify the execution behavior of polymorphic shellcode. Our analysis demonstrates that the proposed

Defending Browsers against Drive-by Downloads: Mitigating Heap-Spraying Code Injection Attacks

“…We currently use a value of 32 bytes as the threshold for the minimal length of a shellcode sequence. We found that this value works well in our experiments, and it is also significantly shorter than all Windows shellcode encountered in the wild [26].…”

“…More precisely, their prototype implementation identifies long valid sequences of instructions in HTTP requests, thus detecting the NOP sledge that commonly accompanies shellcode. Continuing this work, Polychronakis et al [24,26] proposed to apply lightweight emulation on network data to identify polymorphic shellcode. This approach relies on the so-called GetPC heuristic.…”

Efficient Shellcode Detection on Commodity Hardware

“…In the past few years, many detection approaches [1], [3], [4], [6], [9] have been proposed. Basically, these methods can be divided into two categories: static analysis [3], [4] and dynamic analysis [1], [6], [9].…”

“…Basically, these methods can be divided into two categories: static analysis [3], [4] and dynamic analysis [1], [6], [9]. The core idea of static analysis is to disassemble the network stream and then analyze the code-level patterns that could be signatures obtained from existing shellcode.…”

“…However, it is limited in detecting shellcode that employs binary obfuscation techniques. To address this problem, Polychronakis et al propose the dynamic methods based on the network-level emulation [1]. This method first disassembles network data into several execution chains and then emulates these chains.…”