V2e

Abstract

A transparent and extensible malware analysis platform is essential for defeating malware. This platform should be transparent so malware cannot easily detect and bypass it. It should also be extensible to provide strong support for heavyweight instrumentation and analysis efficiency. However, no existing platform can meet both requirements. Leveraging hardware virtualization technology, analysis platforms like Ether can achieve good transparency, but its instrumentation support and analysis efficiency are weak. In contrast, software emulation provides strong support for code instrumentation and good analysis efficiency by using dynamic binary translation. However, analysis platforms based on software emulation can be easily detected by malware and thus is poor in transparency. To achieve both transparency and extensibility, we propose a new analysis platform that combines hardware virtualization and software emulation. The essence is precise heterogeneous replay: the malware execution is recorded via hardware virtualization and then replayed in software. Our design ensures the execution replay to be precise. Moreover, with page-level recording granularity, the platform can easily adjust to analyze various forms of malware (a process, a kernel module, or a shared library). We implemented a prototype called V2E and demonstrated its capability and efficiency by conducting an extensive evaluation with both synthetic samples and 14 realworld emulation-resistant malware samples.

Using Hardware Features for Increased Debugging Transparency

“…They focus on understanding malware behaviors, instead of achieving better transparency. V2E [7] combines both hardware virtualization and software emulation. HyperDbg [6] uses the hardware virtualization that allows the late launching of VMX modes to install a virtual machine monitor and run the analysis code in the VMX root mode.…”

“…The base address of the IDT will be different in a VM than on a bare-metal machine because there is only one IDT register shared by both host-OS and guest-OS. Additionally, QEMU can be detected by accessing a reserved Model Specific Register (MSR) [7]. This invalid access causes a General Protection (GP) exception on a bare-metal machine, but QEMU does not.…”

“…In addition, many packers contain anti-debugging and anti-VM features, further increasing the challenge of reverseengineering packed malware. [12] API Call Kernel32!IsDebuggerPresent returns 1 if a target process is being debugged ntdll!NtQueryInformationProcess: ProcessInformation field set to -1 if the process is being debugged kernel32!CheckRemoteDebuggerPresent returns 1 in debugger process NtSetInformationThread with ThreadInformationClass set to 0x11 will detach some debuggers kernel32!DebugActiveProcess to prevent other debuggers from attaching to a process PEB Field PEB!IsDebugged is set by the system when a process is debugged PEB!NtGlobalFlags is set if the process was created by a debugger Detection ForceFlag field in heap header (+0x10) can be used to detect some debuggers UnhandledExceptionFilter calls a user-defined filter function, but terminates in a debugging process TEB of a debugged process contains a NULL pointer if no debugger is attached; valid pointer if some debuggers are attached Ctrl-C raises an exception in a debugged process, but the signal handler is called without debugging Inserting a Rogue INT3 opcode can masquerade as breakpoints Trap flag register manipulation to thwart tracers If entryPoint RVA is set to 0, the magic MZ value in PE files is erased ZwClose system call with invalid parameters can raise an exception in an attached debugger Direct context modification to confuse a debugger 0x2D interrupt causes debugged program to stop raising exceptions Some In-circuit Emulators (ICEs) can be detected by observing the behavior of the undocumented 0xF1 instruction Searching for 0xCC instructions in program memory to detect software breakpoints TLS-callback to perform checks [17] Invalid instruction behavior [14] Using memory deduplication to detect various hypervisors including VMware ESX server, Xen, and Linux KVM [56] Anti-emulation Bochs Visible debug port [10] QEMU cpuid returns less specific information [7] Accessing reserved MSR registers raises a General Protection (GP) exception in real hardware; QEMU does not [15] Attempting to execute an instruction longer than 15 bytes raises a GP exception in real hardware; QEMU does not [15] Undocumented icebp instruction hangs in QEMU [7], while real hardware raises an exception Unaligned memory references raise exceptions in real hardware; unsupported by QEMU [15] Bit 3 of FPU Control World register is always 1 in real hardware, while QEMU contains a 0 [7] Other Using CPU bugs or errata to create CPU fingerprints via public chipset documentation [15] To demonstrate the transparency of MALT, we use popular packing tools to pack the Notepad.exe application in a Windows environment and run this packed application in MALT with near return stepping mode, OllyDbg [31], DynamoRIO [57], and a Windows virtual machine, respectively. Ten packing tools are used, including UPX, Obsidium, ASPack, Armadillo, Themida, RLPack, PELock, VMProtect, eXPressor, and PECompact.…”

“…Traditional malware analysis employs virtualization [4], [5], [6] and emulation [7], [8], [9] technologies to dissect malware behavior at runtime. This approach runs the malware in a Virtual Machine (VM) or emulator and uses an analysis program to introspect the malware from the outside so that the malware cannot infect the analysis program.…”

“…Thus, it has a smaller Trusted Code Base (TCB) than hypervisor-based debugging systems [4], [7], [8], [9], which significantly reduces the attack surface of MALT. Moreover, MALT is OS-agnostic and immune to hypervisor attacks (e.g., VM-escape attacks [20], [21]).…”