Generalized just-in-time trace compilation using a parallel task farm in a dynamic binary translator

Abstract

Dynamic Binary Translation (DBT) is the key technology behind cross-platform virtualization and allows software compiled for one Instruction Set Architecture (ISA) to be executed on a processor supporting a different ISA. Under the hood, DBT is typically implemented using Just-In-Time (JIT) compilation of frequently executed program regions, also called traces. The main challenge is translating frequently executed program regions as fast as possible into highly efficient native code. As time for JIT compilation adds to the overall execution time, the JIT compiler is often decoupled and operates in a separate thread independent from the main simulation loop to reduce the overhead of JIT compilation. In this paper we present two innovative contributions. The first contribution is a generalized trace compilation approach that considers all frequently executed paths in a program for JIT compilation, as opposed to previous approaches where trace compilation is restricted to paths through loops. The second contribution reduces JIT compilation cost by compiling several hot traces in a concurrent task farm. Altogether we combine generalized light-weight tracing, large translation units, parallel JIT compilation and dynamic work scheduling to ensure timely and efficient processing of hot traces. We have evaluated our industry-strength, LLVM-based parallel DBT implementing the ARCompact ISA against three benchmark suites (EEMBC, BIOPERF and SPEC CPU2006) and demonstrate speedups of up to 2.08 on a standard quad-core Intel Xeon machine. Across short-and long-running benchmarks our scheme is robust and never results in a slowdown. In fact, using four processors total execution time can be reduced by on average 11.5% over state-of-the-art decoupled, parallel (or asynchronous) JIT compilation.

Finding the limit

“…There have been explorations on using concurrent JIT to enhance the performance of virtual machines [7,15,22]. They have not systematically explored the influence of the order of compilation events.…”

Exploring single and multilevel JIT compilation policy for modern machines 1

“…Existing JVMs, such as Sun's HotSpot server VM [Paleczny et al 2001] and the Azul VM (derived from HotSpot), support multiple compiler threads but do not present any discussions on ideal compilation strategies for multicore machines. Prior work by Böhm et al explores the issue of parallel JIT compilation with a priority queue-based dynamic work scheduling strategy in the context of their dynamic binary translator [Böhm et al 2011]. Esmaeilzadeh et al study the scalability of various Java workloads and their power/performance tradeoffs across several different architectures [Esmaeilzadeh et al 2011].…”

Efficiently parallelizing instruction set simulation of embedded multi-core processors using region-based just-in-time dynamic binary translation

“…loop header, method entry) [2,15,17,19,36], or always enabled when interpreting code [4]. Various backbone data structures have been suggested to capture recorded traces such as trace-trees [14], control-flow-graphs (CFG) of traced basic blocks [4], or hybrids between trace-trees and CFGs called trace-regions [2] or trace-graphs [18]. In general, recording approaches for multi-threaded applications can be categorized based on how CFG recordings are accessed and constructed:…”

“…At the end of each interval frequently executed regions are dispatched to a region translation priority queue, before continuing to interpret the program. A detailed description of the priority function used to order regions can be found in [4]. JIT workers operating in parallel threads can then claim a region from the queue, JIT compile it, and update the region translation state, thereby indicating the availability of native code for that recorded region to the interpreter loop.…”