Generalized just-in-time trace compilation using a parallel task farm in a dynamic binary translator

Böhm, Igor; Koch, Tobias J.K. Edler von; Kyle, Stephen C.; Franke, Björn; Topham, Nigel

doi:10.1145/1993316.1993508

Cited by 13 publications

(12 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hybrid interpreter/DBT systems [17] offload the expensive JIT compilation of work-units to threads [3], whilst still making forward progress in the interpreter, and thus hiding the latency of JIT compilation. Such set-ups (described as Asynchronous Mixed-mode Translation by [24]) have a greater scope for implementation, and raise questions such as what, when, and how guest code should be translated.…”

Section: Background and Motivating Examplementioning

confidence: 99%

“…Many VEEs, ranging from DBT systems [1,3,12] through language VMs [4,10,25] to specialized accelerator runtime environments [16], rely on JIT compilation as a key performance enabler. While some of these VEEs use JIT compilers specifically designed for this purpose, the LLVM [19] compiler framework finds increasing use in both academic projects, and commercial products as a JIT compiler.…”

Section: Introductionmentioning

confidence: 99%

“…As a way of hiding this JIT compilation latency, and to increase the throughput of the JIT compiler, parallel and concurrent JIT compilation task farms have been developed [3,18] and, later, adopted by industry, e.g. to drive data center software infrastructure like in HHVM [21], to improve the start-up times of managed application as in Microsoft's .Net Multicore JIT [7], or to power browser based applications like in Google's Chrome v8 JavaScript engine.…”

Section: Introductionmentioning

confidence: 99%

“…While effective, we show how this decoupled approach to JIT compilation introduces a new problem: Maintaining the right balance between the selection of code regions for JIT compilation, and scheduling regions for compilation is non-trivial. We show how state-of-the-art scheduling approaches [3,18] can arrive at non-optimal decisions resulting in poor application performance as hot code regions are compiled too late.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Mitigating JIT compilation latency in virtual execution environments

Kristien

Spink

Wagstaff

et al. 2019

Proceedings of the 15th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments

Self Cite

View full text Add to dashboard Cite

Many Virtual Execution Environments (VEEs) rely on Justin-time (JIT) compilation technology for code generation at runtime, e.g. in Dynamic Binary Translation (DBT) systems or language Virtual Machines (VMs). While JIT compilation improves native execution performance as opposed to e.g. interpretive execution, the JIT compilation process itself introduces latency. In fact, for highly optimizing JIT compilers or compilers not specifically designed for JIT compilation, e.g. LLVM, this latency can cause a substantial overhead. While existing work has introduced asynchronously decoupled JIT compilation task farms to hide this JIT compilation latency, we show that this on its own is not sufficient to mitigate the impact of JIT compilation latency on overall performance. In this paper, we introduce a novel JIT compilation scheduling policy, which performs continuous low-cost profiling of code regions already dispatched for JIT compilation, right up to the point where compilation commences. We have integrated our novel JIT compilation scheduling approach into a commercial LLVM-based DBT system and demonstrate speedups of 1.32× on average, and up to 2.31×, over its state-of-the-art concurrent task-farm based JIT compilation scheme across the SPEC CPU2006 and BioPerf benchmark suites.CCS Concepts • Hardware → Simulation and emulation; • Software and its engineering → Simulator / interpreter; Just-in-time compilers.

show abstract

Section: Background and Motivating Examplementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Mitigating JIT compilation latency in virtual execution environments

Kristien

Spink

Wagstaff

et al. 2019

Proceedings of the 15th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments

Self Cite

View full text Add to dashboard Cite

show abstract

“…Simulation methods, such as memory and instruction set simulation, are powerful tools to design and evaluate such systems. A large number of simulation tools are available [15]; in this paper we further improve upon current tools by introducing novel 'functional' and 'detailed' hardware simulation packages, that can simulate individual cores and also complete CPU/GPU systems (Section IV-A). Also novel profiling (Section IV-B) and specialisation (Section IV-C) techniques are introduced which allow us to custom-design chips for SLAM and computer vision applications.…”

Section: A Backgroundmentioning

confidence: 99%

Navigating the Landscape for Real-Time Localization and Mapping for Robotics and Virtual and Augmented Reality

et al. 2018

View full text Add to dashboard Cite

Visual understanding of 3D environments in realtime, at low power, is a huge computational challenge. Often referred to as SLAM (Simultaneous Localisation and Mapping), it is central to applications spanning domestic and industrial robotics, autonomous vehicles, virtual and augmented reality. This paper describes the results of a major research effort to assemble the algorithms, architectures, tools, and systems software needed to enable delivery of SLAM, by supporting applications specialists in selecting and configuring the appropriate algorithm and the appropriate hardware, and compilation pathway, to meet their performance, accuracy, and energy consumption goals. The major contributions we present are (1) tools and methodology for systematic quantitative evaluation of SLAM algorithms, (2) automated, machine-learning-guided exploration of the algorithmic and implementation design space with respect to multiple objectives, (3) end-to-end simulation tools to enable optimisation of heterogeneous, accelerated architectures for the specific algorithmic requirements of the various SLAM algorithmic approaches, and (4) tools for delivering, where appropriate, accelerated, adaptive SLAM solutions in a managed, JIT-compiled, adaptive runtime context. Performance EvaluationRuntime Architecture Compiler and Algorithm Design Space Exploration -Machine Learning Fig. 1: The objective of the paper is to create a pipeline that aligns computer vision requirements with hardware capabilities. The paper's focus is on three layers: algorithms, compiler and runtime, and architecture. The goal is to develop a system that allows us to achieve power and energy efficiency, speed and runtime improvement, and accuracy/robustness at each layer and also holistically through design space exploration and machine learning techniques.

show abstract