Cross-layer workload characterization of meta-tracing JIT VMs

Ilbeyi, Berkin; Bolz-Tereick, Carl Friedrich; Batten, Christopher

doi:10.1109/iiswc.2017.8167760

Cited by 4 publications

(4 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, there is at least 2.8x increase in execution time on average moving from a C-like program to a Python program running on CPython due to language and interpreter overheads. This observation is inline with corresponding results reported in the existing literature [30].…”

Section: Listing 2: Redundant Code Removal Code Snippetsupporting

confidence: 93%

“…C/C++ Memory Management [13, 14, 15] [29, 16, 17, 18] Qualitative/Quantitative Analysis [19,20,30 reducing the performance and memory overhead of loading, traversing, and manipulating tree data structures in Python applications. The authors of [22], propose a Python-based optimization framework for high-performance genomics, that combines the advantages of high-level languages such as Python with the performance of lower-level languages such as C/C++.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A memory footprint optimization framework for Python applications targeting edge devices

Katsaragakis

Papadopoulos

Konijnenburg

et al. 2023

Journal of Systems Architecture

View full text Add to dashboard Cite

Section: Listing 2: Redundant Code Removal Code Snippetsupporting

confidence: 93%

Section: Related Workmentioning

confidence: 99%

A memory footprint optimization framework for Python applications targeting edge devices

Katsaragakis

Papadopoulos

Konijnenburg

et al. 2023

Journal of Systems Architecture

View full text Add to dashboard Cite

“…Ilbeyi et al [14] propose a cross-layer performance evaluation methodology to analyze meta-tracing JIT performance at the application, framework, interpreter, JIT compiler and microarchitecture levels. Meta-tracing JIT frameworks decouple the language definition from the VM internals to reduce the effort for developing custom JIT compilers.…”

Section: Related Workmentioning

confidence: 99%

“…Python is a particularly popular scripting language: it is the most popular scripting language and the third most popular programming language after C and Java, according to the TIOBE index as of July 2020. 1 Unfortunately, rigorous benchmarking and performance analysis methodologies are lacking for scripting language workloads, and different researchers and practitioners use different ad-hoc methodologies [14], [15], [21]- [23]: some seem to measure start-up performance while others seem to measure steady-state performance (without explicitly stating so), and most seem to lack statistical data analysis. In particular, the PyPy website 2 mentions that 'on average, the PyPy Just-in-Time (JIT) compiler is 4.4 times faster than the CPython interpreter'.…”

Section: Introductionmentioning

confidence: 99%

A Rigorous Benchmarking and Performance Analysis Methodology for Python Workloads

Crape

Eeckhout

2020

2020 IEEE International Symposium on Workload Characterization (IISWC)

View full text Add to dashboard Cite

Computer architecture and computer systems research and development is heavily driven by benchmarking and performance analysis. It is thus of paramount importance that rigorous methodologies are used to draw correct conclusions and steer research and development in the right direction. While rigorous methodologies are widely used for native and managed programming language workloads, scripting language workloads are subject to ad-hoc methodologies which lead to incorrect and misleading conclusions. In particular, we find incorrect public statements regarding different virtual machines for Python, the most popular scripting language. The incorrect conclusion is a result of using the geometric mean speedup and not making a distinction between start-up and steady-state performance. In this paper, we propose a statistically rigorous benchmarking and performance analysis methodology for Python workloads, which makes a distinction between start-up and steady-state performance and which summarizes average performance across a set of benchmarks using the harmonic mean speedup. We find that a rigorous methodology makes a difference in practice. In particular, we find that the PyPy JIT compiler outperforms the CPython interpreter by 1.76× for steady-state while being 2% slower for start-up, which refutes the statement on the PyPy website that 'PyPy outperforms CPython by 4.4× on average' based on the geometric mean speedup and not making a distinction between start-up and steady-state. We use the proposed methodology to analyze Python workloads which yields several interesting findings regarding PyPy versus CPython performance, start-up versus steady-state performance, the impact of a workload's input size, and Python workload execution characteristics at the microarchitecture level.

show abstract

Quantitative Overhead Analysis for Python

Ismail

Suh

2018

2018 IEEE International Symposium on Workload Characterization (IISWC)

View full text Add to dashboard Cite

Cross-layer workload characterization of meta-tracing JIT VMs

Cited by 4 publications

References 28 publications

A memory footprint optimization framework for Python applications targeting edge devices

A memory footprint optimization framework for Python applications targeting edge devices

A Rigorous Benchmarking and Performance Analysis Methodology for Python Workloads

Quantitative Overhead Analysis for Python

Contact Info

Product

Resources

About