Pythran: Enabling Static Optimization of Scientific Python Programs

Guelton, Serge; Brunet, Pierrick; Raynaud, Alan; Merlini, Adrien; Amini, Mehdi

doi:10.25080/majora-8b375195-008

Cited by 9 publications

(8 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Libraries in the scientific Python ecosystem provide fast implementations of most important algorithms. Where extreme optimization is warranted, compiled languages can be used, such as Cython 43 , Numba 44 and Pythran 45 ; these languages extend Python and transparently accelerate bottlenecks. Owing to NumPy's simple memory model, it is easy to write low-level, hand-optimized code, usually in C or Fortran, to manipulate NumPy arrays and pass them back to Python.…”

Section: Discussionmentioning

confidence: 99%

Array programming with NumPy

Harris¹,

Millman

Walt

et al. 2020

Nature

13,155

7,505

View full text Add to dashboard Cite

Array programming provides a powerful, compact and expressive syntax for accessing, manipulating and operating on data in vectors, matrices and higher-dimensional arrays. NumPy is the primary array programming library for the Python language. It has an essential role in research analysis pipelines in fields as diverse as physics, chemistry, astronomy, geoscience, biology, psychology, materials science, engineering, finance and economics. For example, in astronomy, NumPy was an important part of the software stack used in the discovery of gravitational waves1 and in the first imaging of a black hole2. Here we review how a few fundamental array concepts lead to a simple and powerful programming paradigm for organizing, exploring and analysing scientific data. NumPy is the foundation upon which the scientific Python ecosystem is constructed. It is so pervasive that several projects, targeting audiences with specialized needs, have developed their own NumPy-like interfaces and array objects. Owing to its central position in the ecosystem, NumPy increasingly acts as an interoperability layer between such array computation libraries and, together with its application programming interface (API), provides a flexible framework to support the next decade of scientific and industrial analysis.

show abstract

Section: Discussionmentioning

confidence: 99%

Array programming with NumPy

Harris¹,

Millman

Walt

et al. 2020

Nature

13,155

7,505

View full text Add to dashboard Cite

show abstract

“…Instead of define static types explicitly, Shed [9] uses type inference techniques to determine the implicit types used in a Python program and translate typed Python programs into optimized C++. Pythran [10] further enables static optimization of scientific Python programs by using explicit thread-level parallelism through OpenMP annotations, false variable polymorphism pruning, and automatic vector instruction generation such as AVX or SSE. With developing of deep learning, supervised [11] and unsupervised [12] methods are also proposed to translate programming languages.…”

Section: Introductionmentioning

confidence: 99%

Svar: A Tiny C++ Header Brings Unified Interface for Multiple programming Languages

Zhao,

et al. 2021

Preprint

View full text Add to dashboard Cite

There are numerous types of programming languages developed in the last decades, and most of them provide interface to call C++ or C for high efficiency implementation. The motivation of Svar is to design an efficient, light-weighted and general middle-ware for multiple languages, meanwhile, brings the dynamism features from script language to C++ in a straightforward way. Firstly, a Svar class with JSON like data structure is designed to hold everything exists in C++, including basic values, functions or user defined classes and objects. Secondly, arguments are auto cast to and from Svar efficiently with compile time pointers, references and shared ptr detection. Thirdly, classes and functions are binded with string names to support reflection, this means all functions and classes in a shared library can be exported to a Svar object, which also calls a Svar module. The Svar modules can be accessed by different languages and this paper demonstrates how to import and use a Svar module in Python and Node.js. Moreover, the Svar modules or even a python module can also be imported by C++ at runtime, which makes C++ more easier to compile and use since headers are not required anymore. We compare the performance of Svar with two state-of-the-art binding tool for Python and Node.js, and the result demonstrates that Svar is efficient, elegant and general. The core of this project is one single tiny modern C++ header with less than 5000 lines code without extra dependency. To help developers using Svar, all the source codes related are public available on github http://github.com/zdzhaoyong/Svar, including documentations and benchmarks.

show abstract

“…Following its success, many other projects accelerate Python, either through bindings to existing C libraries such as CuPy [4], PyOpenCL [5], and PyKokkos [6] or by compiling a subset of Python to native code. Examples of the latter approach are Numba [7], Pythran [8], or Data-Centric Python [9].…”

Section: Introductionmentioning

confidence: 99%

Performance Evaluation of Python Parallel Programming Models: Charm4Py and mpi4py

Fink¹,

Liu²,

Choi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Python is rapidly becoming the lingua franca of machine learning and scientific computing. With the broad use of frameworks such as Numpy, SciPy, and TensorFlow, scientific computing and machine learning are seeing a productivity boost on systems without a requisite loss in performance. While highperformance libraries often provide adequate performance within a node, distributed computing is required to scale Python across nodes and make it genuinely competitive in large-scale highperformance computing. Many frameworks, such as Charm4Py, DaCe, Dask, Legate Numpy, mpi4py, and Ray, scale Python across nodes. However, little is known about these frameworks' relative strengths and weaknesses, leaving practitioners and scientists without enough information about which frameworks are suitable for their requirements. In this paper, we seek to narrow this knowledge gap by studying the relative performance of two such frameworks: Charm4Py and mpi4py.We perform a comparative performance analysis of Charm4Py and mpi4py using CPU and GPU-based microbenchmarks other representative mini-apps for scientific computing.

show abstract

Pythran: Enabling Static Optimization of Scientific Python Programs

Cited by 9 publications

References 0 publications

Array programming with NumPy

Array programming with NumPy

Svar: A Tiny C++ Header Brings Unified Interface for Multiple programming Languages

Performance Evaluation of Python Parallel Programming Models: Charm4Py and mpi4py

Contact Info

Product

Resources

About