HL-LHC Computing Review Stage 2, Common Software Projects: Data Science Tools for Analysis

Pivarski, J.; Rodrigues, E.; Pedro, K.; Shadura, O.; Krikler, B.; Stewart, G. A.

doi:10.48550/arxiv.2202.02194

Cited by 1 publication

(2 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nowadays, more front-end users use Python [8], but large-scale processing still needs to have high performance of C++ [7]. That is why we combine Python and C++ to take advantage of the best features of both languages so that we can have a Python user interface and, at the same time, take advantage of the performance and memory management of C++.…”

Section: Python-c++ Integrationmentioning

confidence: 99%

See 1 more Smart Citation

The Awkward World of Python and C++

Goyal¹,

Osborne²,

Pivarski³

2023

Preprint

View full text Add to dashboard Cite

There are undeniable benefits of binding Python and C++ to take advantage of the best features of both languages. This is especially relevant to the HEP and other scientific communities that have invested heavily in the C++ frameworks and are rapidly moving their data analyses to Python. Version 2 of Awkward Array, a Scikit-HEP Python library, introduces a set of header-only C++ libraries that do not depend on any application binary interface. Users can directly include these libraries in their compilation rather than linking against platformspecific libraries. This new development makes the integration of Awkward Arrays into other projects easier and more portable as the implementation is easily separable from the rest of the Awkward Array codebase. The code is minimal, it does not include all of the code needed to use Awkward Arrays in Python, nor does it include references to Python or pybind11. The C++ users can use it to make arrays and then copy them to Python without any specialized data types -only raw buffers, strings, and integers. This C++ code also simplifies the process of just-in-time (JIT) compilation in ROOT. This implementation approach solves some of the drawbacks, like packaging projects where native dependencies can be challenging. In this paper, we demonstrate the technique to integrate C++ and Python by using a header-only approach. We also describe the implementation of a new LayoutBuilder and a GrowableBuffer. Furthermore, examples of wrapping the C++ data into Awkward Arrays and exposing Awkward Arrays to C++ without copying them are discussed.

show abstract

Section: Python-c++ Integrationmentioning

confidence: 99%

“…HEP and other scientific communities have extensively invested in the C++ frameworks and are swiftly migrating their data analyses to Python. These communities are particularly interested in bridging the gap between the two languages [8]. This raises an important question: 'How to do Python-C++ integration the right way?…”

Section: Python-c++ Integrationmentioning

confidence: 99%