Kokkos 3: Programming Model Extensions for the Exascale Era

Trott, Christian Robert; Lebrun-Grandié, Damien; Arndt, Daniel; Ciesko, Jan; Dang, Vinh Quang; Ellingwood, Nathan David; Gayatri, Rahulkumar; Harvey, Evan; Hollman, Daisy S.; Ibanez, Daniel Alejandro; Liber, Nevin; Madsen, Jonathan; Miles, Jeff; Poliakoff, David; Powell, Amy Jo; Rajamanickam, Sivasankaran; Simberg, Mikael; Sunderland, Dan; Turcksin, Bruno; Wilke, Jeremiah J

doi:10.1109/tpds.2021.3097283

Cited by 217 publications

(101 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kokkos is an open source, performance portable programming model for manycore devices implemented as a C++ template based library (Carter Edwards et al 2014;Trott et al 2021). As such it provides abstractions to leverage hardware features, e.g., threading or multi-level memory hierarchies, through various backends.…”

Section: Kokkosmentioning

confidence: 99%

See 1 more Smart Citation

Parthenon -- a performance portable block-structured adaptive mesh refinement framework

Grete¹,

Dolence²,

Miller³

et al. 2022

Preprint

View full text Add to dashboard Cite

On the path to exascale the landscape of computer device architectures and corresponding programming models has become much more diverse. While various low-level performance portable programming models are available, support at the application level lacks behind. To address this issue, we present the performance portable block-structured adaptive mesh refinement (AMR) framework PARTHENON, derived from the well-tested and widely used ATHENA++ astrophysical magnetohydrodynamics code, but generalized to serve as the foundation for a variety of downstream multi-physics codes. PARTHENON adopts the KOKKOS programming model, and provides various levels of abstractions from multi-dimensional variables, to packages defining and separating components, to launching of parallel compute kernels. PARTHENON allocates all data in device memory to reduce data movement, supports the logical packing of variables and mesh blocks to reduce kernel launch overhead, and employs one-sided, asynchronous MPI calls to reduce communication overhead in multi-node simulations. Using a hydrodynamics miniapp, we demonstrate weak and strong scaling on various architectures including AMD and NVIDIA GPUs, Intel and AMD x86 CPUs, as well as Fujitsu A64FX CPUs. At the largest scale, the miniapp reaches a total of 3.5 × 10 12 zone-cycles/s on 4096 Summit nodes (24576 GPUs) at ≈ 55% weak scaling parallel efficiency (starting from a single node). In combination with being an open, collaborative project, this makes PARTHENON an ideal framework to target exascale simulations in which the downstream developers can focus on their specific application rather than on the complexity of handling massivelyparallel, device-accelerated AMR.

show abstract

Section: Kokkosmentioning

confidence: 99%

“…To exploit on-node data parallelism, PARTHENON internally uses the performance portability programming model KOKKOS (Carter Edwards et al 2014;Trott et al 2021). This way PARTHENON inherits the capability to target various device architectures enabling code development using a single programming model.…”

Section: Introductionmentioning

confidence: 99%

Parthenon -- a performance portable block-structured adaptive mesh refinement framework

Grete¹,

Dolence²,

Miller³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The HEP/CCE-PPS group is currently evaluating Kokkos [6,7], SYCL [3], Alpaka [8,9,10], OpenMP/OpenACC [11] and std::execution::parallel [12] by porting a small • Patatrack and P2R from CMS [13] which perform pixel detector pattern recognition and tracking…”

Section: The Portable Parallelization Strategies Projectmentioning

confidence: 99%

Portability: A Necessary Approach for Future Scientific Software

Bhattacharya¹,

Calafiura²,

Childers³

et al. 2022

Preprint

View full text Add to dashboard Cite

Today's world of scientific software for High Energy Physics (HEP) is powered by x86 code, while the future will be much more reliant on accelerators like GPUs and FPGAs. The portable parallelization strategies (PPS) project of the High Energy Physics Center for Computational Excellence (HEP/CCE) is investigating solutions for portability techniques that will allow the coding of an algorithm once, and the ability to execute it on a variety of hardware products from many vendors, especially including accelerators. We think without these solutions, the scientific success of our experiments and endeavors is in danger, as software development could be expert driven and costly to be able to run on available hardware infrastructure. We think the best solution for the community would be an extension to the C++ standard with a very low entry bar for users, supporting all hardware forms and vendors. We are very far from that ideal though. We argue that in the future, as a community, we need to request and work on portability solutions and strive to reach this ideal.

show abstract

“…It exposes a single programming interface and allows the use of different optimizations for backends such as Cuda, HIP, SYCL, HPX, OpenMP, and C++ threads. [11], [4]. This was previously infeasible with a Cuda implementation, which is only suitable for execution on Nvidia GPUs.…”

Section: Kokkos Implementationmentioning

confidence: 99%

m-CUBES An efficient and portable implementation of multi-dimensional integration for gpus

Sakiotis¹,

Arumugam²,

Paterno³

et al. 2022

Preprint

View full text Add to dashboard Cite

The task of multi-dimensional numerical integration is frequently encountered in physics and other scientific fields, e.g., in modeling the effects of systematic uncertainties in physical systems and in Bayesian parameter estimation. Multi-dimensional integration is often time-prohibitive on CPUs. Efficient implementation on many-core architectures is challenging as the workload across the integration space cannot be predicted a priori. We propose m-Cubes, a novel implementation of the well-known Vegas algorithm for execution on GPUs. Vegas transforms integration variables followed by calculation of a Monte Carlo integral estimate using adaptive partitioning of the resulting space. m-Cubes improves performance on GPUs by maintaining relatively uniform workload across the processors. As a result, our optimized Cuda implementation for Nvidia GPUs outperforms parallelization approaches proposed in past literature. We further demonstrate the efficiency of m-Cubes by evaluating a six-dimensional integral from a cosmology application, achieving significant speedup and greater precision than the Cuba library's CPU implementation of Vegas. We also evaluate m-Cubes on a standard integrand test suite. m-Cubes outperforms the serial implementations of the Cuba and Gsl libraries by orders of magnitude speedup while maintaining comparable accuracy. Our approach yields a speedup of at least 10 when compared against publicly available Monte Carlo based GPU implementations. In summary, m-Cubes can solve integrals that are prohibitively expensive using standard libraries and custom implementations. A modern C++ interface header-only implementation makes m-Cubes portable, allowing its utilization in complicated pipelines with easy to define stateful integrals. Compatibility with non-Nvidia GPUs is achieved with our initial implementation of m-Cubes using the Kokkos framework.

show abstract

Kokkos 3: Programming Model Extensions for the Exascale Era

Cited by 217 publications

References 18 publications

Parthenon -- a performance portable block-structured adaptive mesh refinement framework

Parthenon -- a performance portable block-structured adaptive mesh refinement framework

Portability: A Necessary Approach for Future Scientific Software

m-CUBES An efficient and portable implementation of multi-dimensional integration for gpus

Contact Info

Product

Resources

About