Asynchronous distributed-memory task-parallel algorithm for compressible flows on unstructured 3D Eulerian grids

Bakosi, József; Bird, Robert; González, Francisco Javier Miranda; Junghans, Christoph; Li, W.; Luo, Hong; Pandare, Aditya; Waltz, Jacob

doi:10.1016/j.advengsoft.2020.102962

Cited by 8 publications

(7 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed p‐ adaptive DG method has been implemented in Quinoa (https://quinoacomputing.org/). 33 In this section, we apply the p ‐adaptive DG method to a number of testcases to verify both the accuracy of the algorithm and the parallel performance obtained by the combination of the p ‐adaptive DG method and Charm++ parallel library. The testcases below is designed to evaluate the capability for the p ‐adaptive algorithm to capture both the rarefaction waves and discontinuities.…”

Section: Numerical Resultsmentioning

confidence: 99%

See 1 more Smart Citation

A parallel p‐adaptive discontinuous Galerkin method for the Euler equations with dynamic load‐balancing on tetrahedral grids

Li,

Pandare,

Luo

et al. 2023

Numerical Methods in Fluids

View full text Add to dashboard Cite

A novel p‐adaptive discontinuous Galerkin (DG) method has been developed to solve the Euler equations on three‐dimensional tetrahedral grids. Hierarchical orthogonal basis functions are adopted for the DG spatial discretization while a third order TVD Runge‐Kutta method is used for the time integration. A vertex‐based limiter is applied to the numerical solution in order to eliminate oscillations in the high order method. An error indicator constructed from the solution of order and is used to adapt degrees of freedom in each computational element, which remarkably reduces the computational cost while still maintaining an accurate solution. The developed method is implemented with under the Charm++ parallel computing framework. Charm++ is a parallel computing framework that includes various load‐balancing strategies. Implementing the numerical solver under Charm++ system provides us with access to a suite of dynamic load balancing strategies. This can be efficiently used to alleviate the load imbalances created by p‐adaptation. A number of numerical experiments are performed to demonstrate both the numerical accuracy and parallel performance of the developed p‐adaptive DG method. It is observed that the unbalanced load distribution caused by the parallel p‐adaptive DG method can be alleviated by the dynamic load balancing from Charm++ system. Due to this, high performance gain can be achieved. For the testcases studied in the current work, the parallel performance gain ranged from 1.5× to 3.7×. Therefore, the developed p‐adaptive DG method can significantly reduce the total simulation time in comparison to the standard DG method without p‐adaptation.

show abstract

Section: Numerical Resultsmentioning

confidence: 99%

“…The data that support the findings of this study are available by running our open source code: Quinoa (https://quinoacomputing.org/). 33 …”

Section: Data Availability Statementmentioning

confidence: 99%

A parallel p‐adaptive discontinuous Galerkin method for the Euler equations with dynamic load‐balancing on tetrahedral grids

Li,

Pandare,

Luo

et al. 2023

Numerical Methods in Fluids

View full text Add to dashboard Cite

show abstract

“…As a result, various tasks (e.g., computation and communication) are performed independently, in arbitrary order, and are overlapped due to Charm++'s asynchronous-by-default paradigm. For an example of how task-parallelism can be specified in Charm++ with a different Euler solver, see [14].…”

Section: Automatic Load Balancingmentioning

confidence: 99%

Complex-Geometry 3D Computational Fluid Dynamics with Automatic Load Balancing

Bakosi,

Constans,

Horváth

et al. 2023

Fluids

View full text Add to dashboard Cite

We present an open-source code, Xyst, intended for the simulation of complex-geometry 3D compressible flows. The software implementation facilitates the effective use of the largest distributed-memory machines, combining data-, and task-parallelism on top of the Charm++ runtime system. Charm++’s execution model is asynchronous by default, allowing arbitrary overlap of computation and communication. Built-in automatic load balancing enables redistribution of arbitrarily heterogeneous computational load based on real-time CPU load measurement at negligible cost. The runtime system also features automatic checkpointing, fault tolerance, resilience against hardware failure, and supports power- and energy-aware computation. We verify and validate the numerical method and demonstrate the benefits of automatic load balancing for irregular workloads.

show abstract

“…Many open problems in physics involve vastly varying length-and time-scales. Some examples, drawn from astrophysics, include the deposition and redistribution of energy from active galactic nuclei (Bourne and Sijacki 2021;Glines et al, 2020;Meece et al, 2017;Prasad et al, 2020) relativistic accretion flows around compact objects (Miller et al, 2019b(Miller et al, , 2020Ryan et al, 2018;Ressler et al, 2020), the in-spiral and merger of neutron stars and black holes Alcubierre (2008); Miller and Schnetter (2016), and, more generally, turbulence simulations (Federrath et al, 2021;Grete et al, 2021b).…”

Section: Introductionmentioning

confidence: 99%

“…This similarly applies to other asynchronous many-task runtime systems such as C harm++ who also start to incorporate GPU support (Choi et al, 2022). One framework using AMR built on top of C harm++ is Q uinoa (Bakosi et al, 2021) that just started to use GPUs. Finally, gamer-2 is astrophysical, multi-physics code with support for GPU-accelerated AMR (Zhang et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

Parthenon—a performance portable block-structured adaptive mesh refinement framework

Grete

Dolence

Miller

et al. 2022

The International Journal of High Performance Computing Applica

Self Cite

View full text Add to dashboard Cite

On the path to exascale the landscape of computer device architectures and corresponding programming models has become much more diverse. While various low-level performance portable programming models are available, support at the application level lacks behind. To address this issue, we present the performance portable block-structured adaptive mesh refinement (AMR) framework Parthenon, derived from the well-tested and widely used Athena++ astrophysical magnetohydrodynamics code, but generalized to serve as the foundation for a variety of downstream multi-physics codes. Parthenon adopts the Kokkos programming model, and provides various levels of abstractions from multidimensional variables, to packages defining and separating components, to launching of parallel compute kernels. Parthenon allocates all data in device memory to reduce data movement, supports the logical packing of variables and mesh blocks to reduce kernel launch overhead, and employs one-sided, asynchronous MPI calls to reduce communication overhead in multi-node simulations. Using a hydrodynamics miniapp, we demonstrate weak and strong scaling on various architectures including AMD and NVIDIA GPUs, Intel and AMD x86 CPUs, IBM Power9 CPUs, as well as Fujitsu A64FX CPUs. At the largest scale on Frontier (the first TOP500 exascale machine), the miniapp reaches a total of 1.7 × 1013 zone-cycles/s on 9216 nodes (73,728 logical GPUs) at [Formula: see text] weak scaling parallel efficiency (starting from a single node). In combination with being an open, collaborative project, this makes Parthenon an ideal framework to target exascale simulations in which the downstream developers can focus on their specific application rather than on the complexity of handling massively-parallel, device-accelerated AMR.

show abstract

Asynchronous distributed-memory task-parallel algorithm for compressible flows on unstructured 3D Eulerian grids

Cited by 8 publications

References 13 publications

A parallel p‐adaptive discontinuous Galerkin method for the Euler equations with dynamic load‐balancing on tetrahedral grids

A parallel p‐adaptive discontinuous Galerkin method for the Euler equations with dynamic load‐balancing on tetrahedral grids

Complex-Geometry 3D Computational Fluid Dynamics with Automatic Load Balancing

Parthenon—a performance portable block-structured adaptive mesh refinement framework

Contact Info

Product

Resources

About