Scalable parallel regridding algorithms for block‐structured adaptive mesh refinement

Abstract

SUMMARYBlock-structured adaptive mesh refinement (BSAMR) is widely used within simulation software because it improves the utilization of computing resources by refining the mesh only where necessary. For BSAMR to scale onto existing petascale and eventually exascale computers all portions of the simulation need to weak scale ideally. Any portions of the simulation that do not will become a bottleneck at larger numbers of cores. The challenge is to design algorithms that will make it possible to avoid these bottlenecks on exascale computers. One step of existing BSAMR algorithms involves determining where to create new patches of refinement. The Berger-Rigoutsos algorithm is commonly used to perform this task. This paper provides a detailed analysis of the performance of two existing parallel implementations of the BergerRigoutsos algorithm and develops a new parallel implementation of the Berger-Rigoutsos algorithm and a tiled algorithm that exhibits ideal scalability. The analysis and computational results up to 98 304 cores are used to design performance models which are then used to predict how these algorithms will perform on 100 M cores.

Investigating applications portability with the Uintah DAG-based runtime system on PetaScale supercomputers

“…This Uintah grid can contain one or more levels with different resolutions while each level is further divided into smaller hexahedral patches. When running with SAMR, finer grid levels are created by the Uintah Regridder [36]. Since finer grid levels may not be continuous across the domain, Uintah uses a binary bounding volume hierarchy (BVH) tree to save patches on a particular grid level.…”

“…For example, Uintah uses a "data warehouse" through which all data transfer takes place and which ensures that application codes are isolated from the MPI communications layer. This degree of separation between the runtime system of Uintah and the applications codes written using a componentoriented parallel programming model makes it possible to achieve great increases in scalability through changes to the runtime system that executes the taskgraph, without changes to the applications components themselves.Particular advances made in Uintah are scalable adaptive mesh refinement [36][37][38] coupled to challenging multiphysics problems. [5,35] and a load balancing data assimilation and feedback approach [35], which out-performs traditional cost models.…”

Scalable large‐scale fluid–structure interaction solvers in the Uintah framework via hybrid task‐based parallelism algorithms

“…In the case of codes such as Uintah, which solves large systems of partial differential equations on a mesh, refining the mesh increases the accuracy of the simulation. The new regridder [11] defines a set of fixed-sized tiles throughout the domain. In the last 4 years, improvements within Uintah to the regridding and load-balancing algorithms have led to a 40 increase in the scalability of AMR [10,11].…”

“…When applying Uintah to fluid-structure interaction problems, the combination of adaptive meshing and the movement of structures through space present a formidable challenge in terms of achieving scalability on large-scale parallel computers. This mesh is also used as a scratch pad for the MPM calculation of the movement and deformation of the solid [10,11]. For this approach to be successful, it is necessary to design data structures that large numbers of cores can simultaneously access without contention.…”

“…The broad range and challenging nature of Uintah applications and the software itself are described in [5]. This mesh is also used as a scratch pad for the MPM calculation of the movement and deformation of the solid [10,11]. Uintah uses a combination of computational fluid dynamics (CFD) algorithms in its implicit continuous-fluid Eulerian (ICE) solver [6,7] and couples ICE to a particle-based solver for solids known as the material point method (MPM) [8,9].…”