Scalable implicit finite element solver for massively parallel processing with demonstration to 160K cores

Abstract

Implicit methods for partial differential equations using unstructured meshes allow for an efficient solution strategy for many real-world problems (e.g., simulation-based virtual surgical planning). Scalable solvers employing these methods not only enable solution of extremely-large practical problems but also lead to dramatic compression in timeto-solution. We present a parallelization paradigm and associated procedures that enable our implicit, unstructured flow-solver to achieve strong scalability.We consider fluid-flow examples in two application areas to show the effectiveness of our procedures that yield nearperfect strong-scaling on various (including near-petascale) systems. The first area includes a double-throat nozzle (DTN) whereas the second considers a patient-specific abdominal aortic aneurysm (AAA) model. We present excellent strongscaling on three cases ranging from relatively small to large; a DTN model with O(10 6 ) elements up to 8,192 cores (9 core-doublings), an AAA model with O(10 8 ) elements up to 32,768 cores (6 core-doublings) and O(10 9 ) elements up to 163,840 cores.

Parallel Adaptive Boundary Layer Meshing for CFD Analysis

“…Since there is little computation performed during mesh adaptation relative to the substantial increase in communications required as the given mesh is distributed to more processors, the scaling decreases on high core counts (note that a strong scaling study is performed and therefore, the problem size is the same). However, the analysis have been shown to scale strongly with the similar amount of work load provided [21,41]. The fact that mesh modification routines are able to scale on bigger core count with more entities involved into communication supports the statement that it is likely to at a minimum provide the equivalent scaling with more work load on the same number of parts.…”

Petascale, Adaptive CFD (ALCF ESP Technical Report): ALCF-2 Early Science Program Technical Report

“…Both of these work types can be equally divided among the processors by partitioning the aggregate mesh into equal load parts [12,13]. So far, PHASTA is a pure MPI based code and each process executes of copy of the analysis code to handle the computation work and interactions corresponding to its mesh part.…”

Evaluation of Message Passing Communication Patterns in Finite Element Solution of Coupled Problems

“…The more complex message passing parallel model employs peer-to-peer (p2p) message exchanges among processors and takes advantage of communication and computation overlapping. Following [4], the point-to-point (p2p) communication strategy is based on a master-slave relationship between processors. This relationship is established by creating a hierarchy based on host partition numbers.…”