Efficient Mesh Management in Firedrake Using PETSc DMPlex

Lange, Michael; Mitchell, Lawrence; Knepley, Matthew G.; Gorman, Gerard J.

doi:10.1137/15m1026092

Cited by 43 publications

(23 citation statements)

References 20 publications

(24 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Firedrake uses DMPlex [23] to handle meshes. DMPlex is responsible for partitioning, distributing over multiple processes, and locally reordering a mesh.…”

Section: Firedrake/dmplex: the S-depth Mechanism For Extended Halo Rementioning

confidence: 99%

Automated Tiling of Unstructured Mesh Computations with Application to Seismological Modeling

Luporini

Lange

Jacobs

et al. 2019

ACM Trans. Math. Softw.

Self Cite

View full text Add to dashboard Cite

Sparse tiling is a technique to fuse loops that access common data, thus increasing data locality. Unlike traditional loop fusion or blocking, the loops may have di erent iteration spaces and access shared datasets through indirect memory accesses, such as A[map [i]] -hence the name "sparse". One notable example of such loops arises in discontinuous-Galerkin nite element methods, because of the computation of numerical integrals over di erent domains (e.g., cells, facets). The major challenge with sparse tiling is implementationnot only is it cumbersome to understand and synthesize, but it is also onerous to maintain and generalize, as it requires a complete rewrite of the bulk of the numerical computation. In this article, we propose an approach to extend the applicability of sparse tiling based on raising the level of abstraction. Through a sequence of compiler passes, the mathematical speci cation of a problem is progressively lowered, and eventually sparse-tiled C for-loops are generated. Besides automation, we advance the state-of-the-art by introducing: a revisited, more e cient sparse tiling algorithm; support for distributed-memory parallelism; a range of ne-grained optimizations for increased run-time performance; implementation in a publicly-available library, SLOPE; and an in-depth study of the performance impact in Seigen, a real-world elastic wave equation solver for seismological problems, which shows speed-ups up to 1.28× on a platform consisting of 896 Intel Broadwell cores.(or equivalent data structure), which leads to indirect memory accesses within the loop nests. Indirections break static analysis, thus making purely compiler-based approaches insu cient. Runtime data dependence analysis is essential for sparse tiling, so integration of compiler and run-time tracking algorithms becomes necessary. Realistic datasets not tting in a single node Real-world simulations often operate on terabytes of data, hence execution on multi-node systems is often required. We have extended the original sparse tiling algorithm to enable distributed-memory parallelism.Sparse tiling does not change the semantics of a numerical method -only the order in which some iterations are executed. Therefore, if most sections of a PDE solver su er from computational boundedness and standard optimizations such as vectorization have already been applied, then sparse tiling, which targets memory-boundedness, will only provide marginal bene ts (if any). Likewise, if a global reduction is present in between two loops, then there is no way for sparse tiling to be applied, unless the numerical method itself is rethought. This is regardless of whether the reduction is explicit (e.g., the rst loop updates a global variable that is read by the second loop) or implicit (i.e., within an external function, as occurs for example in most implicit nite element solvers). These are probably the two greatest limitations of the technique; otherwise, sparse tiling may provide substantial performance bene ts.The rest of the article is structured a...

show abstract

“…Firedrake uses DMPlex [23] to handle meshes. DMPlex is responsible for partitioning, distributing over multiple processes, and locally reordering a mesh.…”

Section: Firedrake/dmplex: the S-depth Mechanism For Extended Halo Rementioning

confidence: 99%

Automated Tiling of Unstructured Mesh Computations with Application to Seismological Modeling

Luporini

Lange

Jacobs

et al. 2019

ACM Trans. Math. Softw.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Figure 10 illustrates the concept. On the top figure, we illustrate the action of node renumbering [37,38] to achieve a better data locality: nodes are "grouped" according their global numbering [37,38]. For example, when assembling element 3945, the gather is more efficient after renumbering (top right part of the figure) as nodal array positions are closer in memory.…”

Section: Spatial and Temporal Localitymentioning

confidence: 99%

High-Performance Computing: Dos and Don’ts

Houzeaux¹,

Borrell²,

Fournier³

et al. 2018

Computational Fluid Dynamics - Basic Instruments and Applications in Science

View full text Add to dashboard Cite

Computational fluid dynamics (CFD) is the main field of computational mechanics that has historically benefited from advances in high-performance computing. Highperformance computing involves several techniques to make a simulation efficient and fast, such as distributed memory parallelism, shared memory parallelism, vectorization, memory access optimizations, etc. As an introduction, we present the anatomy of supercomputers, with special emphasis on HPC aspects relevant to CFD. Then, we develop some of the HPC concepts and numerical techniques applied to the complete CFD simulation framework: from preprocess (meshing) to postprocess (visualization) through the simulation itself (assembly and iterative solvers).

show abstract

“…Note that a more advanced algorithm could consider a relation between the vertex and element ordering (see e.g. [36]), although here the two sets are ordered independently. The tool makes use of the SCOTCH library [37], using interfacing code derived from DOLFIN 1.5.0 [38].…”

Section: Mesh Reorderingmentioning

confidence: 99%

Adapting and optimising Fluidity for high-fidelity coastal modelling

2018

View full text Add to dashboard Cite

Work undertaken to improve the performance of Fluidity, an open-source finite-element computational fluid dynamics solver from Imperial College London, for both general computational fluid dynamics and tidal modelling problems is outlined. Optimising the general computational structure of Fluidity, along with work to improve the data decomposition and parallel load balancing enabled simulations to be run over three times faster than with the original code, even when using thousands of computational cores. This changes the level of detail at which fluids problems can be studied with Fluidity, and impacts upon research that examines high Reynolds number turbulent flows. This is of particular relevance in areas such as engineering aerodynamics, wind energy, marine energy, and environmental or pollution modelling.

show abstract

Efficient Mesh Management in Firedrake Using PETSc DMPlex

Cited by 43 publications

References 20 publications

Automated Tiling of Unstructured Mesh Computations with Application to Seismological Modeling

Automated Tiling of Unstructured Mesh Computations with Application to Seismological Modeling

High-Performance Computing: Dos and Don’ts

Adapting and optimising Fluidity for high-fidelity coastal modelling

Contact Info

Product

Resources

About