Automated Tiling of Unstructured Mesh Computations with Application to Seismological Modeling

Abstract

Sparse tiling is a technique to fuse loops that access common data, thus increasing data locality. Unlike traditional loop fusion or blocking, the loops may have di erent iteration spaces and access shared datasets through indirect memory accesses, such as A[map [i]] -hence the name "sparse". One notable example of such loops arises in discontinuous-Galerkin nite element methods, because of the computation of numerical integrals over di erent domains (e.g., cells, facets). The major challenge with sparse tiling is implementationnot only is it cumbersome to understand and synthesize, but it is also onerous to maintain and generalize, as it requires a complete rewrite of the bulk of the numerical computation. In this article, we propose an approach to extend the applicability of sparse tiling based on raising the level of abstraction. Through a sequence of compiler passes, the mathematical speci cation of a problem is progressively lowered, and eventually sparse-tiled C for-loops are generated. Besides automation, we advance the state-of-the-art by introducing: a revisited, more e cient sparse tiling algorithm; support for distributed-memory parallelism; a range of ne-grained optimizations for increased run-time performance; implementation in a publicly-available library, SLOPE; and an in-depth study of the performance impact in Seigen, a real-world elastic wave equation solver for seismological problems, which shows speed-ups up to 1.28× on a platform consisting of 896 Intel Broadwell cores.(or equivalent data structure), which leads to indirect memory accesses within the loop nests. Indirections break static analysis, thus making purely compiler-based approaches insu cient. Runtime data dependence analysis is essential for sparse tiling, so integration of compiler and run-time tracking algorithms becomes necessary. Realistic datasets not tting in a single node Real-world simulations often operate on terabytes of data, hence execution on multi-node systems is often required. We have extended the original sparse tiling algorithm to enable distributed-memory parallelism.Sparse tiling does not change the semantics of a numerical method -only the order in which some iterations are executed. Therefore, if most sections of a PDE solver su er from computational boundedness and standard optimizations such as vectorization have already been applied, then sparse tiling, which targets memory-boundedness, will only provide marginal bene ts (if any). Likewise, if a global reduction is present in between two loops, then there is no way for sparse tiling to be applied, unless the numerical method itself is rethought. This is regardless of whether the reduction is explicit (e.g., the rst loop updates a global variable that is read by the second loop) or implicit (i.e., within an external function, as occurs for example in most implicit nite element solvers). These are probably the two greatest limitations of the technique; otherwise, sparse tiling may provide substantial performance bene ts.The rest of the article is structured a...

Inter-loop optimization in RAJA using loop chains

“…Luporini et al [31] has a similar goal of enabling the source code that uses a library (in this case Devito [32]) to schedule across loop computations that share data. The loops in their case have indirect array accesses.…”