Role-Oriented Code Generation in an Engine for Solving Hyperbolic PDE Systems

Abstract

The development of a high performance PDE solver requires the combined expertise of interdisciplinary teams w.r.t. application domain, numerical scheme and low-level optimization. In this paper, we present how the ExaHyPE engine facilitates the collaboration of such teams by isolating three rolesapplication, algorithms, and optimization expert -thus allowing team members to focus on their own area of expertise, while integrating their contributions into an HPC production code.Inspired by web application development practices ExaHyPE relies on two custom code generation modules, the Toolkit and the Kernel Generator, which follow a Model-View-Controller architectural pattern on top of the Jinja2 template engine library. Using Jinja2's templates to abstract the critical components of the engine and generated glue code, we isolate the application development from the engine. The template language also allows us to define and use custom macros that isolate low-level optimizations from the numerical scheme described in the templates.We present three use cases, each focusing on one of our user roles, showcasing how the design of the code generation modules allows to easily expand the solver schemes to support novel demands from applications, to add optimized algorithmic schemes (with reduced memory footprint, e.g.), or provide improved lowlevel SIMD vectorization support.

Vectorization and Minimization of Memory Footprint for Linear High-Order Discontinuous Galerkin Schemes

“…To write an ExaHyPE application users provide a specification file, which is passed to the ExaHyPE Toolkit. The Toolkit creates glue code, empty application-specific classes and (most important for this paper) core kernels that are tailored towards application and architecture (light-green box) [7]. Users need to complete the application-specific classes by providing PDE-and scenario-specific implementations (of flux functions, boundary conditions, etc.).…”

“…by extending support for AVX-512 instruction sets. A more detailed description of the Kernel Generator and Toolkit design can be found in [7].…”

“…LIBXSMM is integrated into the Kernel Generator via a custom matmul Jinja2 template macro as described in [7]. This macro can also generate a generic triple-loop matrix multiplication, if LIBXSMM is not available for an architecture.…”

“…Multiple variants of each kernel exist, allowing the user to adapt the scheme to a given application's numerical requirements -for example, choosing between a scheme for a linear or a non-linear PDE system. Furthermore, code generation utilities are used before compilation to produce kernels that are tailored toward both the given application and the target architecture, as specified by the user [7].…”

SYCL compute kernels for ExaHyPE

“…Our research stands in the tradition of work that distinguishes the role of the performance engineer strictly from the role of domain scientists, numerics experts, research software engineers, and further specialists participating in the computational sciences workflow [5]. We focus on the performance aspect in a GPGPU context.…”