A massively parallel and memory-efficient FEM toolbox with a hybrid total FETI solver with accelerator support

Abstract

In this paper we present the ESPRESO FEM library, which includes a FEM toolbox with interfaces to professional and open-source simulation tools, and a massively parallel Hybrid Total FETI (HTFETI) solver which can fully utilize the OLCF Titan supercomputer, and achieves super-linear scaling. This paper presents several new techniques for FETI solvers designed for efficient utilization of supercomputers with a focus on: (i) performance-we present a fivefold reduction of solver runtime for the Laplace equation by redesigning the FETI solver, and offloading the key workload to the accelerator. We compare Intel Xeon Phi 7120p and Tesla K80 and P100 accelerators to Intel Xeon E5-2680v3 and Xeon Phi 7210 CPUs; and (ii) memory efficiency-we present two techniques which increase the efficiency of the HTFETI solver 1.8 times, and pushes the limits of the largest possible problem ESPRESO can solve from 124 to 223 billion unknowns for problems with unstructured meshes. Finally we show that by dynamicly tuning hardware parameters we can reduce energy consumption by up to 33 %.

Parallel Harmonic Balance Method for Analysis of Nonlinear Dynamical Systems

“…However, the code we developed currently only uses MPI so we are unable to take advantage of this option. In order to achieve better scalability it is necessary to switch to iterative solvers coupled with efficient preconditioners [24]. It will be the focus of the next development of our software.…”

Toward an End-to-End Auto-tuning Framework in HPC PowerStack

“…Each parameter's value is then changed at every xth iteration of a loop in which the instrumentation is, which requires support at the application side to handle such modification during runtime. The key input information for both static and dynamic application parameters tuning is not only a list of parameter values to set but also dependency ESPRESO FETI solver [27] was tuned by using the READEX tools for optimal hardware configuration as well as using the ATP to find an optimal solver, preconditioner, and domain size, all of which have a major impact on the time to solution of the issued problem. The application has been instrumented with a set of regions as presented in Figure 5.…”

“…The application has been instrumented with a set of regions as presented in Figure 5. Such optimization led to major runtime and energy savings as well as to improved scalability [21], [27].…”

Application instrumentation for performance analysis and tuning with focus on energy efficiency

“…This approach can save both time and/or energy. The energy savings can be up to 30% and in addition, it can improve the scalability of the application as shown in References 16,17. The reachable savings strongly rely on the dynamism of the application and its proper instrumentation.…”