A massively space-time parallel N-body solver

Abstract

We present a novel space-time parallel version of the Barnes-Hut tree code PEPC using PFASST, the Parallel Full Approximation Scheme in Space and Time. The naive use of increasingly more processors for a fixed-size N-body problem is prone to saturate as soon as the number of unknowns per core becomes too small. To overcome this intrinsic strongscaling limit, we introduce temporal parallelism on top of PEPC's existing hybrid MPI/PThreads spatial decomposition. Here, we use PFASST which is based on a combination of the iterations of the parallel-in-time algorithm parareal with the sweeps of spectral deferred correction (SDC) schemes. By combining these sweeps with multiple space-time discretization levels, PFASST relaxes the theoretical bound on parallel efficiency in parareal. We present results from runs on up to 262,144 cores on the IBM Blue Gene/P installation JUGENE, demonstrating that the spacetime parallel code provides speedup beyond the saturation of the purely space-parallel approach.

Integrating an N-Body Problem with SDC and PFASST

“…Besides providing higher accuracy, PFASST also introduces an additional layer of parallelism. Provided that the spatial parallelization is already saturated, the application of PFASST can push the strong-scaling limit further by distributing the temporal integration across multiple time-processes, as shown in [26]. To shed more light on this concept, Figure 2 shows the number of f evaluations required by PFASST(8, 7) on one to eight time-processors with 16 time-steps.…”

“…While grid-based spatial coarsening by multi-grid techniques is well understood, spatial coarsening of particles systems is less straightforward. One possibility is to control the quality of the approximation of f using multipole methods instead of direct summation [26]. Thus, the use of fast summation algorithms not only allows extreme-scale simulations as demonstrated in [27], but also introduces a promising way of particle-based spatial "coarsening".…”

“…Furthermore, PFASST employs Full Approximation Scheme (FAS) corrections to increase the accuracy of SDC iterations on coarse levels. Many details of SDC, Parareal and PFASST have been omitted here for brevity, and the reader is referred to the more detailed discussions in e. g. [6,7,15,17,20,26].…”

“…The unfavorable quadratic complexity can be overcome by computing approximate interactions using e. g. BarnesHut tree codes [1] or the Fast Multipole Method [11]. Results on the strong scaling of PFASST on extreme scales, simulating merely 4 million particles on up to 262,144 cores, are reported in [26], where the massively parallel Barnes-Hut tree code PEPC [9,10,23,24,27] is applied. There, however, only a very brief discussion of accuracy is given, aiming solely at identifying parameters that generate time-parallel and time-serial solutions of comparable quality that allow for a meaningful comparison in terms of runtimes.…”

“…In Section 4 we summarize our results and comment on how further efficiency can be achieved for particle-based methods as a prelude to the large-scale simulations performed in [26].…”