Up to 700k GPU Cores, Kepler, and the Exascale Future for Simulations of Star Clusters Around Black Holes

Abstract

Abstract. We present direct astrophysical N-body simulations with up to a few million bodies using our parallel MPI/CUDA code on large GPU clusters in China, Ukraine and Germany, with different kinds of GPU hardware. These clusters are directly linked under the Chinese Academy of Sciences special GPU cluster program in the cooperation of ICCS (International Center for Computational Science). We reach about the half the peak Kepler K20 GPU performance for our ϕ-GPU 1 code [2], in a real application scenario with individual hierarchically block time-steps with the high (4 th , 6 th and 8 th ) order Hermite integration schemes and a real core-halo density structure of the modeled stellar systems. The code and hardware are mainly used to simulate star clusters [23,24] and galactic nuclei with supermassive black holes [20], in which correlations between distant particles cannot be neglected.

Fast coalescence of post-Newtonian Supermassive Black Hole Binaries in real galaxies

“…For the two BHB dy nam i cal or bit in te gra tion, we use the pub licly avail able jGPU * [7,8] with a 4 th or der Hermite in te gra tor and block hi er ar chi cal in di vid ual time step scheme. This Hermite scheme re quires us to know the ac cel er a tion and its first time-de riv a tive, called jerk.…”

The secular evolution of the Kuiper belt after a close stellar encounter

“…For the gravitational N -body problem these accelerators give a manifold speed increase with respect to code running on CPU (Nyland et al 2007;Portegies Zwart et al 2007;Bédorf & Portegies Zwart 2012). Parallel computers equipped more than one hundred GPUs have been utilized for various studies Berczik et al 2011Berczik et al , 2013 have been run efficiently in parallel to provide the computational power necessary to perform direct many body simulations. Access to such large GPU-equipped supercomputers, however, is not easy, in particular when the computations required a considerable fraction of the available hardware.…”

StePS: A multi-GPU cosmological N-body Code for compactified simulations

“…Our test simulation with ∼ 7.37·10 5 particles achieved ∼ 800 times acceleration with just two GPUs with single precision, compared to one Xeon E5520 CPU node, used in the MPI-OpenMP benchmark. We have sustained 9.3Tflop/s on this node, and this is ∼ 24% of the theoretical peak performance and roughly 50% of the efficiency of the ϕ-GPU direct N-body code [19]. Also, it is clear from this test, that using multiple GPUs is only worth it if the particle number is large enough.…”