Extracting ultra-scale Lattice Boltzmann performance via hierarchical and distributed auto-tuning

Abstract

We are witnessing a rapid evolution of HPC node architectures and on-chip parallelism as power and cooling constraints limit increases in microprocessor clock speeds. In this work, we demonstrate a hierarchical approach towards effectively extracting performance for a variety of emerging multicore-based supercomputing platforms. Our examined application is a structured grid-based Lattice Boltzmann computation that simulates homogeneous isotropic turbulence in magnetohydrodynamics. First, we examine sophisticated sequential auto-tuning techniques including loop transformations, virtual vectorization, and use of ISA-specific intrinsics. Next, we present a variety of parallel optimization approaches including programming model exploration (flat MPI, MPI/OpenMP, and MPI/Pthreads), as well as data and thread decomposition strategies designed to mitigate communication bottlenecks. Finally, we evaluate the impact of our hierarchical tuning techniques using a variety of problem sizes via large-scale simulations on state-of-the-art Cray XT4, Cray XE6, and IBM BlueGene/P platforms. Results show that our unique tuning approach improves performance and energy requirements by up to 3.4× using 49,152 cores, while providing a portable optimization methodology for a variety of numerical methods on forthcoming HPC systems.

Performance Analysis of the Lattice Boltzmann Model Beyond Navier-Stokes

“…In previous tuning studies conducted on Blue Gene/P, flat MPI and MPI/OpenMP programming models were shown to offer similar performance results for the LBM [21]. We found similar results as depicted in Fig.…”

“…Each processor adds an extra row to its domain of interest, as shown in blue, and populates this ghost cell row with the border data from its neighboring processor. The use of an extra row of ghost cells is often found in large-scale models [21], [18]; however, by stopping at one row, potential for further tuning is being left unexplored. Kjolstad and Snir discussed implementation methods of the ghost cell pattern in [22] and suggested the investigation of deep halos as a potential method to trade off computation for communication.…”

Topology exploration with hierarchical landscapes

“…Our high-dimensional data example is based on a study of auto-tuning strategies for HPC systems [Williams et al 2011] with the goal of designing an auto-tuner for large complicated algorithm runs. This study uses a lattice Boltzmann magnetohydrodynamics algorithm and its performance evaluation on different HPC systems to explore the available parameters to tune.…”

Esoteric Twist: An Efficient in-Place Streaming Algorithmus for the Lattice Boltzmann Method on Massively Parallel Hardware

“…Our performance model is based on the observation that the LBM is typically a memory bandwidth limited algorithm [3,[29][30][31][32]. The EsoTwist method for a 27 speed lattice requires 27 distributions that have to be read and written in every time step.…”