Dynamic Load Balancing for Compressible Multiphase Turbulence

Abstract

CMT-nek is a new scientific application for performing high fidelity predictive simulations of particle laden explosively dispersed turbulent flows. CMT-nek involves detailed simulations, is compute intensive and is targeted to be deployed on exascale platforms. The moving particles are the main source of load imbalance as the application is executed on parallel processors. In a demonstration problem, all the particles are initially in a closed container until a detonation occurs and the particles move apart. If all processors get an equal share of the fluid domain, then only some of the processors get sections of the domain that are initially laden with particles, leading to disparate load on the processors. In order to eliminate load imbalance in different processors and to speedup the makespan, we present different load balancing algorithms for CMT-nek on large scale multi-core platforms consisting of hundred of thousands of cores. The detailed process of the load balancing algorithms are presented. The performance of the different load balancing algorithms are compared and the associated overheads are analyzed. Evaluations on the application with and without load balancing are conducted and these show that with load balancing, simulation time becomes faster by a factor of up to 9.97.

On the Benefits of Anticipating Load Imbalance for Performance Optimization of Parallel Applications

“…Recently, Zhai et al [7] enhanced the work of Menon et al [6] by improving its adaptive LB capability. They allowed to activate a LB step before the optimal LB interval by incorporating an iterative computation of the performance degradation over time.…”

“…As proposed in the previous section, the load balancer is called every time the degradation due to load imbalance overcomes the average LB cost plus the overhead of ULBA. We implemented this behavior using the approach proposed by Zhai et al [7] that computes the exact degradation of each iteration w.r.t. a reference iteration (in our case, the one just after the last LB call).…”

“…2) A numerical study using an application that reproduces the computation of a fluid model with non-uniform erosion. In this application, we implemented a centralized LB technique on which we applied the adaptive LB approach of Zhai et al [7] and ULBA. We compared the computation time and the average PE usage of these two approaches.…”

“…Note that, we executed ULBA with α = 0.4. Figure 4a shows the performance comparison between the standard LB method with the adaptive LB approach of Zhai et al [7] and ULBA. Figure 4b shows the average PE usage (in Figure 4a, we observe that ULBA performs better than the standard LB method on all the test cases except for one where the performance is equal.…”

Dynamic load balancing for a mesh‐based scientific application

“…The underlying physical processes of CMT‐nek are complex and require tremendous computing power. As CMT‐nek is being readied for deployment on exascale supercomputers, one of the main challenges is load balancing the application on the available cores as shown in our previous work . A static load‐balancing strategy is used in Nek5000, where the simulation domain is first partitioned into spectral elements using a recursive spectral bisection algorithm .…”