A Local Parallel Communication Algorithm for Polydisperse Rigid Body Dynamics

Abstract

The simulation of large ensembles of particles is usually parallelized by partitioning the domain spatially and using message passing to communicate between the processes handling neighboring subdomains. The particles are represented as individual geometric objects and are assigned to the subdomains. Handling collisions and migrating particles between subdomains, as required for proper parallel execution, requires a complex communication protocol. Typically, the parallelization is restricted to handling only particles that are smaller than a subdomain. In many applications, however, particle sizes may vary drastically with some of them being larger than a subdomain. In this article we propose a new communication and synchronization algorithm that can handle the parallelization without size restrictions on the particles. Despite the additional complexity and extended functionality, the new algorithm introduces only minimal overhead. We demonstrate the scalability of the previous and the new communication algorithms up to almost two million parallel processes and for handling ten billion (10 10 ) geometrically resolved particles on a state-of-theart petascale supercomputer. Different scenarios are presented to analyze the performance of the new algorithm and to demonstrate its capability to simulate polydisperse scenarios, where large individual particles can extend across several subdomains.

A systematic comparison of runtime load balancing algorithms for massively parallel rigid particle dynamics

“…All algorithms introduced in chapter 2.3 are implemented in the waLBerla multiphysics framework that is freely available at www.walberla.net and licensed under GPLv3. Previous results obtained with this framework show very good scalability for rigid body dynamics simulations [3,5]. All tests are executed on the Juqueen supercomputer located at Jülich Supercomputing Center (JSC) and ranked 22 in the TOP500 list 1 as of November 2017.…”

“…Therefore the runtime and memory complexity is already an essential concern that must be addressed in the early stages of the algorithm and software design process. In the following sections we will outline our approach that leads to excellent scalability also for very large simulations [27,28,5].…”

“…Well known methods in this field are the Discrete Element Method (DEM) [1] and methods based on non-smooth granular dynamics [2,3,4]. Together with the increasing compute power of today's supercomputers, scenarios comprising billions (10 10 ) of non-spherical particles and contacts are possible [3,5]. To achieve simulations this large, a carefully engineered software is needed with a focus on highly parallel algorithms.…”

waLBerla: A block-structured high-performance framework for multiphysics simulations

“…For distributed memory parallel simulations, an additional synchronization algorithm has to be called. The framework provides efficient communication schemes for regular [82] and strongly polydisperse settings [83].…”

“…A detailed analysis of the scaling behavior can be found in Ref. 83. The simulation of large particle ensembles often results in a very inhomogeneous distribution of particles over the simulation domain.…”