Towards Lattice-Boltzmann on Dynamically Adaptive Grids — Minimally-Invasive Grid Exchange in Espresso

Abstract

Abstract. We present the minimally-invasive exchange of the regular Cartesian grid in the lattice-Boltzmann solver of ESPResSo by a dynamically-adaptive octree grid. Octree grids are favoured by computer scientists over other grid types as they are very memory-efficient. In addition, they represent a natural generalisation of regular Cartesian grids, such that most discretisation details of a regular grid solver can be maintained. Optimised codes, however, require a special tree-oriented grid traversal, which typically conflicts with existing simulation codes using various iterators, some for only parts of the grid, e.g., boundaries. ESPResSo is a large software package developed for soft-matter simulations involving fluid flow, electrostatic, and electrokinetic effects, and molecular dynamics. The currently used regular Cartesian grid hinders the simulation of realistic domain sizes and significant time periods, a problem that can be solved using grid adaptivity. In a first step, we focus on the lattice-Boltzmann flow solver in ESPResSo.p4est is a grid framework, that already provides dynamically adaptive quadtree and octree grids together with high-level interfaces for flexible grid traversals with direct neighbour access in all grid components. In this paper, we first describe extensions of p4est that were necessary to fulfill certain application requirements. The second part of our work consists of the minimally-invasive changes in ESPResSo preserving the expertise accumulated in the software's implementation over years. Our numerical results demonstrate physical correctness of the implementation, good parallel scalability and low overhead of the dynamical grid adaptivity. These are prerequisites to actually profit from grid adaptivity in terms of being able to simulate larger domains over longer time periods with limited computational resources. Thus, the current status forms a solid basis for further steps such as the development of refinement criteria, the setup of more realistic application scenarios, and a GPU implementation.

Extreme-Scale Block-Structured Adaptive Mesh Refinement

“…As such, the approach demonstrates state-of-the-art scalability. To the best knowledge of the authors, the scale as well as the performance of the benchmarks presented in this article significantly exceed the data previously published for LBM-based simulations capable of AMR [29,35,40,44,47,59,65].…”

“…As such, the implementation presented in this article is an example for a state-of-the-art SAMR code. Moreover, to the best knowledge of the authors, the total number of cells that can be handled and the overall performance achieved significantly exceed the performance data that has been published for the LBM on nonuniform grids to date [29,35,40,44,47,59,65].…”

“…Recently, the AMR scheme of [26] was further extended to support two-phase flow in [25]. First results of a cell-based AMR implementation build on top of the p4est library are presented in [40]. This implementation, however, does not yet support levelwise load balancing as it is necessary for balanced simulations in the context of the AMR schemes for the LBM.…”

A passenger model for simulating boarding and alighting in spatially confined transportation scenarios

“…As mentioned in the Introduction, many soft matter systems host concurrent interactions encompassing six or more decades in space and nearly twice as many in time. Two major directions can be endorsed to face this situation: the first consists in developing sophisticated multiscale methods capable of covering five-six spatial decades through a clever combination of advanced computational techniques, such as local grid-refinement, adaptive grids, or grid-particle combinations [45,46,47,48,49].…”

Towards Exascale Design of Soft Mesoscale Materials

“…As mentioned in the Introduction, many soft matter systems host concurrent interactions encompassing six or more decades in space and nearly twice as many in time. Two major directions can be endorsed to face this situation: the first consists in developing sophisticated multiscale methods capable of covering five-six spatial decades through a clever combination of advanced computational techniques, such as local grid-refinement, adaptive grids, or grid-particle combinations [47,48,49,50,51].…”