No abstract

waLBerla is a massively parallel software framework for simulating complex flows with the lattice Boltzmann method (LBM). Performance and scalability results are presented for SuperMUC, the world's fastest x86-based supercomputer ranked number 6 on the Top500 list, and JUQUEEN, a Blue Gene/Q system ranked as number 5.We reach resolutions with more than one trillion cells and perform up to 1.93 trillion cell updates per second using 1.8 million threads. The design and implementation of waLBerla is driven by a careful analysis of the performance on current petascale supercomputers. Our fully distributed data structures and algorithms allow for efficient, massively parallel simulations on these machines. Elaborate node level optimizations and vectorization using SIMD instructions result in highly optimized compute kernels for the single-and two-relaxation-time LBM. Excellent weak and strong scaling is achieved for a complex vascular geometry of the human coronary tree.

No abstract

No abstract

Programming current supercomputers efficiently is a challenging task. Multiple levels of parallelism on the core, on the compute node, and between nodes need to be exploited to make full use of the system. Heterogeneous hardware architectures with accelerators further complicate the development process. waLBerla addresses these challenges by providing the user with highly efficient building blocks for developing simulations on block-structured grids. The block-structured domain partitioning is flexible enough to handle complex geometries, while the structured grid within each block allows for highly efficient implementations of stencil-based algorithms. We present several example applications realized with waLBerla, ranging from lattice Boltzmann methods to rigid particle simulations. Most importantly, these methods can be coupled together, enabling multiphysics simulations. The framework uses meta-programming techniques to generate highly efficient code for CPUs and GPUs from a symbolic method formulation. To ensure software quality and performance portability, a continuous integration toolchain automatically runs an extensive test suite encompassing multiple compilers, hardware architectures, and software configurations.

In EEG/MEG source analysis, a mathematical dipole is widely used as the "atomic" structure of the primary current distribution. When using realistic finite element models for the forward problem, the current dipole introduces a singularity on the right-hand side of the governing differential equation that has to be treated specifically. We evaluated and compared three different numerical approaches, a subtraction method, a direct approach using partial integration and a direct approach using the principle of Saint Venant. Evaluation and comparison were carried out in a fourlayer sphere model using quasi-analytical formulas.

This article studies the performance and scalability of a geometric multigrid solver implemented within the hierarchical hybrid grids (HHG) software package on current high performance computing clusters up to nearly 300, 000 cores. HHG is based on unstructured tetrahedral finite elements that are regularly refined to obtain a block-structured computational grid. One challenge is the parallel mesh generation from an unstructured input grid that roughly approximates a human head within a 3D magnetic resonance imaging data set. This grid is then regularly refined to create the HHG grid hierarchy. As test platforms, a BlueGene/P cluster located at Jülich supercomputing center and an Intel Xeon 5650 cluster located at the local computing center in Erlangen are chosen. To estimate the quality of our implementation and to predict runtime for the multigrid solver, a detailed performance and communication model is developed and used to evaluate the measured single node performance, as well as weak and strong scaling experiments on both clusters. Thus, for a given problem size, one can predict the number of compute nodes that minimize the overall runtime of the multigrid solver. Overall, HHG scales up to the full machines, where the biggest linear system solved on Jugene had more than one trillion unknowns.

This study concentrates on finite-element-method (FEM)-based electroencephalography (EEG) forward simulation in which the electric potential evoked by neural activity in the brain is to be calculated at the surface of the head. The main advantage of the FEM is that it allows realistic modeling of tissue conductivity inhomogeneity. However, it is not straightforward to apply the classical model of a dipolar source with the FEM, due to its strong singularity and the resulting irregularity. The focus of this study is on comparing different methods to cope with this problem. In particular, we evaluate the accuracy of Whitney (Raviart-Thomas)-type dipole-like source currents compared to two reference dipole modeling methods: the St. Venant and partial integration approach. Common to all these methods is that they enable direct approximation of the potential field utilizing linear basis functions. In the present context, Whitney elements are particularly interesting, as they provide a simple means to model a divergence-conforming primary current vector field satisfying the square integrability condition. Our results show that a Whitney-type source model can provide simulation accuracy comparable to the present reference methods. It can lead to superior accuracy under optimized conditions with respect to both source location and orientation in a tetrahedral mesh. For random source orientations, the St. Venant approach turns out to be the method of choice over the interpolated version of the Whitney model. The overall moderate differences obtained suggest that practical aspects, such as the focality, should be prioritized when choosing a source model.