Parallel Scientific Computation

Abstract

This book explains how to use the bulk synchronous parallel (BSP) model to design and implement parallel algorithms in the areas of scientific computing and big data. Furthermore, it presents a hybrid BSP approach towards new hardware developments such as hierarchical architectures with both shared and distributed memory. The book provides a full treatment of core problems in scientific computing and big data, starting from a high-level problem description, via a sequential solution algorithm to a parallel solution algorithm and an actual parallel program written in the communication library BSPlib. Numerical experiments are presented for parallel programs on modern parallel computers ranging from desktop computers to massively parallel supercomputers. The introductory chapter of the book gives a complete overview of BSPlib, so that the reader already at an early stage is able to write his/her own parallel programs. Furthermore, it treats BSP benchmarking and parallel sorting by regular sampling. The next three chapters treat basic numerical linear algebra problems such as linear system solving by LU decomposition, sparse matrix-vector multiplication (SpMV), and the fast Fourier transform (FFT). The final chapter explores parallel algorithms for big data problems such as graph matching. The book is accompanied by a software package BSPedupack, freely available online from the author’s homepage, which contains all programs of the book and a set of test programs.

A staggered‐grid multilevel incomplete LU for steady incompressible flows

“…In the same manner, we can compare a 3D domain of size s x × s x × s x /2 to the rotated parallelepiped, and find a factor of . We remark that the truncated octahedron that is used in Bisseling 31 for the 3D domain and has a factor of 1.68 cannot be used for our multilevel method, since truncated octahedra are not self‐similar.…”

“…Another advantage of using a skew domain partitioning is that the amount of communication that is required is reduced when compared to a square partitioning. In Bisseling, 31 it is estimated that for the Laplace problem, the communication is asymptotically reduced by a factor of for the 2D diamond shape. If we instead compare the diamond shape to a rectangular domain with the same number of nodes (having the same number of nodes with a square domain is impossible), we find that communication is reduced by a factor of .…”

Exact k-way sparse matrix partitioning

“…8 illustrates this. It has been shown that the communication volumes of subsequent splits in a general partitioning method for parallel SpMV are additive; a proof can be found in [5], [20]. For RB, this means that…”