We describe a parallel fast multipole method (FMM) for highly nonuniform distributions of particles. We employ both distributed memory parallelism (via MPI) and shared memory parallelism (via OpenMP and GPU acceleration) to rapidly evaluate two-body nonoscillatory potentials in three dimensions on heterogeneous high performance computing architectures. We have performed scalability tests with up to 30 billion particles on 196,608 cores on the AMD/ CRAY-based Jaguar system at ORNL. On a GPU-enabled system (NSF's Keeneland at Georgia Tech/ORNL), we observed 30× speedup over a single core CPU and 7× speedup over a multicore CPU implementation. By combining GPUs with MPI, we achieve less than 10 ns/particle and six digits of accuracy for a run with 48 million nonuniformly distributed particles on 192 GPUs.

We describe our software package Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) recently publicly released. BLOPEX is available as a stand-alone serial library, as an external package to PETSc (Portable, Extensible Toolkit for Scientific Computation, a general purpose suite of tools developed by Argonne National Laboratory for the scalable solution of partial differential equations and related problems), and is also built into hypre (High Performance Preconditioners, a scalable linear solvers package developed by Lawrence Livermore National Laboratory). The present BLOPEX release includes only one solver-the Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) method for symmetric eigenvalue problems. hypre provides users with advanced high-quality parallel multigrid preconditioners for linear systems. With BLOPEX, the same preconditioners can now be efficiently used for symmetric eigenvalue problems. PETSc facilitates the integration of independently developed application modules, with strict attention to component interoperability, and makes BLOPEX extremely easy to compile and use with preconditioners that are available via PETSc. We present the LOBPCG algorithm in BLOPEX for hypre and PETSc. We demonstrate numerically the scalability of BLOPEX by testing it on a number of distributed and shared memory parallel systems, including a Beowulf system, SUN Fire 880, an AMD dual-core Opteron workstation, and IBM BlueGene/L supercomputer, using PETSc domain decomposition and hypre multigrid preconditioning. We test BLOPEX on a model problem, the standard 7-point finite-difference approximation of the 3-D Laplacian, with the problem size in the range of 10 5 -10 8 .

No abstract

This work presents the first extensive study of singlenode performance optimization, tuning, and analysis of the fast multipole method (FMM) on modern multicore systems. We consider single-and double-precision with numerous performance enhancements, including low-level tuning, numerical approximation, data structure transformations, OpenMP parallelization, and algorithmic tuning.Among our numerous findings, we show that optimization and parallelization can improve doubleprecision performance by 25× on Intel's quad-core Nehalem, 9.4× on AMD's quad-core Barcelona, and 37.6× on Sun's Victoria Falls (dual-sockets on all systems). We also compare our single-precision version against our prior state-of-the-art GPU-based code and show, surprisingly, that the most advanced multicore architecture (Nehalem) reaches parity in both performance and power efficiency with NVIDIA's most advanced GPU architecture.

Abstract. We analyze the conjugate gradient (CG) method with variable preconditioning for solving a linear system with a real symmetric positive definite (SPD) matrix of coefficients A. We assume that the preconditioner is SPD on each step, and that the condition number of the preconditioned system matrix is bounded above by a constant independent of the step number. We show that the CG method with variable preconditioning under this assumption may not give improvement, compared to the steepest descent (SD) method. We describe the basic theory of CG methods with variable preconditioning with the emphasis on "worst case" scenarios, and provide complete proofs of all facts not available in the literature. We give a new elegant geometric proof of the SD convergence rate bound. Our numerical experiments, comparing the preconditioned SD and CG methods, not only support and illustrate our theoretical findings, but also reveal two surprising and potentially practically important effects. First, we analyze variable preconditioning in the form of inner-outer iterations. In previous such tests, the unpreconditioned CG inner iterations are applied to an artificial system with some fixed preconditioner as a matrix of coefficients. We test a different scenario, where the unpreconditioned CG inner iterations solve linear systems with the original system matrix A. We demonstrate that the CG-SD inner-outer iterations perform as well as the CG-CG inner-outer iterations in these tests. Second, we compare the CG methods using a two-grid preconditioning with fixed and randomly chosen coarse grids, and observe that the fixed preconditioner method is twice as slow as the method with random preconditioning.

No abstract

SUMMARY We propose a new technique based on element agglomeration for constructing coarse subspaces of the lowest‐order tetrahedral Raviart–Thomas finite element space. The coarse spaces are spanned by local basis functions associated with each coarse face (i.e., with an interface between two agglomerated elements). Each such face is associated with up to four coarse basis functions. The support of these functions extends into the neighboring agglomerated elements, and the construction of these functions involves solution of certain local mixed finite element problem on each neighboring agglomerated element. In contrast to some previous work, the thus constructed coarse subspace exhibits improved approximation properties because under certain conditions, it locally contains (i.e., interpolates exactly) all vector constants. Our construction is general; in particular, we do not assume that the coarse faces are planar. Possible applications of the coarse Raviart–Thomas spaces are in constructing multigrid methods for the H(div) bilinear forms and (on the basis of the approximation properties of these spaces) in upscaling of mixed formulation of diffusion problems. Copyright © 2012 John Wiley & Sons, Ltd.

-We present two novel coarse spaces (H 1 -and H(curl)-conforming) based on element agglomeration on unstructured tetrahedral meshes. Each H 1 -conforming coarse basis function is continuous and piecewise-linear with respect to an original tetrahedral mesh. The H(curl)-conforming coarse space is a subspace of the lowest order Nédélec space of the first type. The H 1 -conforming coarse space exactly interpolates affine functions on each agglomerate. The H(curl)-conforming coarse space exactly interpolates vector constants on each agglomerate. Combined with the H(div)-and L 2 -conforming spaces developed previously in [8], the newly constructed coarse spaces form a sequence (with respect to exterior derivatives) which is exact as long as the underlying sequence of fine-grid spaces is exact. The constructed coarse spaces inherit the approximation and stability properties of the underlying fine-grid spaces supported by our numerical experiments. The new coarse spaces, in addition to multigrid, can be used for upscaling of broad range of PDEs involving curl, div and grad differential operators.2010 Mathematical subject classification: 65N20, 65M25.