Harnessing the Power of Multi-GPU Acceleration into the Quantum Interaction Computational Kernel Program

A novel implementation of the self-consistent field (SCF) procedure specifically designed for high-performance execution on multiple graphics processing units (GPUs) is presented. The algorithm offloads to GPUs the three major computational stages of the SCF, namely, the calculation of oneelectron integrals, the calculation and digestion of electron repulsion integrals, and the diagonalization of the Fock matrix, including SCF acceleration via DIIS. Performance results for a variety of test molecules and basis sets show remarkable speedups with respect to the state-of-the-art parallel GAMESS CPU code and relative to other widely used GPU codes for both single and multi-GPU execution. The new code outperforms all existing multi-GPU implementations when using eight V100 GPUs, with speedups relative to Terachem ranging from 1.2× to 3.3× and speedups of up to 28× over QUICK on one GPU and 15× using eight GPUs. Strong scaling calculations show nearly ideal scalability up to 8 GPUs while retaining high parallel efficiency for up to 18 GPUs.

Section: Introductionmentioning

confidence: 99%

Faster Self-Consistent Field (SCF) Calculations on GPU Clusters

Barca

Alkan

Galvez-Vallejo

et al. 2021

“…The most direct comparison can be made to results reported by Merz and co-workers on the same processors, NVIDIA Tesla K80 GPUs. They report the parallel efficiency of ERI computation in their QUICK program to be 71% on 16 K80 GPUs for a water system with 2250 basis functions . While the authors use a different integral algorithm (OSHGP , ), molecular system, and basis set (def2-SVP) than the systems we tested, we are interested in comparing workloads and can use a computation taking comparable time in this work: 9 DNA base pairs and 3755 basis functions.…”

Section: Performance and Discussionmentioning

confidence: 99%

“…Schemes to scale GPU implementations to multiple nodes have only emerged in the past few years. 38,45,72,73 Merz and co-workers have extended the capability of their QUICK program, which computes J and the exchange correlation potential, to 4 GPU nodes using an MPI root-worker model. 72 Kussmann and Ochsenfeld have extended their PreLinK scheme for efficiently computing exact exchange to 10 GPU nodes using MPI.…”

Section: Introductionmentioning

confidence: 99%

“…38,45,72,73 Merz and co-workers have extended the capability of their QUICK program, which computes J and the exchange correlation potential, to 4 GPU nodes using an MPI root-worker model. 72 Kussmann and Ochsenfeld have extended their PreLinK scheme for efficiently computing exact exchange to 10 GPU nodes using MPI. 38 Gordon and co-workers demonstrate multinode scaling of their HF build implementation to 3 nodes on the Summit supercomputer.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multinode Multi-GPU Two-Electron Integrals: Code Generation Using the Regent Language

Johnson

Mirchandaney

Hoag

et al. 2022

The computation of two-electron repulsion integrals (ERIs) is often the most expensive step of integral-direct self-consistent field methods. Formally it scales as O(N 4), where N is the number of Gaussian basis functions used to represent the molecular wave function. In practice, this scaling can be reduced to O(N 2) or less by neglecting small integrals with screening methods. The contributions of the ERIs to the Fock matrix are of Coulomb (J) and exchange (K) type and require separate algorithms to compute matrix elements efficiently. We previously implemented highly efficient GPU-accelerated J-matrix and K-matrix algorithms in the electronic structure code TeraChem. Although these implementations supported the use of multiple GPUs on a node, they did not support the use of multiple nodes. This presents a key bottleneck to cutting-edge ab initio simulations of large systems, e.g., excited state dynamics of photoactive proteins. We present our implementation of multinode multi-GPU J- and K-matrix algorithms in TeraChem using the Regent programming language. Regent directly supports distributed computation in a task-based model and can generate code for a variety of architectures, including NVIDIA GPUs. We demonstrate multinode scaling up to 45 GPUs (3 nodes) and benchmark against hand-coded TeraChem integral code. We also outline our metaprogrammed Regent implementation, which enables flexible code generation for integrals of different angular momenta.

“…While efficient parallel schemes for Fock matrix construction have been devised, ,− even when using high-performance eigensolvers, the diagonalization of large Fock matrices does not achieve a high parallel efficiency. , Therefore, as computer systems dedicated to scientific calculations move toward massively parallel architectures with hundreds to millions of processor cores, the Fock matrix diagonalization becomes increasingly inefficient in taking advantage of the FLOP capabilities of the hardware and a major impediment to achieving larger molecular sizes in SCF calculations.…”

Section: Introductionmentioning

confidence: 99%

Q-Next: A Fast, Parallel, and Diagonalization-Free Alternative to Direct Inversion of the Iterative Subspace

Seidl

Barca

2022

As computer systems dedicated to scientific calculations become massively parallel, the poor parallel performance of the Fock matrix diagonalization becomes a major impediment to achieving larger molecular sizes in self-consistent field (SCF) calculations. In this Article, a novel, highly parallel, and diagonalization-free algorithm for the accelerated convergence of the SCF procedure is presented. The algorithm, called Q-Next, draws on the second-order SCF, quadratically convergent SCF, and direct inversion of the iterative subspace (DIIS) approaches to enable fast convergence while replacing the Fock matrix diagonalization SCF bottleneck with higher parallel efficiency matrix multiplications. Performance results on both parallel multicore CPU and GPU hardware for a variety of test molecules and basis sets are presented, showing that Q-Next achieves a convergence rate comparable to the DIIS method while being, on average, one order of magnitude faster.