Faster Self-Consistent Field (SCF) Calculations on GPU Clusters

Abstract

A novel implementation of the self-consistent field (SCF) procedure specifically designed for high-performance execution on multiple graphics processing units (GPUs) is presented. The algorithm offloads to GPUs the three major computational stages of the SCF, namely, the calculation of oneelectron integrals, the calculation and digestion of electron repulsion integrals, and the diagonalization of the Fock matrix, including SCF acceleration via DIIS. Performance results for a variety of test molecules and basis sets show remarkable speedups with respect to the state-of-the-art parallel GAMESS CPU code and relative to other widely used GPU codes for both single and multi-GPU execution. The new code outperforms all existing multi-GPU implementations when using eight V100 GPUs, with speedups relative to Terachem ranging from 1.2× to 3.3× and speedups of up to 28× over QUICK on one GPU and 15× using eight GPUs. Strong scaling calculations show nearly ideal scalability up to 8 GPUs while retaining high parallel efficiency for up to 18 GPUs.

Q-Next: A Fast, Parallel, and Diagonalization-Free Alternative to Direct Inversion of the Iterative Subspace

“…Q-Next was developed on top of the EXtreme-scale Electronic Structure System (EXESS) high performance quantum chemistry code-base, ,, which is interfaced with the GAMESS quantum chemistry package. EXESS provides the code for the Fock build on multiple NVIDIA GPUs, which is shared among the Q-Next and traditional DIIS implementations.…”

“…While efficient parallel schemes for Fock matrix construction have been devised, ,− even when using high-performance eigensolvers, the diagonalization of large Fock matrices does not achieve a high parallel efficiency. , Therefore, as computer systems dedicated to scientific calculations move toward massively parallel architectures with hundreds to millions of processor cores, the Fock matrix diagonalization becomes increasingly inefficient in taking advantage of the FLOP capabilities of the hardware and a major impediment to achieving larger molecular sizes in SCF calculations.…”

“…The Q-Next algorithm was implemented for both multicore CPU and Graphics Processing Unit (GPU) architectures, as part of the SCF module of the EXtreme-scale Eletronic Structure System (EXESS) library ,,, interfaced with GAMESS US . The underlying theory is discussed in section , while the Q-Next algorithm and its implementation are presented in sections and , respectively.…”

Runtime Power Allocation Based on Multi-GPU Utilization in GAMESS

“…The high performance multi-GPU capabilities in the GAMESS/LibCChem [20] suite of programs include a GPU-accelerated Fock build [21], a full implementation of the Self-Consistent-Field (SCF) method [22] including the oneelectron integrals and the Direct-Inversion of the Iterative Subspace (DIIS) algorithm. These routines have been scaled up to the entirety of the Summit supercomputer [23] and have demonstrated excellent parallel efficiency when coupled with fragmentation methods [24] [25].…”

“…The overarching scheme for the SCF program includes a coordinator/worker dynamic work balancing algorithm, the steps of which are as follows: Firstly, the basis set information is accepted, the shells and shell pairs are constructed. Secondly, the shell pairs are sorted and stored in the binned batch container [22],…”