Multinode Multi-GPU Two-Electron Integrals: Code Generation Using the Regent Language

Abstract

The computation of two-electron repulsion integrals (ERIs) is often the most expensive step of integral-direct self-consistent field methods. Formally it scales as O(N 4), where N is the number of Gaussian basis functions used to represent the molecular wave function. In practice, this scaling can be reduced to O(N 2) or less by neglecting small integrals with screening methods. The contributions of the ERIs to the Fock matrix are of Coulomb (J) and exchange (K) type and require separate algorithms to compute matrix elements efficiently. We previously implemented highly efficient GPU-accelerated J-matrix and K-matrix algorithms in the electronic structure code TeraChem. Although these implementations supported the use of multiple GPUs on a node, they did not support the use of multiple nodes. This presents a key bottleneck to cutting-edge ab initio simulations of large systems, e.g., excited state dynamics of photoactive proteins. We present our implementation of multinode multi-GPU J- and K-matrix algorithms in TeraChem using the Regent programming language. Regent directly supports distributed computation in a task-based model and can generate code for a variety of architectures, including NVIDIA GPUs. We demonstrate multinode scaling up to 45 GPUs (3 nodes) and benchmark against hand-coded TeraChem integral code. We also outline our metaprogrammed Regent implementation, which enables flexible code generation for integrals of different angular momenta.

A Perspective on Sustainable Computational Chemistry Software Development and Integration

“…While it would be possible to leverage community software in the implementation of many of these kernels, it is often the case that the development of highly specialized kernels and algorithms leads to significant performance improvements on modern hardware. For example, the development of highly optimized recursions − and chemistry-specific quadrature schemes − for operator integral evaluation and contraction , are known to outperform generic numerical integral machinery on both CPU and accelerator architectures. In addition, domain-driven tensor frameworks − typically outperform generic tensor frameworks for computational chemistry workloads.…”

Quantum Mechanics/Molecular Mechanics Simulations on NVIDIA and AMD Graphics Processing Units

“…Typically, computing QM forces takes more than 95% of the total QM/MM time. In the recent past, various efforts have been undertaken to develop computationally affordable novel QM methods or reimplement traditional QM methods to harness the power of massively parallel central processing unit (CPU) and graphics processing unit (GPU) hardware platforms. − Most notably, a number of leading quantum chemistry software packages have been empowered with GPU acceleration allowing users to achieve unprecedented simulation speeds and model larger molecular systems efficiently. For instance, our own GPU-accelerated QUICK ab initio quantum chemistry and density functional theory package is highly efficient on NVIDIA hardware. , QM/MM simulations with QUICK/AMBER have displayed respectable speedups of up to 53 times for a single GPU with respect to a CPU core for a moderate-sized QM region size that was benchmarked at the time .…”

Memory-Efficient Recursive Evaluation of 3-Center Gaussian Integrals

“…All of these factors make it especially challenging to port Gaussian integral kernels onto accelerated coprocessors, such as general-purpose graphical processing units (GPGPUs, or, simply, GPUs), that have become the norm both on the commodity and high-end platforms. Hence there has been an intense effort to address these challenges, both on the modern central processing units (CPUs) with wide single-instruction-multiple-data (SIMD) instructions and on GPUs. ,,,,− …”