On the Efficient Evaluation of the Exchange Correlation Potential on Graphics Processing Unit Clusters

Abstract

The predominance of Kohn–Sham density functional theory (KS-DFT) for the theoretical treatment of large experimentally relevant systems in molecular chemistry and materials science relies primarily on the existence of efficient software implementations which are capable of leveraging the latest advances in modern high-performance computing (HPC). With recent trends in HPC leading toward increasing reliance on heterogeneous accelerator-based architectures such as graphics processing units (GPU), existing code bases must embrace these architectural advances to maintain the high levels of performance that have come to be expected for these methods. In this work, we purpose a three-level parallelism scheme for the distributed numerical integration of the exchange-correlation (XC) potential in the Gaussian basis set discretization of the Kohn–Sham equations on large computing clusters consisting of multiple GPUs per compute node. In addition, we purpose and demonstrate the efficacy of the use of batched kernels, including batched level-3 BLAS operations, in achieving high levels of performance on the GPU. We demonstrate the performance and scalability of the implementation of the purposed method in the NWChemEx software package by comparing to the existing scalable CPU XC integration in NWChem.

Harnessing the Power of Multi-GPU Acceleration into the Quantum Interaction Computational Kernel Program

“…The power of multi-GPUs has been harnessed into a range of traditional computational chemistry tools, [3][4][5][6][7][8][9][10][11][12] however, only a handful of ab initio quantum chemical packages [9][10][11][12][13] are among them. Meanwhile, with multi-GPU nodes increasingly becoming common in contemporary supercomputer centers, opensource quantum chemical codes that can fully exploit their power are in demand.…”

Achieving performance portability in Gaussian basis set density functional theory on accelerator based architectures in NWChemEx

“…We denote N j g = |B j | and N j b = |S j |. For a detailed description of the decomposition of Q into localized cuboids and subsequent screening of S employed in this work, we refer the reader to reference [24].…”

“…[4,5,23] for a review of modern trends as well as Refs. [24][25][26][27][28] for a number of recent developments).…”

“…As such, it is typically justifiable that performance portability may be achieved in large part by ensuring that the application has made use of microarchitecture optimized implementations of critical BLAS operations for the architecture in question, thus delegating the problem of performance portability to the developer of the optimized BLAS library (hardware vendor or otherwise) rather than the developer of the end application. However, as has been recently demonstrated for GPU architectures in the case of Gaussian basis set KS-DFT [24], the efficiency with which batched level-3 BLAS operations may be executed on contemporary GPU architectures shifts the relative computational importance of the BLAS operations on overall timeto-solution to be less than other algorithmic kernels which were previously not dominant. Thus, on accelerator based architectures, the problem of performance portability in KS-DFT software relies both on the use of optimized BLAS libraries and the hand optimization of specific performance critical kernels for microarchitectures of interest.…”

“…For atom-centered bases, the formation of the KS-DFT Fock matrix is dominated by the formation of the Coulomb matrix, exchangecorrelation (XC) matrix, and the exact-exchange matrix in the case of hybrid KS-DFT methods. Recently, a significant amount of research has been directed to the effecient evalution of the Coulomb and exact exchange matrices in Gaussian basis sets on GPU architectures [27,[31][32][33][34][35][36][37][38], while relatively little effort has been afforded to the evaluation of the XC matrix [24,25,27,28]. As such, we focus on the portable implementation of the XC matrix in this work.…”