A global address space approach to automated data management for parallel Quantum Monte Carlo applications

Abstract

Quantum Monte Carlo (QMC) applications perform simulation with respect to an initial state of the quantum mechanical system, which is often captured by using a cubic B-spline basis. This representation is stored as a read-only table of coefficients, and accesses to the table are generated at random as part of the Monte Carlo simulation. Current QMC applications such as QWalk and QMCPACK, replicate this table at every process or node, which limits scalability because increasing the number of processors does not enable larger systems to be run. We present a partitioned global address space (PGAS) approach to transparently managing this data using Global Arrays in a manner that allows the memory of multiple nodes to be aggregated. We develop an automated data management system that significantly reduces communication overheads, enabling new capabilities for QMC codes. Experimental results with the QWalk application demonstrate the effectiveness of the data management system.

MPI + MPI: a new hybrid approach to parallel programming with MPI plus shared memory

“…Initially, it was possible to store one copy of the ensemble data per process on a node. But as memory per hardware execution thread is no longer increasing and scientific objectives have targeted larger systems, this is no longer possible [27]. Thus, developers of several QMC applications-notably QMCPack-have invested significant time and effort into hybridizing existing MPI code with shared-memory libraries, such as OpenMP, in order to share the coefficients table [28].…”

The energy band memory server algorithm for parallel Monte Carlo transport calculations

“…(6) (7) Similar client-server models have been used successfully in quantum Monte Carlo calculations. (8) This analysis represents one specific realization of these general ideas with the goal of gauging feasibility for use in real-world production codes.…”