A Formal Analysis of the NVIDIA PTX Memory Consistency Model

Abstract

This paper presents the first formal analysis of the official memory consistency model for the NVIDIA PTX virtual ISA. Like other GPU memory models, the PTX memory model is weakly ordered but provides scoped synchronization primitives that enable GPU program threads to communicate through memory. However, unlike some competing GPU memory models, PTX does not require data race freedom, and this results in PTX using a fundamentally different (and more complicated) set of rules in its memory model. As such, PTX has a clear need for a rigorous and reliable memory model testing and analysis infrastructure. We break our formal analysis of the PTX memory model into multiple steps that collectively demonstrate its rigor and validity. First, we adapt the English language specification from the public PTX documentation into a formal axiomatic model. Second, we derive an up-to-date presentation of an OpenCL-like scoped C++ model and develop a mapping from the synchronization primitives of that scoped C++ model onto PTX. Third, we use the Alloy relational modeling tool to empirically test the correctness of the mapping. Finally, we compile the model and mapping into Coq and build a full machine-checked proof that the mapping is sound for programs of any size. Our analysis demonstrates that in spite of issues in previous generations, the new NVIDIA PTX memory model is suitable as a sound compilation target for GPU programming languages such as CUDA. CCS Concepts • Hardware → Theorem proving and SAT solving; • Software and its engineering → Consistency.

Systems-on-Chip with Strong Ordering

“…Specifically, if there are two or more stores to overlapping locations from a lockstep execution, a determination as to which one was the last store to that location is not possible and hence the value of the subsequent load to that location is undefined. This is similar to the reasoning provided in prior works where the outcome of racey accesses that "occur at the same time" are undefined [5,23,36,46,55]. As a result, LSC does not impose any restriction on the value of overlapping stores from a lockstep execution.…”

“…HRF defines scopes in terms of the execution hierarchy of GPUs. For example, work-items within the same work-group (threadblock) synchronize through work-group scope, and work-items from different work-groups synchronize through device scope (scopes are present in other models as well [54]). While use of such scopes are well defined for synchronizing between work-items of a GPU, the synchronization between work-items and threads running on other processing elements on the same GPU is not clearly defined.…”

The semantics of shared memory in Intel CPU/FPGA systems

“…Memory Modelling. CPU memory models such as x86 [Owens et al 2009], POWER , Arm [Pulte et al 2017], and RISC-V [Pulte et al 2019] are now fairly well understood, as are some GPU memory models [Alglave et al 2015;Lustig et al 2019]. However, these models do not apply to systems where threads are on different devices.…”

Armed Cats

“…Modelling the concurrency aspects of the Armv8 architecture entails developing a consistency model for Armv8. Consistency models determine what values a read can take; weak consistency models such as the ones of Arm [4,23], IBM [37,38], Intel [39,40], Nvidia [10,33], RISC-V [3], C++ [20,31], Linux [15], and others allow more behaviours than Sequential Consistency (SC) [32].…”