Synchronising C/C++ and POWER

Abstract

Shared memory concurrency relies on synchronisation primitives: compare-and-swap, load-reserve/store-conditional (aka LL/SC), language-level mutexes, and so on. In a sequentially consistent setting, or even in the TSO setting of x86 and Sparc, these have well-understood semantics. But in the very relaxed settings of IBM R POWER R , ARM, or C/C++, it remains surprisingly unclear exactly what the programmer can depend on.This paper studies relaxed-memory synchronisation. On the hardware side, we give a clear semantic characterisation of the load-reserve/store-conditional primitives as provided by POWER multiprocessors, for the first time since they were introduced 20 years ago; we cover their interaction with relaxed loads, stores, barriers, and dependencies. Our model, while not officially sanctioned by the vendor, is validated by extensive testing, comparing actual implementation behaviour against an oracle generated from the model, and by detailed discussion with IBM staff. We believe the ARM semantics to be similar.On the software side, we prove sound a proposed compilation scheme of the C/C++ synchronisation constructs to POWER, including C/C++ spinlock mutexes, fences, and read-modify-write operations, together with the simpler atomic operations for which soundness is already known from our previous work; this is a first step in verifying concurrent algorithms that use load-reserve/storeconditional with respect to a realistic semantics. We also build confidence in the C/C++ model in its own terms, fixing some omissions and contributing to the C standards committee adoption of the C++11 concurrency model.

The Problem of Programming Language Concurrency Semantics

“…The C/C++11 concurrency model remains the state of the art for the semantics of a general-purpose shared-memory concurrent programming languages; it is, to the best of our knowledge, sound with respect to the compiler optimisation behaviour of implementations [29] (in contrast to the JMM [16,34]), it is provably compilable to relaxed hardware models [8,7,32], and our work here establishes a machine-checked DRF-SC theorem. But the thin-air problem shows that it allows too many behaviours, and we have seen here that that cannot be solved in a simple per-candidate-execution way, that the problem is not specific to relaxed atomics, that, while an operational solution for those examples is possible, it brings other difficulties, and that there are further problems with undefined behaviour.…”

“…The behaviour of Intel/AMD x86, IBM Power, and ARM multiprocessors has been clarified by a series of recent papers [35,33,32,26,4]. For x86, normal memory accesses have a Total Store Ordering (TSO) semantics, similar to SPARC TSO [1] -as if there were a FIFO write buffer (with a readback path) for each hardware thread, above a single memory.…”

“…The basic design was outlined by Boehm and Adve [13], and Batty et al [8] developed a formal semantics in the latter stages of the standardisation process, identifying various flaws in the draft standard and feeding back into the ratified standards and later defect reports. C/C++11 concurrency has been supported by GCC and Clang since versions 4.9 and 3.2 respectively, and the model by Batty et al has been used for many purposes, including correctness proofs for compilation schemes to x86, by Batty et al [8], and to IBM Power, by Batty et al [7] and Sarkar et al [32]; compiler testing via a theory of sound optimisations, by Morisset et al [29]; model checking, by Norris and Demsky [30]; compositional library abstraction, by Batty et al [6]; and program logics, by Vafeiadis and Narayan [39] and by Turon et al [37]. Elements of the model have also been incorporated into OpenCL 2.0.…”

Modelling the ARMv8 architecture, operationally: concurrency and ISA

“…for x86 [19], IBM Power [21,22], and ARMv7 [4]. The Power and ARM concurrency architectures are broadly similar, both being relaxed non-multi-copy atomic models that respect only certain program-order relations, and which have cumulative memory barrier instructions.…”

“…But they differ in many important aspects: they have different memory barrier and synchronisation instructions 1 , and the associated microarchitectures are quite different. That is important for us here: that Power model [21,22] does not correspond well to ARM implementations, and so cannot serve as a basis for the discussion with ARM architects that is needed for solid validation that it matches their intent.…”