Pointer life cycle types for lock-free data structures with memory reclamation

Abstract

We consider the verification of lock-free data structures that manually manage their memory with the help of a safe memory reclamation (SMR) algorithm. Our first contribution is a type system that checks whether a program properly manages its memory. If the type check succeeds, it is safe to ignore the SMR algorithm and consider the program under garbage collection. Intuitively, our types track the protection of pointers as guaranteed by the SMR algorithm. There are two design decisions. The type system does not track any shape information, which makes it extremely lightweight. Instead, we rely on invariant annotations that postulate a protection by the SMR. To this end, we introduce angels, ghost variables with an angelic semantics. Moreover, the SMR algorithm is not hard-coded but a parameter of the type system definition. To achieve this, we rely on a recent specification language for SMR algorithms. Our second contribution is to automate the type inference and the invariant check. For the type inference, we show a quadratic-time algorithm. For the invariant check, we give a source-to-source translation that links our programs to off-the-shelf verification tools. It compiles away the angelic semantics. This allows us to infer appropriate annotations automatically in a guess-and-check manner. To demonstrate the effectiveness of our type-based verification approach, we check linearizability for various list and set implementations from the literature with both hazard pointers and epoch-based memory reclamation. For many of the examples, this is the first time they are verified automatically. For the ones where there is a competitor, we obtain a speed-up of up to two orders of magnitude.Proving lock-free data structures linearizable has received a lot of attention (cf. Section 10). Doherty et al. [2004], for instance, give a mechanized proof of a practical lock-free queue. Such proofs require plenty of manual work and take a considerable amount of time. Moreover, they require an understanding of the proof method and the data structure under consideration. To overcome this drawback, we are interested in automated verification. The cave tool by Vafeiadis [2010a,b], for example, is able to establish linearizability for singly-linked data structures fully automatically.The problem with automated verification for lock-free data structures is its limited applicability. Most techniques are restricted to implementations that assume a garbage collector (GC). This assumption, however, does not apply to all programming languages. Take C/C++ as an example. It does not provide an automatic garbage collector that is running in the background. Instead, it is the programmer's obligation to avoid memory leaks by reclaiming memory that is no longer in use (using delete). In lock-free data structures, this task is much harder than it may seem at first glance. The root of the problem is that threads typically traverse the data structure without synchronization. Hence, there may be threads holding pointers to records that have alread...

A Concurrent Program Logic with a Future and History

“…The ghost code is justified by informal arguments (outside the program logic). We do not consider memory reclamation as it can be verified separately [47,48]. Beyond linearizability, history variables have recently been used to give specs to non-linearizable objects [32].…”