StackTrack

Verification of concurrent data structures is one of the most challenging tasks in software verification. The topic has received considerable attention over the course of the last decade. Nevertheless, human-driven techniques remain cumbersome and notoriously difficult while automated approaches suffer from limited applicability. The main obstacle for automation is the complexity of concurrent data structures. This is particularly true in the absence of garbage collection. The intricacy of lock-free memory management paired with the complexity of concurrent data structures makes automated verification prohibitive.In this work we present a method for verifying concurrent data structures and their memory management separately. We suggest two simpler verification tasks that imply the correctness of the data structure. The first task establishes an over-approximation of the reclamation behavior of the memory management. The second task exploits this over-approximation to verify the data structure without the need to consider the implementation of the memory management itself. To make the resulting verification tasks tractable for automated techniques, we establish a second result. We show that a verification tool needs to consider only executions where a single memory location is reused. We implemented our approach and were able to verify linearizability of Michael&Scott's queue and the DGLM queue for both hazard pointers and epoch-based reclamation. To the best of our knowledge, we are the first to verify such implementations fully automatically.Otherwise, this equality may not hold.Property (G5) holds by definition together with Property (P5). It remains to show Properties (G2) to (G4) and (G6).Ad Property (G2). First consider the case where we have q ∈ valid τ . By Auxiliary (A1) we also have q ∈ valid σ . We getThe second equality holds because m τ (q) = b. That is, we preserve b.data in m τ | valid τ and only need to update the mapping of p. Similarly, we getNow consider the case q valid τ . By Auxiliary (A1) we also have q valid σ . We getThe last equality holds because the update does not survive the restriction to a set which is guaranteed not to contain p. Similarly, we getWe now get:The first equality is by definition of restrictions, the second equality is due to Property (P2), the third one is by Auxiliary (A1), and the last is again by definition. This concludes the property.Ad Property (G3). Only the valuation of p is changed by both act and act ′ . So by Property (P3) it suffices to show m τ .act (p)=a ⇐⇒ m σ .act ′ (p)=a. This follows easily:where the first and last equivalence hold due to the updates up and up ′ and the second equality holds by Property (P3).Ad Property (G4). Let c ∈ m τ .act (valid τ .act ). We have c ∈ m τ (valid τ ). To see this, consider some pexp ∈ valid τ .act such that m τ .act (pexp) = c. If pexp p, then m τ .act (pexp) = m τ (pexp) and pexp ∈ valid τ by definition. So c ∈ m τ (valid τ ) follows immediately. Otherwise, in the case of pexp ≡ p, we have m τ .act (p) = m τ (q). ...

show abstract

“…StackTrack (ST) [Alistarh et al 2014]. The basic idea is to use transactions to perform optimistic memory accesses.…”

Section: B Extended Discussion Of Smr Algorithmsmentioning

confidence: 99%

Decoupling lock-free data structures from memory reclamation for static analysis

Meyer

Wolff

2019

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

show abstract

“…Optimized HP implementations include the work by Aghazadeh et al [2014], the work by Dice et al [2016], and Cadence [Balmau et al 2016]. Dynamic Collect [Dragojevic et al 2011], StackTrack [Alistarh et al 2014], and ThreadScan [Alistarh et al 2015] are HP-esque implementations exploring the use of operating system and hardware support. Drop the Anchor [Braginsky et al 2013], Optimistic Access [Cohen and Petrank 2015b], Automatic Optimistic Access [Cohen and Petrank 2015a], QSense [Balmau et al 2016], Hazard Eras [Ramalhete and Correia 2017], and Interval-based Reclamation [Wen et al 2018] combine EBR and HP.…”

Section: Related Workmentioning

confidence: 99%

Pointer life cycle types for lock-free data structures with memory reclamation

Meyer

Wolff

2019

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

We consider the verification of lock-free data structures that manually manage their memory with the help of a safe memory reclamation (SMR) algorithm. Our first contribution is a type system that checks whether a program properly manages its memory. If the type check succeeds, it is safe to ignore the SMR algorithm and consider the program under garbage collection. Intuitively, our types track the protection of pointers as guaranteed by the SMR algorithm. There are two design decisions. The type system does not track any shape information, which makes it extremely lightweight. Instead, we rely on invariant annotations that postulate a protection by the SMR. To this end, we introduce angels, ghost variables with an angelic semantics. Moreover, the SMR algorithm is not hard-coded but a parameter of the type system definition. To achieve this, we rely on a recent specification language for SMR algorithms. Our second contribution is to automate the type inference and the invariant check. For the type inference, we show a quadratic-time algorithm. For the invariant check, we give a source-to-source translation that links our programs to off-the-shelf verification tools. It compiles away the angelic semantics. This allows us to infer appropriate annotations automatically in a guess-and-check manner. To demonstrate the effectiveness of our type-based verification approach, we check linearizability for various list and set implementations from the literature with both hazard pointers and epoch-based memory reclamation. For many of the examples, this is the first time they are verified automatically. For the ones where there is a competitor, we obtain a speed-up of up to two orders of magnitude.Proving lock-free data structures linearizable has received a lot of attention (cf. Section 10). Doherty et al. [2004], for instance, give a mechanized proof of a practical lock-free queue. Such proofs require plenty of manual work and take a considerable amount of time. Moreover, they require an understanding of the proof method and the data structure under consideration. To overcome this drawback, we are interested in automated verification. The cave tool by Vafeiadis [2010a,b], for example, is able to establish linearizability for singly-linked data structures fully automatically.The problem with automated verification for lock-free data structures is its limited applicability. Most techniques are restricted to implementations that assume a garbage collector (GC). This assumption, however, does not apply to all programming languages. Take C/C++ as an example. It does not provide an automatic garbage collector that is running in the background. Instead, it is the programmer's obligation to avoid memory leaks by reclaiming memory that is no longer in use (using delete). In lock-free data structures, this task is much harder than it may seem at first glance. The root of the problem is that threads typically traverse the data structure without synchronization. Hence, there may be threads holding pointers to records that have alread...

show abstract

“…Other recent solutions rely on either hardware or operating system support. In particular, Dragojevic et al [19] and Alistarh et al [2] proposed schemes based on hardware transactional memory. (The latter scheme is also semiautomatic, in that it only requires the programmer to specify a fallback path.…”

Section: Related Workmentioning

confidence: 99%

“…By assumption (1), every reachable node will have a non-zero reference count at the end of the scan. By assumption (2), no unreachable node can have non-zero reference count at the end of the scan. These two properties will imply that no reachable memory blocks can be allocated.…”

Section: (No False Positives)mentioning

confidence: 99%

“…A second, fine-grained approach is to employ one of many existing concurrent memory reclamation schemes, e.g. [2,4,26,31,32,34,44], which we discuss in detail in the next section. These are methods specifically designed to solve the problem of concurrent access to reclamation candidates, and can reduce the performance overheads of garbage collection.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Forkscan

Alistarh

Leiserson

Matveev

et al. 2017

Proceedings of the Twelfth European Conference on Computer Systems

Self Cite

View full text Add to dashboard Cite

The problem of efficient concurrent memory reclamation in unmanaged languages such as C or C++ is one of the major challenges facing the parallelization of billions of lines of legacy code. Garbage collectors for C/C++ can be inefficient; thus, programmers are often forced to use finelycrafted concurrent memory reclamation techniques. These techniques can provide good performance, but require considerable programming effort to deploy, and have strict requirements, allowing the programmer very little room for error. In this work, we present Forkscan, a new conservative concurrent memory reclamation scheme which is fully automatic and surprisingly scalable. Forkscan's semantics place it between automatic garbage collectors (it requires the programmer to explicitly retire nodes before they can be reclaimed), and concurrent memory reclamation techniques (as it does not assume that nodes are completely unlinked from the data structure for correctness). Forkscan's implementation exploits these new semantics for efficiency: we leverage parallelism and optimized implementations of signaling and copy-on-write in modern operating systems to efficiently obtain and process consistent snapshots of memory that can be scanned concurrently with the normal program operation. Empirical evaluation on a range of classical concurrent data structure microbenchmarks shows that Forkscan can preserve the scalability of the original code, while main

show abstract

StackTrack

Cited by 44 publications

References 27 publications

Decoupling lock-free data structures from memory reclamation for static analysis

Decoupling lock-free data structures from memory reclamation for static analysis

Pointer life cycle types for lock-free data structures with memory reclamation

Forkscan

Contact Info

Product

Resources

About