Verifying strong eventual consistency in distributed systems

Abstract

Data replication is used in distributed systems to maintain up-to-date copies of shared data across multiple computers in a network. However, despite decades of research, algorithms for achieving consistency in replicated systems are still poorly understood. Indeed, many published algorithms have later been shown to be incorrect, even some that were accompanied by supposed mechanised proofs of correctness. In this work, we focus on the correctness of Conflict-free Replicated Data Types (CRDTs), a class of algorithm that provides strong eventual consistency guarantees for replicated data. We develop a modular and reusable framework in the Isabelle/HOL interactive proof assistant for verifying the correctness of CRDT algorithms. We avoid correctness issues that have dogged previous mechanised proofs in this area by including a network model in our formalisation, and proving that our theorems hold in all possible network behaviours. Our axiomatic network model is a standard abstraction that accurately reflects the behaviour of real-world computer networks. Moreover, we identify an abstract convergence theorem, a property of order relations, which provides a formal definition of strong eventual consistency. We then obtain the first machine-checked correctness theorems for three concrete CRDTs: the Replicated Growable Array, the Observed-Remove Set, and an Increment-Decrement Counter. We find that our framework is highly reusable, developing proofs of correctness for the latter two CRDTs in a few hours and with relatively little CRDT-specific code.

Semantics, Specification, and Bounded Verification of Concurrent Libraries in Replicated Systems

“…More directly related are proposals to deal with the specification and verification of various properties of CRDTs [12,18,31,41,42]. CRDTs also offer a library interface to clients and have been implemented for various data structures such as set, list, map, etc.…”

Conflict-Free Replicated Data Types CRDTs

“…The most commonly used approach is to have proofs when designs are proposed or to use some verification tools for the specific data type, such as TLA [45] or Isabelle 18 . There has also been some works that proposed general techniques for the verification of CRDTs [21,74,31], which can be used by CRDT developers to verify the correctness of their designs. Some of these works [74,31] include specific frameworks that help the developer in the verification process.…”

From Secure Messaging to Secure Collaboration

“…Fortunately, this problem has been studied extensively in the distributed systems literature. We propose using Conflict-free Replicated Data Types or CRDTs [8,14], a family of algorithms that provide abstractions and protocols for automatically resolving conflicts due to concurrent modifications.…”

“…CRDTs provide a consistency property called strong eventual consistency, which guarantees that whenever any two devices have seen the same set of updates (even if the updates were delivered in a di↵erent order), the data on those devices is in the same state [6,14]. This property implies that the state of a device is determined entirely by the set of updates it has seen.…”