A Highly-Available Move Operation for Replicated Trees

Abstract

Replicated tree data structures are a fundamental building block of distributed filesystems, such as Google Drive and Dropbox, and collaborative applications with a JSON or XML data model. These systems need to support a move operation that allows a subtree to be moved to a new location within the tree. However, such a move operation is difficult to implement correctly if different replicas can concurrently perform arbitrary move operations, and we demonstrate bugs in Google Drive and Dropbox that arise with concurrent moves. In this paper we present a CRDT algorithm that handles arbitrary concurrent modifications on trees, while ensuring that the tree structure remains valid (in particular, no cycles are introduced), and guaranteeing that all replicas converge towards the same consistent state. Our algorithm requires no synchronous coordination between replicas, making it highly available in the face of network partitions. We formally prove the correctness of our algorithm using the Isabelle/HOL proof assistant, and evaluate the performance of our formally verified implementation in a geo-replicated setting.

Typed Image-based Programming with Structure Editing

“…Note that unique IDs had to be incorporated into Ins operations to support Proposition 4, so the status of IDs is still unclear. It would be interesting to see if recent CRDT advances [26] could provide an alternative solution. Cannot retract an edit at a location through the insert of that location.…”

An Efficient Approach to Move Elements in a Distributed Geo-Replicated Tree

“…CRDTs have become an indispensable component of many modern distributed systems that guarantee some form of eventual consistency [7]. Clients update their replicas concurrently without coordination to provide high availability even when the network is partitioned.…”

“…An interior node in the tree represents a directory, while a leaf node represents a file. This system runs a daemon on the client's machine that keeps track of changes by monitoring the designated directory [7], [8]. Clients can read and update files locally on their systems, which can then be synchronized with other replicas.…”

“…Move operations on the replicated tree are difficult to implement because concurrent operations by multiple clients may result in cycles; additionally, the tree structure may be broken [7], [8], [10]. Due to the concurrent operations, a concurrency control mechanism is required to ensure data correctness.…”

“…Further, ensuring correctness while providing low latency, high throughput, and availability can be challenging. Kleppmann et al [7] proposed an approach that first undoes all move operations applied with a higher timestamp (ts) than the ts of a move operation received from other replicas; applies the received operation, and finally, redoes the undone operations. They maintain total global order between the operations and ensure strong eventual consistency.…”