QLOC: Quorums With Local Reconstruction Codes

Abstract

In this paper we study the problem of consistency in distributed storage systems relying on erasure coding for storage efficient fault-tolerance. We propose QLOC -a flexible framework for supporting the storage of warm data, i.e., data which, while not being very frequently in use, nevertheless continues to be accessed for reads or writes regularly. QLOC builds upon (1) a generic family of local reconstruction codes with guarantees in terms of fault-tolerance, efficient recovery from failures and degraded mode operations, and can be instantiated with parameters customized to requirements such as storage overhead and reliability dictated by user needs and operational environments, and (2) quorum-based consistency mechanisms with support for read-modify-write operations without any underlying atomic primitives, providing deployment choices trading-off fault-tolerance, consistency and concurrency requirements. We carry out a theoretical analysis of the code properties, and experimentally benchmark the performance of the consistency enforcement mechanisms, demonstrating the practicality of the proposed approach.

Concurrency Control and Consistency Over Erasure Coded Data

“…A fundamental differences distinguish [25], [26], [27] from works reviewed earlier: they explicitly take into consideration the structural properties of the code, in particular, that the systematic pieces can be considered independent of each other, while the parities are dependent from (a subset of) systematic pieces. This naturally makes the operations at the granularity of individual pieces and implies that quorum membership can be explicitly determined based on the code structure, where the quorums for reads and writes are depen-dent on the particular systematic piece(s) involved.…”

“…The same trend is observed for existing works on concurrency control and consistency: early works mostly rely on MDS codes, or on generic codes [13], [14], [15], [25], which could be in particular either MDS or LRC codes. Only [27] focuses specifically on LRC codes. While the approach in [24] is general enough to work for arbitrary codes, the existing implementation employs n = k + 1 MDS codes since it was geared towards multi-datacenter deployments, i.e., only a single parity was actually used for cross data-center dispersal.…”

“…Observing that only the systematic piece d x has changed [14], [15], [31], [27], [26], [25], one can recompute the new parity values incrementally without the need to carry out a full computation, as c ′ j = c j +ξ j,x δ x where δ x = d ′ x −d x . This involves a one time computation of δ x which is effectively a single addition operation, and the new value for each parity can be computed using a single multiplication and addition operation, as opposed to the k multiplications and k − 1 additions required above to recompute a parity from scratch.…”

“…[15], where one subvariant uses the client, while another subvariant uses a systematic storage node to disseminate the differential to the parity nodes. The works [25], [26], [27] also use this approach, but they are ambivalent regarding the entity which initially disseminates the differential among (some of) the parity nodes, and they additionally apply a gossip mechanism among parity nodes to continue the dissemination.…”

“…In [25], [26], [27], atomic Read-Modify-Write is achieved. This necessitates a blocking algorithm, hence the assumption of a distributed locking service, e.g., [35].…”