On Grid Quorums for Erasure Coded Data

Abstract

We consider the problem of designing grid quorum systems for maximum distance separable (MDS) erasure code based distributed storage systems. Quorums are used as a mechanism to maintain consistency in replication based storage systems, for which grid quorums have been shown to produce optimal load characteristics. This motivates the study of grid quorums in the context of erasure code based distributed storage systems. We show how grid quorums can be built for erasure coded data, investigate the load characteristics of these quorum systems, and demonstrate how sequential consistency is achieved even in the presence of storage node failures.

Concurrency Control and Consistency Over Erasure Coded Data

“…A fundamental differences distinguish [25], [26], [27] from works reviewed earlier: they explicitly take into consideration the structural properties of the code, in particular, that the systematic pieces can be considered independent of each other, while the parities are dependent from (a subset of) systematic pieces. This naturally makes the operations at the granularity of individual pieces and implies that quorum membership can be explicitly determined based on the code structure, where the quorums for reads and writes are depen-dent on the particular systematic piece(s) involved.…”

“…Observing that only the systematic piece d x has changed [14], [15], [31], [27], [26], [25], one can recompute the new parity values incrementally without the need to carry out a full computation, as c ′ j = c j +ξ j,x δ x where δ x = d ′ x −d x . This involves a one time computation of δ x which is effectively a single addition operation, and the new value for each parity can be computed using a single multiplication and addition operation, as opposed to the k multiplications and k − 1 additions required above to recompute a parity from scratch.…”

“…[15], where one subvariant uses the client, while another subvariant uses a systematic storage node to disseminate the differential to the parity nodes. The works [25], [26], [27] also use this approach, but they are ambivalent regarding the entity which initially disseminates the differential among (some of) the parity nodes, and they additionally apply a gossip mechanism among parity nodes to continue the dissemination.…”

“…In [25], [26], [27], atomic Read-Modify-Write is achieved. This necessitates a blocking algorithm, hence the assumption of a distributed locking service, e.g., [35].…”

“…Two mechanisms guarantee atomic Read-Modify-Write consistency, given the dependency of parity pieces on (subsets of) the systematic pieces: (i) simultaneous writes on distinct parity pieces should not render the system in an inconsistent state, where parity pieces are modified out of order; this is particularly relevant for correct degraded reads, and (ii) all live parity nodes should ideally reflect their latest value, updated correspondingly to the latest changes to relevant systematic pieces. This second clause is not strictly necessary, but desirable to make optimal use of storage overhead, and this is thus a pragmatic choice adopted in [25], [26], [27], in contrast to other quorum based approaches in Subsection IV-B. There, not all parities are updated, instead additional redundancy is deployed up-front (e.g., many such mechanisms tolerate f failures where f is lower bounded by n−k 2 , even though an MDS (n, k) code can inherently tolerate n − k failures) and a decoding reliant on a majority of the nodes capturing latest value is performed, subject to an assumed bound in the number of failures in the system.…”