Consistency and fault tolerance for erasure-coded distributed storage systems

Abstract

One challenge in applying erasure codes (or error-correcting codes) to distributed storage systems is to maintain consistency between data and redundancy blocks in the face of crashing servers. We present two access protocols that provide sequential consistency and maximum distance separable fault tolerance at the same time. The protocols use sequence numbers to recover a consistent version in the presence of failures or partial writes. The first (pessimistic) PSW protocol uses a master per stripe to execute updates in sequence. The second (optimistic) OCW protocol allows concurrent writes to blocks in the same stripe to happen in parallel at the cost of additional buffer space.We present empirical performance results for PSW and OCW and compare them to other protocols. Our results show that OCW is as fast as simple replication while providing better fault tolerance and/or reduced storage overhead. This demonstrates that erasure coding can be used as a space-efficient alternative to replication in distributed storage systems.

On Grid Quorums for Erasure Coded Data

“…The parities outside the quorum too need to be updated, but this is let to happen in the background. It can be carried out efficiently using a standard technique using differentials [ 7 , 8 , 9 , 10 ], which is outside the scope of this work.…”

“…Apart from quorum systems, another popular mechanism for consistency is the class of primary-based protocols, where each data block x has an associated primary, which is responsible for coordinating operations on x . For example, Reference [ 9 ] proposed an update model assuming a primary which serializes the update executions, and, similarly, in Reference [ 10 ], the node storing a data block enforces serialized writes, while updates are disseminated in a best effort manner to the redundant blocks, and stale reads are possible, i.e., there is no consistency guarantee.…”

“…Processes to maintain the amount of redundancy over time in the presence of node failures for erasure code based distributed storage systems have been profusely studied (see, e.g., Reference [5,6] for surveys on erasure coding techniques enabling redundancy maintenance in distributed storage systems). Designs of mechanisms that support efficient updates of coded data have also been considered; see, e.g., Reference [7][8][9][10].…”

An Efficient I/O-Redirection-Based Reconstruction Scheme for Erasure-Coded Storage Clusters

“…If D 1,1 is a frequently updated data block (i.e., block D 1,1 is modified by D 1,1 ), then all parity blocks of the same stripe should be updated. Parity blocks are updated by a commutative operation in an RS-coded system [23]; therefore, the updates at parity nodes S 3 and S 4 can be done in any order. …”

“…Most existing studies on I/O scheduling in erasure-coded storage systems (i.e., RAID storage systems, and erasurecoded storage clusters) are focused on the issues of reconstruction optimization [ [23], and so forth. Unfortunately, little attention has been paid to the topic of 'node recoveries concurrent with user writes'.…”