RAPID: A Fast Data Update Protocol in Erasure Coded Storage Systems for Big Data

Abstract

Erasure codes are nowadays used extensively in\ud distributed storage systems that handle big data, since they\ud offer significant fault tolerance with low storage overhead. Even\ud though erasure coded systems are space efficient, these involve\ud higher network bandwidth and computational complexity in their\ud operations. In this paper, we present RAPID, a protocol for fast\ud data updates, which works by choosing a subset of code blocks\ud for updates and adapts the strength of the subset based on the\ud predicted number of failures. The proposal uses a prediction\ud based heuristic in which the set of failures that may happen in\ud the near future is represented as a function of past failures. A\ud hybrid protocol that uses both locking and buffering mechanisms\ud is adopted in the solution to maintain the consistency on the data\ud and code blocks updates. Our experimental results demonstrate\ud improvement in the performance of data updates by 30% and\ud the failure prediction mechanism proposed shows an accuracy of\ud 80%

Erasure Coding-Oriented Data Update for Cloud Storage: A Survey

“…As stated before, DU depends on encoding or its delta style. Several excellent techniques have been proposed in the literature, which significantly improve encoding efficiency, such as Bitmatrix Normalization (BN) [41], Smart Scheduling (SS) [41], Randomization [6], Interference Alignment [45], Update-Efficient Regenerating Codes (UERC) [26], Matching [27], RAPID [4]. While Vectorization [64] is a typical technique of accelerating the computation process.…”

“…Eq. (11) shows an example of BDM with [7,4,3] Hamming code [6], which has update complexity of 4. That is, if any single block is changed, 4 blocks are required to be updated at most (e.g., if d 4 is changed, the corresponding p 1 , p 2 , p 3 are required to be updated).…”

“…Another approach for optimizing DU is to reduce the total number of parity updates, which is called RAPID [4]. Akash et al recognized that it is not necessary to update parity nodes if no failures occur in this update round.…”

An Ant Colony Optimization Based Data Update Scheme for Distributed Erasure-Coded Storage Systems

“…Therefore, it is significant to improve the update efficiency for erasure codes. Motivated thereby, a plethora of efforts have been devoted recently to optimize the updating performance both in reducing I/Os and network transmission latency [6], [14]- [16]. Existing update schemes for erasure codes, such as Azure [17] and CodFS [14], adopt a log-based data update or a hybrid of in-place data updates and log-based parity updates to reduce I/Os by sequentially appending updates.…”

“…Existing update schemes for erasure codes, such as Azure [17] and CodFS [14], adopt a log-based data update or a hybrid of in-place data updates and log-based parity updates to reduce I/Os by sequentially appending updates. Alternatively, the authors in [6], [15], [16] try to mitigate the network transfer overhead by optimizing the update schedule and procedure.…”