Scaling Up of E-Msr Codes Based Distributed Storage Systems with Fixed Number of Redundancy Nodes

Abstract

Distributed storage systems are becoming more and more popular with the rapidly increasing demand for large-scale data storage. To increase the capacity and I/O performance of a distributed storage system, scaling it up is a common method. Regenerating Codes are a class of distributed storage codes that offer good reliability through encoding and provide good bandwidth cost on failed nodes repairing. This paper studies the scaling problem of E-MSR codes based distributed storage systems with fixed number of redundancy codes. We generate the encoding matrices of an storage system carefully from the encoding matrices of an storage system to minimize the changes of encoded blocks when scaling. Therefore the system can be scaled up with relatively low bandwidth cost and computation cost.

Parity Resynchronization using a Block-level Journaling for Software RAID

“…However, for longer sync intervals (4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15)(16), the accumulated IB may need two or more scattered writes and is outperformed by the merged IL. The basic IL records a log at every cache synchronization and causes severe performance degradation due to frequent synchronization.…”

“…Research topics on RAID include access method [2,3], reconstruction [4][5][6][7], scrubbing [8][9][10], scaling [11][12][13][14][15], data layout [16][17][18][19][20][21], erasure code [22][23][24][25][26], and resynchronization [27][28][29]. This paper is focused on resynchronization for software RAID systems.…”

Rethinking RAID-5 Data Layout for Better Scalability

“…Zhao et al [8] proposed an approach generating the encoding matrices for distributed storage systems based on E-MSR codes. This approach minimizes the changes of encoded blocks when scaling.…”

“…This can meet the increasing requirements on storage systems in various online applications [4], while avoiding the extremely high downtime cost [5]. Second, RAID-based architectures are also used for clusters and largescale storage systems, where scalability plays a significant role [6]- [8]. Such disk addition is termed "RAID-5 scaling".…”