A Class of MSR Codes for Clustered Distributed Storage

Sohn, Jy-yong; Choi, Byung-Hee; Moon, Jaekyun

doi:10.1109/isit.2018.8437458

Cited by 15 publications

(21 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, 11, while the constructions in [6] and [7] can only support k = 9 and k = 6, 9, respectively. Note that the exactrepair construction of the minimum storage codes in [8] is given in the later work [9], and it can only support r = 2 and n = 2k.…”

Section: A Contributionsmentioning

confidence: 99%

“…We can show that the cross-rack repair bandwidth of RRC is less than that of the codes in [8] for most parameters. Later, Sohn et al [9], [10] present exact-repair constructions for the minimum storage point and the minimum bandwidth point of the codes in [8].…”

Section: Related Workmentioning

confidence: 99%

“…respectively. Therefore, we obtain α coded symbols which are the multiplication of the matrix in (9) and s (f −1)αn/r+1 s (f −1)αn/r+2 · · · s f αn/r by subtracting the coded symbols downloaded in racks m + 1, . .…”

Section: Hybrid Msrr Codes For αN/r ≥ M + αTmentioning

confidence: 99%

“…Therefore, we can recover two data symbols s 1 , s 2 by first subtracting the above two symbols from the four interference symbols s 3 , s 4 , s 5 , s 6 in the first rack and then solving the resulting two linear systems, because the 2 × 2 sub-matrix of the above matrix in the right is non-singular according to (9). Nodes 2 and 3 can be recovered similarly.…”

Section: Hybrid Msrr Codes For αN/r ≥ M + αTmentioning

confidence: 99%

See 3 more Smart Citations

Rack-Aware Regenerating Codes for Data Centers

Hou

Lee

Shum

et al. 2019

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

Erasure coding is widely used for massive storage in data centers to achieve high fault tolerance and low storage redundancy. Since the cross-rack communication cost is often high, it is critical to design erasure codes that minimize the crossrack repair bandwidth during failure repair. In this paper, we analyze the optimal trade-off between storage redundancy and cross-rack repair bandwidth specifically for data centers, subject to the condition that the original data can be reconstructed from a sufficient number of any non-failed nodes. We characterize the optimal trade-off curve under functional repair, and propose a general family of erasure codes called rack-aware regenerating codes (RRC), which achieve the optimal trade-off. We further propose exact repair constructions of RRC that have minimum storage redundancy and minimum cross-rack repair bandwidth, respectively. We show that (i) the minimum storage redundancy constructions support a wide range of parameters and have crossrack repair bandwidth that is strictly less than that of the classical minimum storage regenerating codes in most cases, and (ii) the minimum cross-rack repair bandwidth constructions support all the parameters and have less cross-rack repair bandwidth than that of the minimum bandwidth regenerating codes for almost all of the parameters.

show abstract

Section: A Contributionsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Hybrid Msrr Codes For αN/r ≥ M + αTmentioning

confidence: 99%

Section: Hybrid Msrr Codes For αN/r ≥ M + αTmentioning

confidence: 99%

See 2 more Smart Citations

Rack-Aware Regenerating Codes for Data Centers

Hou

Lee

Shum

et al. 2019

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…The two-cluster model was further developed in recent papers [21], [22] which assumed that the encoded data is placed in a number of clusters (generally more than two), and derived a cut-set type bound on the repair bandwidth for this case. Moreover, [22] showed existence of optimal-repair codes for their model, and [21] gave an explicit construction of MDS array codes of rate 1/2 for storage with two clusters. We also mention [15], [18], [26], [28] which discuss other variations of clustered storage architectures and are less related to our work.…”

mentioning

confidence: 99%

Explicit Constructions of MSR Codes for Clustered Distributed Storage: The Rack-Aware Storage Model

Chen

Barg

2020

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

The paper is devoted to the problem of erasure coding in distributed storage. We consider a model of storage that assumes that nodes are organized into equally sized groups, called racks, that within each group the nodes can communicate freely without taxing the system bandwidth, and that the only information transmission that counts is the one between the racks. This assumption implies that the nodes within each of the racks can collaborate before providing information to the failed node. The main emphasis of the paper is on code construction for this storage model. We present an explicit family of MDS array codes that support recovery of a single failed node from any number of helper racks using the minimum possible amount of inter-rack communication (such codes are said to provide optimal repair). The codes are constructed over finite fields of size comparable to the code length.We also derive a bound on the number of symbols accessed at helper nodes for the purposes of repair, and construct a code family that approaches this bound, while still maintaining the optimal repair property.Finally, we present a construction of scalar Reed-Solomon codes that support optimal repair for the rackoriented storage model.The problems of centralized and cooperative repair have been addressed in multiple recent papers, and there are explicit constructions of optimal-repair regenerating codes that cover the entire range of admissible parameters, require small-size ground alphabet compared to the length n of the encoding block, and attain the smallest possible repair bandwidth [16], [23], [28], [19], [27], [30], [13] (more references are given in a recent survey [2]). The availability of optimal constructions has motivated a shift of attention toward studying data recovery not only under communication, but also connectivity constraints, in other words, storage models in which communication cost between nodes differs depending on their location in the storage cluster. One of the simple extensions from the basic setting of homogeneous storage suggests that the nodes are joined into several groups (clusters), and repair of a node can be based on information from both the nodes within its own group and from nodes in the other groups. This permits to differentiate between communication within the cluster and the inter-cluster downloads, and the natural assumption is that the former is easier (contributes less to the repair bandwidth) than the latter.Erasure coding for clustered architectures was introduced several years ago and affords several variations. One of the first questions analyzed for heterogeneous storage models was related to repair under the condition that the system contains a group of nodes, downloading information from which contributes more to the repair bandwidth than downloading the same amount of information from the other nodes [1]. Later works [6], [14] observed that a more realistic version of non-homogeneous storage should assume that the cost of downloading information depends on the relative location of th...

show abstract

Functional repair codes: a view from projective geometry

Paterson

2019

Des. Codes Cryptogr.

View full text Add to dashboard Cite

Storage codes are used to ensure reliable storage of data in distributed systems. Here we consider functional repair codes, where individual storage nodes that fail may be repaired efficiently and the ability to recover original data and to further repair failed nodes is preserved. There are two predominant approaches to repair codes: a coding theoretic approach and a vector space approach. We explore the relationship between the two and frame the later in terms of projective geometry. We find that many of the constructions proposed in the literature can be seen to arise from natural and well-studied geometric objects, and that this perspective gives a framework that provides opportunities for generalisations and new constructions that can lead to greater flexibility in trade-offs between various desirable properties. We also frame the cut-set bound obtained from network coding in terms of projective geometry.We explore the notion of strictly functional repair codes, for which there exist nodes that cannot be replaced exactly. Currently only one known example is given in the literature, due to Hollmann and Poh. We examine this phenomenon from a projective geometry point of view, and discuss how strict functionality can arise.Finally, we consider the issue that the view of a repair code as a collection of sets of vector/projective subspaces is recursive in nature and makes it hard to visualise what a collection of nodes looks like and how one might approach a construction. Here we provide another view of using directed graphs that gives us non-recursive criteria for determining whether a family of collections of subspaces constitutes a function, exact, or strictly functional repair code, which may be of use in searching for new codes with desirable properties.

show abstract

A Class of MSR Codes for Clustered Distributed Storage

Cited by 15 publications

References 22 publications

Rack-Aware Regenerating Codes for Data Centers

Rack-Aware Regenerating Codes for Data Centers

Explicit Constructions of MSR Codes for Clustered Distributed Storage: The Rack-Aware Storage Model

Functional repair codes: a view from projective geometry

Contact Info

Product

Resources

About