Capacity of Clustered Distributed Storage

Abstract

A new system model reflecting the clustered structure of distributed storage is suggested to investigate interplay between storage overhead and repair bandwidth as storage node failures occur. Large data centers with multiple racks/disks or local networks of storage devices (e.g. sensor network) are good applications of the suggested clustered model. In realistic scenarios involving clustered storage structures, repairing storage nodes using intact nodes residing in other clusters is more bandwidth-consuming than restoring nodes based on information from intra-cluster nodes. Therefore, it is important to differentiate between intra-cluster repair bandwidth and cross-cluster repair bandwidth in modeling distributed storage. Capacity of the suggested model is obtained as a function of fundamental resources of distributed storage systems, namely, node storage capacity, intra-cluster repair bandwidth and cross-cluster repair bandwidth. The capacity is shown to be asymptotically equivalent to a monotonic decreasing function of number of clusters, as the number of storage nodes increases without bound. Based on the capacity expression, feasible sets of required resources which enable reliable storage are obtained in a closed-form solution. Specifically, it is shown that the cross-cluster traffic can be minimized to zero (i.e., intra-cluster local repair becomes possible) by allowing extra resources on storage capacity and intra-cluster repair bandwidth, according to the law specified in the closed-form. The network coding schemes with zero crosscluster traffic are defined as intra-cluster repairable codes, which are shown to be a class of the previously developed locally repairable codes.

Explicit Constructions of MSR Codes for Clustered Distributed Storage: The Rack-Aware Storage Model

“…As before, the value of σ i ,t (J a ) depends only on the helper nodes {c j | j ∈ R}. Writing equations (22) in matrix form, we obtain…”

“…Furthermore, observe that the set of symbols we access in each helper node depends on index of failed node but not the index of the helper node. Indeed, even though the entries of the helper nodes involved in (22) depend on j ∈ R, overall the entries of the helper nodes accessed are indexed by the values i ∈ I (since the sets I a partition the set I). Thus, the repair matrices can be defined independently of the choice of the subset R ⊆ {0, .…”

“…The authors of [6], [14] have assumed that the storage is formed of two clusters and derived versions of the cut-set bound for the minimum repair bandwidth. The two-cluster model was further developed in recent papers [21], [22] which assumed that the encoded data is placed in a number of clusters (generally more than two), and derived a cut-set type bound on the repair bandwidth for this case. Moreover, [22] showed existence of optimal-repair codes for their model, and [21] gave an explicit construction of MDS array codes of rate 1/2 for storage with two clusters.…”

“…The two-cluster model was further developed in recent papers [21], [22] which assumed that the encoded data is placed in a number of clusters (generally more than two), and derived a cut-set type bound on the repair bandwidth for this case. Moreover, [22] showed existence of optimal-repair codes for their model, and [21] gave an explicit construction of MDS array codes of rate 1/2 for storage with two clusters. We also mention [15], [18], [26], [28] which discuss other variations of clustered storage architectures and are less related to our work.…”

Storage and repair bandwidth tradeoff for distributed storage systems with clusters and separate nodes

“…The DSS model with two clusters was considered in [6], [22]. In [27], the authors first proposed algorithms to characterize the capacity of the cluster DSS model also with no relay nodes under the assumption that all the other alive nodes are used to repair a failed one, which maximizes the system capacity as proven in [28]. However, this led to high reconstruction read cost which was defined in [17] as the number of helper nodes to recover a failed one.…”