A Fundamental Trade-Off between the Download Cost and Repair Bandwidth in Distributed Storage Systems

Akhlaghi, Soroush; Kiani, Abbas; Ghanavati, Mohammad Reza

doi:10.1109/netcod.2010.5487685

Cited by 18 publications

(28 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Upper/lower capacity bounds for the heterogeneous DSS are obtained in [20]. An asymmetric repair process is considered in [22], coining the terms, cheap and expensive nodes, based on the amount of data that can be transfered to any newcomer. The authors of [23] considered a flexible distributed storage system where the amount of information from helper nodes may be non-uniform, as long as the total repair bandwidth is bounded from above.…”

Section: B Related Workmentioning

confidence: 99%

“…The authors are with the School of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, 34141, Republic of Korea (e-mail: {jysohn1108, bbzang10, shyoon8}@kaist.ac.kr, jmoon@kaist.edu). A part of this paper was presented [1] at the IEEE Conference on Communications (ICC), Paris, France, May [21][22][23][24][25]2017 a newcomer node regenerates the failed node by contacting a number of survived nodes. This causes traffic burden across the network, taking up significant repair bandwidth.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Capacity of Clustered Distributed Storage

Sohn

Choi

Yoon

et al. 2019

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

A new system model reflecting the clustered structure of distributed storage is suggested to investigate interplay between storage overhead and repair bandwidth as storage node failures occur. Large data centers with multiple racks/disks or local networks of storage devices (e.g. sensor network) are good applications of the suggested clustered model. In realistic scenarios involving clustered storage structures, repairing storage nodes using intact nodes residing in other clusters is more bandwidth-consuming than restoring nodes based on information from intra-cluster nodes. Therefore, it is important to differentiate between intra-cluster repair bandwidth and cross-cluster repair bandwidth in modeling distributed storage. Capacity of the suggested model is obtained as a function of fundamental resources of distributed storage systems, namely, node storage capacity, intra-cluster repair bandwidth and cross-cluster repair bandwidth. The capacity is shown to be asymptotically equivalent to a monotonic decreasing function of number of clusters, as the number of storage nodes increases without bound. Based on the capacity expression, feasible sets of required resources which enable reliable storage are obtained in a closed-form solution. Specifically, it is shown that the cross-cluster traffic can be minimized to zero (i.e., intra-cluster local repair becomes possible) by allowing extra resources on storage capacity and intra-cluster repair bandwidth, according to the law specified in the closed-form. The network coding schemes with zero crosscluster traffic are defined as intra-cluster repairable codes, which are shown to be a class of the previously developed locally repairable codes.

show abstract

Section: B Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Capacity of Clustered Distributed Storage

Sohn

Choi

Yoon

et al. 2019

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…Codes for more relaxed settings with respect to the data recovery/repair requirements are presented in [17], [18]. The papers [19]- [22] provide alternative frameworks for regenerating codes that introduce additional parameters for the system; tradeoffs between storage and repair are derived in each of the papers for the functional repair scenario.…”

Section: Related Workmentioning

confidence: 99%

Distributed Storage Codes With Repair-by-Transfer and Nonachievability of Interior Points on the Storage-Bandwidth Tradeoff

Shah

Rashmi

Kumar

et al. 2012

IEEE Trans. Inform. Theory

233

300

View full text Add to dashboard Cite

Regenerating codes are a class of recently developed codes for distributed storage that, like Reed-Solomon codes, permit data recovery from any subset of k nodes within the n-node network. However, regenerating codes possess in addition, the ability to repair a failed node by connecting to an arbitrary subset of d nodes. It has been shown that for the case of functional-repair, there is a tradeoff between the amount of data stored per node and the bandwidth required to repair a failed node. A special case of functional-repair is exact-repair where the replacement node is required to store data identical to that in the failed node. Exact-repair is of interest as it greatly simplifies system implementation.The first result of the paper is an explicit, exact-repair code for the point on the storage-bandwidth tradeoff corresponding to the minimum possible repair bandwidth, for the case when d = n − 1. This code has a particularly simple graphical description and most interestingly, has the ability to carry out exact-repair through mere transfer of data and without any need to perform arithmetic operations. Hence the term repair-by-transfer.The second result of this paper shows that the interior points on the storage-bandwidth tradeoff cannot be achieved under exact-repair, thus pointing to the existence of a separate tradeoff under exact-repair. Specifically, we identify a set of scenarios, termed 'helper node pooling', and show that it is the necessity to satisfy such scenarios that over-constrains the system.

show abstract

“…Some researches, such as [19][20][21][22][23], jointly considered the repair-cost and heterogeneity of communication(download) cost on each links. They flexibly determine the amount of data to minimize the total repair cost, which is different from the regeneration time.…”

Section: Related Workmentioning

confidence: 99%

Heterogeneity-aware data regeneration in distributed storage systems

Wang

Wei

Yin

et al. 2014

IEEE INFOCOM 2014 - IEEE Conference on Computer Communications

View full text Add to dashboard Cite

Distributed storage systems provide large-scale reliable data storage services by spreading redundancy across a large group of storage nodes. In such big systems, node failures take place on a regular basis. When a node fails or leaves the system, to maintain the same level of redundancy, it is expected to regenerate the redundant data at a replacement node as soon as possible. Previous studies aim to minimize the network traffic in the regeneration process, but in practical networks, where link capacities vary in a wide range, minimizing network traffic does not always mean minimizing regeneration time. Considering the heterogeneous link capacities, Li et al. proposed a tree-structured regeneration scheme, called RCTREE, to bypass the low-capacitated link encountered in direct transmissions. However, we find that RCTREE may rapidly lose data integrity after several regenerations.In this paper, we reconsider the problem of minimizing regeneration time in networks with heterogeneous link capacities. We derive the minimum amount of data to be transmitted through each link to preserve data integrity. We prove that building an optimal regeneration tree is NP-complete and propose a heuristic algorithm for a near-optimal solution. We further introduce a flexible regeneration scheme, which allows providers to generate different amount of coded data. Simulation results show that the flexible tree-structured regeneration scheme can reduce the regeneration time significantly.

show abstract

A Fundamental Trade-Off between the Download Cost and Repair Bandwidth in Distributed Storage Systems

Cited by 18 publications

References 8 publications

Capacity of Clustered Distributed Storage

Capacity of Clustered Distributed Storage

Distributed Storage Codes With Repair-by-Transfer and Nonachievability of Interior Points on the Storage-Bandwidth Tradeoff

Heterogeneity-aware data regeneration in distributed storage systems

Contact Info

Product

Resources

About