Distributed Storage Allocations

Abstract

We examine the problem of allocating a given total storage budget in a distributed storage system for maximum reliability. A source has a single data object that is to be coded and stored over a set of storage nodes; it is allowed to store any amount of coded data in each node, as long as the total amount of storage used does not exceed the given budget. A data collector subsequently attempts to recover the original data object by accessing only the data stored in a random subset of the nodes. By using an appropriate code, successful recovery can be achieved whenever the total amount of data accessed is at least the size of the original data object. The goal is to find an optimal storage allocation that maximizes the probability of successful recovery. This optimization problem is challenging in general because of its combinatorial nature, despite its simple formulation. We study several variations of the problem, assuming different allocation models and access models. The optimal allocation and the optimal symmetric allocation (in which all nonempty nodes store the same amount of data) are determined for a variety of cases. Our results indicate that the optimal allocations often have nonintuitive structure and are difficult to specify. We also show that depending on the circumstances, coding may or may not be beneficial for reliable storage.Comment: Extended version of a journal paper in the IEEE Transactions on Information Theory. 21 pages, 10 figures, 3 table

Erasure Codes-Based Storage-Aware Protocol For Preserving Cloud Data Under Correlation Failures

“…However, there exists a considerably different between our research and that of [14,15]. Despite [14,15] solve the regeneration problem and seek to get a trade off between communication…”

“…Some of the previous works interested in correlated failures mainly concentrate on (i) identifying the vulnerability of the system to such disasters [10,11] (e.g., estimating damage locations that have the maximum impact on network operation [10]), or the effect of such disasters [11], (ii) analyzing network capacity or evaluating its robustness [12], and measuring its reliability [8,13]. They are a few studies [14][15][16] stress how to distribute coded symbols of source SN to a subset of nodes in the network in order to ensure that each of them has one piece of data. The primary goal of this line of research is explicit establishment of a stable topology so that the original data can be recreated by visiting at least K SNs.…”

“…Our work is partly inspired by that of literatures [14,15], which consider stochastic failures and utilize error-correcting codes to replace the data on the disabled SN. However, there exists a considerably different between our research and that of [14,15].…”

Exploiting Rateless Codes in Cloud Storage Systems

“…It is worth noting that our fragment placement policy corresponds to a simple and intuitive symmetric allocation with maximal spreading using the terminology of [22]. Results in [22] are obtained under the hypothesis of optimal coding schemes, i.e., coding schemes where decoding is guaranteed whenever the ratio between the amount of retrieved coded fragments and k is ≥ 1.…”

Efficient Synchronization of Multiple Databases over Broadcast Networks

“…A survey of network coding and distributed storage systems can be found in [13]. Some problems of distributed storage systems, like storage allocations, are addressed in [11,12]. However, our problem differs from that of the distributed storage systems in the sense that our proposal does not aim to provide data reliability in case of node failure, but to allow all involved nodes to acquire local access to the complete dataset at any given time.…”