A self-organized, fault-tolerant and scalable replication scheme for cloud storage

Abstract

Failures of any type are common in current datacenters, partly due to the higher scales of the data stored. As data scales up, its availability becomes more complex, while different availability levels per application or per data item may be required. In this paper, we propose a self-managed key-value store that dynamically allocates the resources of a data cloud to several applications in a costefficient and fair way. Our approach offers and dynamically maintains multiple differentiated availability guarantees to each different application despite failures. We employ a virtual economy, where each data partition (i.e. a key range in a consistent-hashing space) acts as an individual optimizer and chooses whether to migrate, replicate or remove itself based on net benefit maximization regarding the utility offered by the partition and its storage and maintenance cost. As proved by a game-theoretical model, no migrations or replications occur in the system at equilibrium, which is soon reached when the query load and the used storage are stable. Moreover, by means of extensive simulation experiments, we have proved that our approach dynamically finds the optimal resource allocation that balances the query processing overhead and satisfies the availability objectives in a cost-efficient way for different query rates and storage requirements. Finally, we have implemented a fully working prototype of our approach that clearly demonstrates its applicability in real settings.

Brokering Algorithms for Optimizing the Availability and Cost of Cloud Storage Services

“…The main motivation for our strategy is agile reacting to any changes in cloud providers [5]. Similar to [7], Bonvin et al [8] propose a dynamic allocation of resources in a data cloud to host objects so that the availability to different user application is guaranteed in a cost-effective way. Their model is based on a virtual economy, in which each object partition operates as an individual optimizer and decides whether to replicate, migrate or remove itself based on net profit maximization related to the utility provided by the partition and its storage and maintenance.…”

A Performance and Profit Oriented Data Replication Strategy for Cloud Systems

“…Bonvin et al [18] presented Skute, a scattered key-value store. In Skute, virtual nodes act as autonomous agents and decide on behalf of the data owner, without any control from the outside.…”

“…Dealing with data replication, only a few studies are particularly interested in improved response time [14]- [17]. In addition, even fewer of those studies [18], [19] are taking economics of the cloud into account. To the best of our knowledge, lack of performance SLO is also true for the commercial clouds offered by Amazon 1 , Google 2 , and Microsoft 3 .…”

Caju: A Content Distribution System for Edge Networks

“…For instance, the use of non-collaborative LRU caching allows us to easily adapt the replication degree of an content according to its demand. More sophisticated adaptive schemes, such as EAD [7] and Skute [1] tackle content replication by using a cost-benefit approach over decentralized and structured P2P systems. EAD creates and deletes replicas throughout the query path with regards to object hit rate using an exponential moving average technique.…”