High Availability in DHTs: Erasure Coding vs. Replication

Rodrigues, Rodrigo; Liskov, Barbara

doi:10.1007/11558989_21

Cited by 282 publications

(247 citation statements)

References 11 publications

Supporting

Mentioning

235

Contrasting

Unclassified

Order By: Relevance

“…The closest related works are on the repair problem, providing mechanisms for preserving the level of redundancy when one or more nodes fail or leave the system [25]. To this end, different symmetric and asymmetric network and erasure coding mechanisms have been proposed [26], [22], [27] that either rebuild a node with the same functionality as the failed node (functional repair) from the surviving nodes [22], [27], or reconstruct an exact copy of the failed node (exact repair) [28]. In [22], the notion of regenerating codes has been introduced to decrease the repair bandwidth by allowing a new node to communicate functions of the stored data from the surviving nodes.…”

Section: Related Workmentioning

confidence: 99%

An update model for network coding in cloud storage systems

Zakerinasab¹,

Wang²

2012

2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton)

View full text Add to dashboard Cite

Abstract-The fast emerging cloud services have received a tremendous amount of attention in both industry and academia. Storage services such as Dropbox and iCloud enabled us to share files among multiple users or devices. Providing the benefits of network coding in distributed systems such as Peerto-Peer file sharing and multimedia streaming, researchers have also been trying to apply network coding in storage systems. Existing works have been focusing on mechanisms for preserving the level of redundancy when one or more nodes fail or leave the system. However, file updates, the most frequent operations performed on files, pose challenges in maintaining coded information in the system up to date. In other words, any change in the file will impact all coded blocks in the system, so all traces of the file must be completely replaced. This becomes costly since recomputing coded blocks is very CPU intensive and replacing all the coded blocks consumes an excessive amount of bandwidth. To the best of our knowledge, there has been no work addressing this update problem. To this end, we propose a Differential Update Model (DUM) that will update coded blocks by delivering only the changes in the file. We also present our objective view of the model through a complete analysis on computational complexity and bandwidth saving and simulated experiments.

show abstract

Section: Related Workmentioning

confidence: 99%

An update model for network coding in cloud storage systems

Zakerinasab¹,

Wang²

2012

2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton)

View full text Add to dashboard Cite

show abstract

“…Examples of erasure codes are Reed Solomon codes (Plank, 1997), and RNNS (Barsi & Maestrini, 1973). In (Rodrigues & Liskov, 2005), the authors studied the use of erasure codes in peer-to-peer networks with frequent changes in peers' membership. Previous comparisons (Weatherspoon & Kubiatowicz, 2002) mostly argue that erasure coding is the clear victor, due to huge storage savings for the same availability levels (or conversely, huge availability gains for the same storage levels).…”

Section: Dependability Mechanismsmentioning

confidence: 99%

Programming a Sensor Network in a layered middleware architecture

Albano¹,

Chessa²

2010

Sustainable Wireless Sensor Networks

View full text Add to dashboard Cite

Wireless Sensor Networks (WSNs) have become a mature technology aimed at performing environmental monitoring and data collection. Nonetheless, harnessing the power of a WSN presents a number of research challenges. WSN application developers have to deal both with the business logic of the application and with WSN's issues, such as those related to networking (routing), storage, and transport. A middleware can cope with this emerging complexity, and can provide the necessary abstractions for the definition, creation and maintenance of applications. This work discusses the problems related to the development of such a middleware and surveys the state of the art on the field.

show abstract

“…Assume a five years MTTF unit , under the exponential assumption we calculate the probabilities that a data can survive a four months epoch and a 12 months epoch respectively, and get 91.1% and 75.6%. However the real probability is greater than [1][2][3][4][5][6][7][8][9][10] 16 for four months, and greater than 1-10 8 for 12 months.…”

Section: Impact Of Session Durability On System Designmentioning

confidence: 99%

“…As a result, how to improve the availability and the reliability has become a critical and heated issue in the system design. Some approaches, such as TotalRecall [1] and OceanStore [2], have been proposed to improve the reliability as well as availability, and some analytical works have been done, for instance, [3], [4] and [5].…”

Section: Introductionmentioning

confidence: 99%

Understanding the Session Durability in Peer-to-Peer Storage System

Tian

Dai

Wang

et al. 2006

Computational Science – ICCS 2006

View full text Add to dashboard Cite

Abstract. This paper emphasizes that instead of long-term availability and reliability, the short-term session durability analysis will greatly impact the design of the real large-scale Peer-to-Peer storage system. In this paper, we use a Markov chain to model the session durability, and then derive the session durability probability distribution. Subsequently, we show the difference between our analysis and the traditional Mean Time to Failure (MTTF) analysis, from which we conclude that the misuse of MTTF analysis will greatly mislead our understanding of the session durability. We further show the impact of session durability analysis on the real system design. To our best knowledge, this is the first time ever to discuss the effects of session durability in large-scale Peer-toPeer storage system.

show abstract

High Availability in DHTs: Erasure Coding vs. Replication

Cited by 282 publications

References 11 publications

An update model for network coding in cloud storage systems

An update model for network coding in cloud storage systems

Programming a Sensor Network in a layered middleware architecture

Understanding the Session Durability in Peer-to-Peer Storage System

Contact Info

Product

Resources

About