Replication strategies in unstructured peer-to-peer networks

CohenEdith,; Shenker, Scott

doi:10.1145/964725.633043

Cited by 223 publications

(236 citation statements)

References 8 publications

Supporting

Mentioning

236

Contrasting

Order By: Relevance

“…Several software approaches to data availability have also been deployed, especially in distributed and peer-to-peer environments. In particular, replication is widely used and investigated [2,6,7,9,12,21,51]. Similarly, there are many existing studies on erasure coding, and an increasing interest in deploying it in storage and peer-to-peer systems [10,[41][42][43].…”

Section: Related Workmentioning

confidence: 99%

Improving Data Availability for Better Access Performance: A Study on Caching Scientific Data on Distributed Desktop Workstations

Vazhkudai

Zhang

2009

J Grid Computing

View full text Add to dashboard Cite

Client-side data caching serves as an excellent mechanism to store and analyze the rapidly growing scientific data, motivating distributed, client-side caches built from unreliable desktop storage contributions to store and access large scientific data. They offer several desirable properties, such as performance impedance matching, improved space utilization, and high parallel I/O bandwidth. In this context, we are faced with two key challenges: (1) the finite amount of contributed cache space is stretched This work builds on several of our previously published studies. While we included extended introduction of the FreeLoader framework [57,58] as well as our combination of prefix caching and collective download techniques [33], this paper proposes the novel RDPR technique as well as a new striping-enabled cache management algorithm. We also discussed methods to combine RDPR with prefix caching and collective download techniques. Overall, more than 60% of the manuscript's content has not been previously published.X. Ma · Z. Zhang (B) North Carolina State University, Raleigh, NC, USA e-mail: zzhang3@ncsu.edu X. Ma · S. S. Vazhkudai Oak Ridge National Laboratory, Oak Ridge, TN, USA by the ever increasing scientific dataset sizes and (2) the transient nature of volunteered storage nodes impacts data availability. In this article, we address these challenges by exploiting the existence of external, primary copies of datasets. We propose a novel combination of prefix caching, collective download, and remote partial data recovery (RPDR), to deal with optimal cache space consumption and storage node volatility. Our evaluation, performed on our FreeLoader prototype, indicates that prefix caching can significantly improve the cache hit rate and partial data recovery is better than (or comparable to) many persistent-data availability techniques.

show abstract

Section: Related Workmentioning

confidence: 99%

Improving Data Availability for Better Access Performance: A Study on Caching Scientific Data on Distributed Desktop Workstations

Vazhkudai

Zhang

2009

J Grid Computing

View full text Add to dashboard Cite

show abstract

“…P2P research has also analyzed the searching aspect in conjunction with storage and replication of data. Work by [23] and [14] has shown how to minimize exchange of messages between peers whilst providing effective mechanisms to locate data and decide on replication in peer nodes other than the data requestor, as to optimize future requests.…”

Section: Peer To Peer Systemsmentioning

confidence: 99%

Managing Very-Large Distributed Datasets

Branco

Zaluska

Roure

et al. 2008

On the Move to Meaningful Internet Systems: OTM 2008

View full text Add to dashboard Cite

Abstract. In this paper, we introduce a system for handling very large datasets, which need to be stored across multiple computing sites. Data distribution introduces complex management issues, particularly as computing sites may make use of different storage systems with different internal organizations. The motivation for our work is the ATLAS Experiment for the Large Hadron Collider (LHC) at CERN, where the authors are involved in developing the data management middleware. This middleware, called DQ2, is charged with shipping petabytes of data every month to research centers and universities worldwide and has achieved aggregate throughputs in excess of 1.5 Gbytes/sec over the wide-area network. We describe DQ2's design and implementation, which builds upon previous work on distributed file systems, peer-to-peer systems and Data Grids. We discuss its fault tolerance and scalability properties and briefly describe results from its daily usage for the ATLAS Experiment.

show abstract

“…While this controls the overhead of replication, replicas may be found in places where peers do not access the files. In another work [4], the uniform and proportional replication strategies are described. In the uniform strategy, replications are uniformly distributed throughout the network (similar to random), while the proportional strategy replicates popular files more frequently.…”

Section: Related Workmentioning

confidence: 99%