A Simulation Study of Data Distribution Strategies for Large-Scale Scientific Data Collaborations

Abstract

Abstract. Scientific instruments in fields as diverse as high-energy physics and genomics generate enormous volumes of data that need to be processed and analyzed by geographically dispersed communities. Such scientific collaborations require an efficient data dissemination technique. We analyze recent techniques proposed for peer-to-peer data distribution, select a small set of solutions representative for the various approaches currently proposed, and evaluate them, through simulation, in the context of data dissemination in scientific collaborations. This paper focuses on the performance and scalability of our simulator. Additionally, we include several preliminary recommendations for data optimal dissemination in data-intensive scientific collaborations derived form our preliminary simulation results.

Scheduling data-intensive bags of tasks in P2P grids with bittorrent-enabled data distribution

“…BitTorrent does not need an overlay to be constructed on top of Peers and, as such, is easy to deploy. A recent study [20] shows that BitTorrent performs nearly as well as overlay-based techniques in over-provisioned network cores, but also indicates that BitTorrent sustains "equivalent undegraded performance" when the available bandwidth decreases. In other words, BitTorrent has good performance in managed and over-provisioned networks and, as opposed to overlay-based techniques, maintains good performance in bandwidth-constrained networks that are more typical to P2P Grids environments.…”