Lessons learned from moving earth system grid data sets over a 20 Gbps wide-area network

Abstract

In preparation for the Intergovernmental Panel on Climate Change (IPCC) Fifth Assessment Report, the climate community will run the Coupled Model Intercomparison Project phase 5 (CMIP-5) experiments, which are designed to answer crucial questions about future regional climate change and the results of carbon feedback for different mitigation scenarios. The CMIP-5 experiments will generate petabytes of data that must be replicated seamlessly, reliably, and quickly to hundreds of research teams around the globe. As an end-to-end test of the technologies that will be used to perform this task, a multidisciplinary team of researchers moved a small portion (10 TB) of the multimodel Coupled Model Intercomparison Project, Phase 3 data set used in the IPCC Fourth Assessment Report from three sources-the Argonne Leadership Computing Facility (ALCF), Lawrence Livermore National Laboratory (LLNL) and National Energy Research Scientific Computing Center (NERSC)-to the 2009 Supercomputing conference (SC09) show floor in Portland, Oregon, over circuits provided by DOE's ESnet. The team achieved a sustained data rate of 15 Gb/s on a 20 Gb/s network. More important, this effort provided critical feedback on how to deploy, tune, and monitor the middleware that will be used to replicate the upcoming petascale climate datasets. We report on obstacles overcome and the key lessons learned from this successful bandwidth challenge effort.

Optimizing Large Data Transfers over 100Gbps Wide Area Networks

“…The authors of [10] demonstrated transfer of Grid datasets on a 20 Gbps network during Super Computing 2009 Bandwidth Challenge. They achieved a sustained rate of 15 Gbps when moving 10 TB of data.…”

Instant GridFTP

“…Its eXtensible I/O interface [8] allows GridFTP to target high-performance wide-area communication protocols such as UDT [9] and emerging RDMAbased protocols [10]. Globus GridFTP is optimized to handle various types of datasets-from a single, huge file to datasets comprising lots of small files [11,12].…”

“…GridFTP provides significantly higher performance compared with these tools, through various optimizations such as parallelism [4], pipelining [11], concurrency [12], and striping [4]. It also supports third-party transfers in which a user or application at one site can initiate a data transfer operation between two other sites and the data flows directly between two remote sites.…”

Efficient Data Staging Using Performance-Based Adaptation and Policy-Based Resource Allocation

“…One important use case for scientific data transfer is data staging where a scientific data set is transferred or staged from the site where data are created or stored to a computational site where analysis will take place. Staging data for large computational jobs is a resource intensive activity found in many scientific applications and environments [5][6][7].…”