Remote I/O

Foster, Ian; Kohr, David R.; Krishnaiyer, Rakesh; Mogill, Jace

doi:10.1145/266220.266222

Cited by 58 publications

(2 citation statements)

References 21 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The user-level SIOS is implemented with I/O service routines or libraries. The Parallel Virtual File System [6], and the remote I/O project [11] are typical examples of user-level SIOS. This approach has two shortcomings: First, users have to apply specific APIs and identifiers to explore parallelism in I/O.…”

Section: Single I/o Spacementioning

confidence: 99%

Orthogonal striping and mirroring in distributed RAID for I/O-centric cluster computing

Hwang

Jin

2002

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

AbstractÐThis paper presents a new distributed disk-array architecture for achieving high I/O performance in scalable cluster computing. In a serverless cluster of computers, all distributed local disks can be integrated as a distributed-software redundant array of independent disks (ds-RAID) with a single I/O space. We report the new RAID-x design and its benchmark performance results. The advantage of RAID-x comes mainly from its orthogonal striping and mirroring (OSM) architecture. The bandwidth is enhanced with distributed striping across local and remote disks, while the reliability comes from orthogonal mirroring on local disks at the background. Our RAID-x design is experimentally compared with the RAID-5, RAID-10, and chained-declustering RAID through benchmarking on a research Linux cluster at USC. Andrew and Bonnie benchmark results are reported on all four disk-array architectures. Cooperative disk drivers and Linux extensions are developed to enable not only the single I/O space, but also the shared virtual memory and global file hierarchy. We reveal the effects of traffic rate and stripe unit size on I/O performance. Through scalability and overhead analysis, we find the strength of RAID-x in three areas: 1) improved aggregate I/O bandwidth especially for parallel writes, 2) orthogonal mirroring with low software overhead, and 3) enhanced scalability in cluster I/O processing. Architectural strengths and weakness of all four ds-RAID architectures are evaluated comparatively. The optimal choice among them depends on parallel read/write performance desired, the level of fault tolerance required, and the cost-effectiveness in specific I/O processing applications.

show abstract

Section: Single I/o Spacementioning

confidence: 99%

Orthogonal striping and mirroring in distributed RAID for I/O-centric cluster computing

Hwang

Jin

2002

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

show abstract

“…Parallel I/O systems commonly treat access to distributed data as a collective operation [49], and collective communication operations seek to optimize data transformations and transfers by coordinating related activities [35,38]. In two-phase I/O [48] and in Remote I/O [25], data is read and then reorganized via interprocess communication prior to transfer. HPF/MPI [21] used the FALLS (FAmiLy of Line Segments) representation [47] to compute efficient inter-cluster communication schedules.…”

Section: Related Workmentioning

confidence: 99%

The Globus Striped GridFTP Framework and Server

Allcock

Bresnahan

Kettimuthu

et al.

ACM/IEEE SC 2005 Conference (SC'05)

453

313

View full text Add to dashboard Cite

show abstract

Global‐scale distributed I/O with ParaMEDIC

Balaji

Feng

Lin

et al. 2010

Concurrency and Computation

View full text Add to dashboard Cite

SUMMARYAchieving high performance for distributed I/O on a wide-area network continues to be an elusive holy grail. Despite enhancements in network hardware as well as software stacks, achieving high-performance remains a challenge. In this paper, our worldwide team took a completely new and non-traditional approach to distributed I/O, called ParaMEDIC: Parallel Metadata Environment for Distributed I/O and Computing, by utilizing application-specific transformation of data to orders of magnitude smaller metadata before performing the actual I/O. Specifically, this paper details our experiences in deploying a largescale system to facilitate the discovery of missing genes and constructing a genome similarity tree by encapsulating the mpiBLAST sequence-search algorithm into ParaMEDIC. The overall project involved nine computational sites spread across the U.S. and generated more than a petabyte of data that was 'teleported' to a large-scale facility in Tokyo for storage.

show abstract

Remote I/O

Abstract: As high-speed networks make it easier to use distributed resources, it becomes increasingly common that applications and their data are not colocated.

Cited by 58 publications

References 21 publications

Orthogonal striping and mirroring in distributed RAID for I/O-centric cluster computing

Orthogonal striping and mirroring in distributed RAID for I/O-centric cluster computing

The Globus Striped GridFTP Framework and Server

Global‐scale distributed I/O with ParaMEDIC

Contact Info

Product

Resources

About