Parallel I/O subsystems in massively parallel supercomputers

Abstract

Applications executing on massively parallel processors (MPPs) often require a high aggregate bandwidth of lowlatency I/O to secondary storage. In many current MPPs, this requirement has been met by supplying internal parallel I/O subsystems that serve as staging areas for data. Typically, the parallel I/O subsystem is composed of I/O nodes that are linked to the same interconnection network that connects the compute nodes. The I/O nodes each manage their own set of disks. The option of increasing the number of I/O nodes together with the number of compute nodes and with the interconnection network allows for a balanced architecture. We explore the issues motivating the selection of this architecture for secondary storage in MPPs. We survey the designs of some recent and current parallel I/O subsystems, and discuss issues of system configuration, reliability, and file systems.

The impact of spatial layout of jobs on I/O hotspots in mesh networks

“…Although parallel I/O-based contention has been recognized as a potential problem [3,9,15], only a few studies have tried to minimize I/O-based contention through placement techniques. Cho et al [5] avoided data transfers on FDDIor Myrinet-connected network of workstations by changing the placement of part-time I/O nodes.…”

Evaluation of Collective I/O Implementations on Parallel Architectures

“…Also, a, variation of disk-directed IÂO is used in the Panda, runtime library [23]. Excellent overviews of the field of parallel IÂO can be found in [11,14].…”

High‐performance file I/O in Java: Existing approaches and bulk I/O extensions

“…Other than the large body of work related to parallel I/O [9][10][11][12][13][14][15][16][17], the work most closely related to ours is the Jaguar project [18,19], which aims to improve Java I/O performance as one of its goals. Jaguar allows the Java runtime system to be extended with new primitive operations that enable efficient access to hardware resources.…”