An Implementation of MPI-IO on Expand: A Parallel File System Based on NFS Servers

Calderón, Alejandro; García, Félix; Carretero, Jesús; Perez, J.M.; Fernández, Julia Martínez

doi:10.1007/3-540-45825-5_47

Cited by 13 publications

(6 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MPI-IO has been used over GPFS [22], Lustre [23], NFS [24], PVFS2 [25] and OrangeFS [26]. MPI-IO is used at filesystem-level; that is, on top of an existing storage system.…”

Section: Related Workmentioning

confidence: 99%

Extreme-scale computing services over MPI: Experiences, observations and features proposal for next-generation message passing interface

Zounmevo

Kimpe

Ross

et al. 2014

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

The Message Passing Interface (MPI) is one of the most portable high-performance computing (HPC) programming models, with platform-optimized implementations typically delivered with new HPC systems. Therefore, for distributed services requiring portable, high-performance, user-level network access, MPI promises to be an attractive alternative to custom network portability layers, platform-specific methods, or portable but less performant interfaces such as BSD sockets. In this paper, we present our experiences in using MPI as a network transport for a large-scale distributed storage system. We discuss the features of MPI that facilitate adoption as well as aspects which require various workarounds. Based on use cases, we derive a wish-list for both MPI implementations and the MPI forum to facilitate the adoption of MPI large-scale persistent services. The proposals in the wish-list go beyond the sole needs of distributed services; we contend that they will benefit mainstream HPC applications at extreme scales as well.

show abstract

“…MPI-IO has been used over GPFS [22], Lustre [23], NFS [24], PVFS2 [25] and OrangeFS [26]. MPI-IO is used at filesystem-level; that is, on top of an existing storage system.…”

Section: Related Workmentioning

confidence: 99%

Extreme-scale computing services over MPI: Experiences, observations and features proposal for next-generation message passing interface

Zounmevo

Kimpe

Ross

et al. 2014

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

show abstract

“…Some other solutions [4,5,6,7,8] are to make use of multiple NFS servers to provide high NFS I/O performance. However, the management of such environment is complicated and the risk of losing data increases greatly when the number of NFS servers scales larger.…”

Section: Introductionmentioning

confidence: 99%

Research on Key Technologies of Load Balancing for NFS Server with Multiple Network Paths

Chen

Tang

Zhao

et al. 2006

2006 Fifth International Conference on Grid and Cooperative Computing Workshops

View full text Add to dashboard Cite

NFS server is designed to run on a single node. Even if NFS server is configured with multiple network interfaces, each client can only access NFS server through one network interface of the server. In this paper, we design and implement the Multi-path load distribution mechanism in the SunRPC layer of NFS * . This mechanism allows the NFS clients to use the aggregated network bandwidth of the server which has multiple network channels and balances the load among the multiple network channels. The result shows that the extended version of NFS can make good use of multi-path of the server and can tolerate network failure.

show abstract

“…This approach has been implemented in the Expand Parallel File System, but can be applied to other parallel file systems with similar architecture. Expand [11,12] is a parallel file system for heterogeneous clusters that uses standard servers as basis (NFS, FTP, etc.) and provides POSIX and MPI-IO [13] interfaces.…”

Section: Introductionmentioning

confidence: 99%

Fault tolerant file models for parallel file systems: introducing distribution patterns for every file

et al. 2008

View full text Add to dashboard Cite

Parallelism in file systems is obtained by using several independent server nodes supporting one or more secondary storage devices. This approach increases the performance and scalability of the system, but a fault in one single node can stop the whole system. To avoid this problem, data must be stored using some kind of redundant technique, so any data stored in a faulty element can be recovered. Fault tolerance can be provided in I/O systems by using replication or RAID based schemes. However, most of the current systems apply the same technique for all files in the system. This paper describes the fault tolerance support provided by Expand, a parallel file system based on standard servers. This support can be applied to other parallel file systems with many benefices: fault tolerance at file level, flexible definition of fault tolerance scheme to be used, possibility to change the fault tolerant support used for a file, etc.

show abstract

An Implementation of MPI-IO on Expand: A Parallel File System Based on NFS Servers

Cited by 13 publications

References 7 publications

Extreme-scale computing services over MPI: Experiences, observations and features proposal for next-generation message passing interface

Extreme-scale computing services over MPI: Experiences, observations and features proposal for next-generation message passing interface

Research on Key Technologies of Load Balancing for NFS Server with Multiple Network Paths

Fault tolerant file models for parallel file systems: introducing distribution patterns for every file

Contact Info

Product

Resources

About