High Performance Relay Mechanism for MPI Communication Libraries Run on Multiple Private IP Address Clusters

Takano, Ryousei; Matsuda, Motohiko; Kudoh, Tomohiro; Kodama, Y.; Okazaki, Fumihiro; Ishikawa, Yutaka; Yoshizawa, Yasufumi

doi:10.1109/ccgrid.2008.88

Cited by 5 publications

(2 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We show in this paper that it is also useful for wired connections, mostly in the context of MPI application traffic features and with RTT values less than satellite's ones. Several past propositions [11] [12] [13] introduce proxies at the LAN/WAN interfaces to improve performance but the detailed propositions, the proxy level or usage are different from us. Even if we focalized on MPI applications, our proposition is not specific to MPI applications because it is at the socket level and thus, it would apply to any grid application executed above TCP.…”

Section: Related Workmentioning

confidence: 99%

Splitting TCP for MPI Applications Executed on Grids

Glück

Mignot

2011

2011 IEEE Ninth International Symposium on Parallel and Distributed Processing With Applications

View full text Add to dashboard Cite

In this paper, we first study the interaction between MPI applications and TCP on grids. Then, we propose MPI5000, a transparent applicative layer between MPI and TCP, using proxies to improve the execution of MPI applications on grids. Proxies aim at splitting TCP connections in order to detect losses faster and avoid to return in a slowstart phase after an idle time. Finally, we evaluate our layer executing the NAS Parallel Benchmarks on Grid'5000, the French research grid. The results show that our architecture reduces the number of idle timeout and of long-distance retransmissions for BT, SP and LU benchmarks. Using MPI5000, these applications can decrease their execution time by 35%, 28%, and, 15% respectively. A comparison with MPICH-G2 performances shows that our layer can even outperform a grid enabled MPI implementation.

show abstract

Section: Related Workmentioning

confidence: 99%

Splitting TCP for MPI Applications Executed on Grids

Glück

Mignot

2011

2011 IEEE Ninth International Symposium on Parallel and Distributed Processing With Applications

View full text Add to dashboard Cite

show abstract

“…LAM/MPI 5) , HP-MPI 11) , and GridMPI 16) implement IMPI. Among these MPI implementations, the aggregate router method is similar to metaMPICH and GridMPI 25) in that the router method is adopted and multiple router nodes can be used in each cluster. The difference is that multiple routers are used only for static load balancing with metaMPICH and GridMPI.…”

Section: Related Workmentioning

confidence: 99%

Aggregate Router: An Efficient Inter-Cluster MPI Communication Facility

Matsuba

Ishikawa

2009

IPSJ Online Transactions

Self Cite

View full text Add to dashboard Cite

At a cluster of clusters used for parallel computing, it is important to fully utilize the inter-cluster network. Existing MPI implementations for cluster of clusters have two issues: 1) Single point-to-point communication cannot utilize the bandwidth of the high-bandwidth inter-cluster network because a Gigabit Ethernet interface is used at each node for inter-cluster communication, while more bandwidth is available between clusters. 2) Heavy packet loss and performance degradation occur on the TCP/IP protocol when many nodes generate short-term burst traffic. In order to overcome these issues, this paper proposes a novel method called the aggregate router method. In this method, multiple router nodes are set up in each cluster and inter-cluster communication is performed via these router nodes. By striping a single message to multiple routers, the bottleneck caused by network interfaces is reduced. The packet congestion issue is also avoided by using high-speed interconnects in a cluster, instead of the TCP/IP protocol. The aggregated router method is evaluated using the HPC Challenge Benchmarks and the NAS Parallel Benchmarks. The result shows that the proposed method outperforms the existing method by 24% in the best case.

show abstract