No abstract

No abstract

In this paper, we present a comprehensive performance comparison of MPI implementations over InfiniBand, Myrinet and Quadrics. Our performance evaluation consists of two major parts. The first part consists of a set of MPI level micro-benchmarks that characterize different aspects of MPI implementations. The second part of the performance evaluation consists of application level benchmarks. We have used the NAS Parallel Benchmarks and the sweep3D benchmark. We not only present the overall performance results, but also relate application communication characteristics to the information we acquired from the micro-benchmarks. Our results show that the three MPI implementations all have their advantages and disadvantages. For our 8-node cluster, InfiniBand can offer significant performance improvements for a number of applications compared with Myrinet and Quadrics when using the PCI-X bus. Even with just the PCI bus, InfiniBand can still perform better if the applications are bandwidth-bound.

No abstract

No abstract

No abstract

No abstract

Recently, high speed interconnects capable of remote direct memory access (RDMA) such as InfiniBand and iWARP have gained considerable popularity due to their superb latency and bandwidth. Most existing studies about RDMA focus on its performance aspect. However, as power management has become essential for high-end systems such as enterprise servers and high performance computing nodes which are often equipped with RDMA capable network adapters, it is very important for us to take a fresh look at the benefits of RDMA from the power perspective.In the paper, we provide a detailed empirical study of the benefits of RDMA in terms of power savings compared with traditional communication protocols such as TCP/IP. We used two popular RDMA adapters in our evaluations: Mellanox ConnectX InfiniBand HCAs and Chelsio T3 10GE RNICs. In order to isolate the impact of communication on power consumption, our evaluation focused on using microbenchmarks which perform different communication patterns. We have also studied several important factors that may have an impact on the performance and the power consumption of RDMA adapters such as the use of polling versus blocking, CPU speeds, and extra memory copies.We show that using high speed RDMA adapters can result in significant amount of power consumption during communication. (In one test, the system power has increased by as much as 50 watts, or over 30% of the idle power.) We found that RDMA generally has better power efficiency compared to that of TCP/IP, especially for communication intensive phases, for example when large messages are transferred. The power savings of RDMA are achieved by minimizing the interactions between the network adapters and other system components such as the CPUs and the memory: Although nearly the same amount of data must be going through the network adapters for both RDMA and TCP/IP, RDMA rePermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. quires much fewer CPU cycles for protocol processing and also generates less memory bus traffic, both of which contribute to its power savings.Overall, our research demonstrated that RDMA not only provides high communication performance, but also offers excellent power efficiency, making it a desirable choice in environments that have strict power/energy constraints and demand high communication performance.