Virtual machine aware communication libraries for high performance computing

Abstract

As the size and complexity of modern computing systems keep increasing to meet the demanding requirements of High Performance Computing (HPC) applications, manageability is becoming a critical concern to achieve both high performance and high productivity computing. Meanwhile, virtual machine (VM) technologies have become popular in both industry and academia due to various features designed to ease system management and administration. While a VM-based environment can greatly help manageability on large-scale computing systems, concerns over performance have largely blocked the HPC community from embracing VM technologies.In this paper, we follow three steps to demonstrate the ability to achieve near-native performance in a VM-based environment for HPC. First, we propose Inter-VM Communication (IVC), a VM-aware communication library to support efficient shared memory communication among computing processes on the same physical host, even though they may be in different VMs. This is critical for multi-core systems, especially when individual computing processes are hosted on different VMs to achieve fine-grained control. Second, we design a VM-aware MPI library based on MVAPICH2 (a popular MPI library), called MVAPICH2-ivc, which allows HPC MPI applications to transparently benefit from IVC. Finally, we evaluate MVAPICH2-ivc on clusters featuring multi-core systems and high performance InfiniBand interconnects. Our evaluation demonstrates that MVAPICH2-ivc can improve NAS Parallel Benchmark performance by up to 11% in VM-based environment on eight-core Intel Clovertown systems, where each compute process is in a separate VM. A detailed performance evaluation for up to 128 processes (64 node dual-socket single-core systems) shows only a marginal performance overhead of MVAPICH2-ivc as compared with MVAPICH2 running in a native environment.

Prioritizing local inter-domain communication in Xen

“…Other research [6]- [9] concentrates on establishing a shared memory data channel between two guest domains via the Xen VMM. As a result, the time spent in the manager domain is avoided, and the performance is improved close to that of native socket communication.…”

“…As a result, the time spent in the manager domain is avoided, and the performance is improved close to that of native socket communication. IVC [6] provides a user level communication library for High Performance Computing applications. XWay [7] modifies the AF NET network protocol stack to switch dynamically between TCP/IP (using the original kernel) and their XWay channel (using shared memory).…”

Analyzing and Improving MPI Communication Performance in Overcommitted Virtualized Systems

“…The techniques proposed by these studies include VMM-bypassing [12], using XenLoop [13] alike techniques to replace the loopback network with the shared memory [14], and using a dedicated CPU core to handle all communications [15]. However, none of these studies addressed the overcommitted scenarios.…”

Efficient Shared Memory Message Passing for Inter-VM Communications

“…This approach has been studied using Xen page sharing capability to provide fast socket [10] and MPI [3] communication.…”

“…Some high performance Network Interface Cards can be made accessible directly from within VMs, thus providing near native communication performance when VMs are hosted on different physical machines [3]. However, implementing fast communication between virtual machines over shared memory is also crucial considering the emergence of multicore cluster nodes.…”