GPU Passthrough Performance: A Comparison of KVM, Xen, VMWare ESXi, and LXC for CUDA and OpenCL Applications

Abstract

Abstract-As more scientific workloads are moved into the cloud, the need for high performance accelerators increases. Accelerators such as GPUs offer improvements in both performance and power efficiency over traditional multi-core processors; however, their use in the cloud has been limited. Today, several common hypervisors support GPU passthrough, but their performance has not been systematically characterized.In this paper we show that low overhead GPU passthrough is achievable across 4 major hypervisors and two processor microarchitectures. We compare the performance of two generations of NVIDIA GPUs within the Xen, VMWare ESXi, and KVM hypervisors, and we also compare the performance to that of Linux Containers (LXC). We show that GPU passthrough to KVM achieves 98-100% of the base system's performance across two architectures, while Xen and VMWare achieve 96-99% of the base systems performance, respectively. In addition, we describe several valuable lessons learned through our analysis and share the advantages and disadvantages of each hypervisor/GPU passthrough solution.

High Performance Computing and Network Function Virtualization: A major challenge towards network programmability

“…The next step was moving GPU's to the cloud within the "Infrastructure as a Service" framework. However, even if many research efforts have been spent so far, A mazon is probably the only known example of a ma jor OTT provider offering access to GPU-enabled services to its customers [13]. However, when moved to the NFV context, the problem of v irtualizing h/w accelerators must be addressed with a much wider scope: such resources, in fact, must be managed at all levels, fro m the offering of accelerated VNF's to the customers, to the lowest-level technology aspects at the infrastructure level.…”

“…In the virtualization context, the problem of virtualizing a GPU is now well-known, and can be stated as follows: a guest Virtual Machine (VM), running on a hardware platform provided with GPU-based accelerators, must be able to concurrently and independently access the GPU's, without incurring in security issues [13], [14]. Many techniques to achieve GPU virtualizat ion have been presented.…”

“…Many techniques to achieve GPU virtualizat ion have been presented. However, all the proposed methods can be divided in two main categories, which are usually referred to as API remoting [13] (also known as split driver model or driver paravirtualizat ion) and PCI pass -through [13] (also known as direct device assignment [14]), respectively.…”

“…The front-end driver is included in each VM. The backend driver runs in a privileged domain (like the Dom0 in Xen, [13]) and includes the GPU drivers and (in the case of NVIDIA devices) the CUDA library. Calls to the GPU library made within a VM are sent from the front-end to the back-end driver, which executes the CUDA API on the physical GPU on behalf of the VM.…”

Object Tracking In Images And Videos

“…Accessing one or more GPUs within a virtual machine is typically accomplished by one of two strategies: 1) via API remoting with device emulation; or 2) using PCI passthrough. We characterize GPGPU performance within virtual machines across two hardware systems, 4 hypervisors, and 3 application sets [6]. KVM again performs well across both the Delta and Bespin systems.…”