Lightweight and Generic RDMA Engine Para-Virtualization for the KVM Hypervisor

Abstract

Remote DMA (RDMA) engines are widely used in clusters/data-centres to improve the performance of data transfers between applications running on different nodes of a computing system. RDMAs are today supported by most network architectures and distributed programming models. However, with the massive usage of virtualization most applications will use RDMAs from virtual machines, and the virtualization of such I/O devices poses several challenges. This paper describes a generic para-virtualization framework based on API Remoting, providing at the same time the flexibility of software based virtualization, and the low overhead of hardware-assisted solutions.The solution presented in this paper is targeting the KVM hypervisor, but is not bound to any target network architecture or specific RDMA engine, thanks to the virtualization at the level of the programming API. In addition, two of the major limitations of para-virtualization are addressed: data sharing between host and guest, and interactions between guests and hypervisor. A set of experimental results showed a near to native performance for the final user of the RDMA (i.e., maximum transfer bandwidth), with a higher overhead only to simulate the API functions used to initialize the RDMA device or allocate/deallocate RDMA buffers.

Empowering Cloud Computing With Network Acceleration: A Survey

“…This Section describes three main techniques that have been proposed to fulfill these requirements, which we visually represent in Fig. 9: new forms of hardware offload (AccelNet [81]), a more efficient paravirtualization technique (virtio-PMD [122], FreeFlow [56], KuberneTSN [59], VSocket [57]), and hybrid solutions that combine the advantages of both (virtio-rdma [123], HyV [53], Mouzakitis et al [124], MasQ [54], SocksDirect [58]). We summarize the key insights of these proposals in Table IV: we underline the main optimizations each work introduces over standard techniques for an efficient integration of hardware-accelerated networking into the existing infrastructures.…”

“…3) Hybrid virtualization: A number of works have introduced the concept of hybrid virtualization to balance the flexibility of the paravirtualization mechanism with the performance efficiency of the hardware-based option [53], [54], [58], [123], [124]. The goal of these approaches is a complete separation between the data and control planes, in order to preserve the flexibility of a software control path, typical of paravirtualization techniques, while also leveraging the efficiency of a bare-metal access to the NIC, typical of hardwareassisted virtualization techniques.…”

Interdomain I/O Optimization in Virtualized Sensor Networks

“…In the area of high-performance computing, DMA (Direct Memory Access)-capable interconnections provide ultralow latency and high bandwidth in distributed storage and data-processing systems. However, it is difficult to deploy such systems in virtualized data centers due to a lack of flexible and high-performance virtualization solutions for RDMA (Remote Direct Memory Access) network interfaces [64]. Hybrid virtualization (HyV) [28] was proposed to separate paths for control and data operations available in RDMA.…”