KRCORE: a microsecond-scale RDMA control plane for elastic computing

Abstract

This paper presents KRCORE, an RDMA library with a microsecond-scale control plane on commodity RDMAcapable network interface cards. KRCORE can establish a network connection to any node in the cluster within 10µs (hundreds or thousands of times faster than existing approaches), while only maintaining a (small) fixed-size connection metadata at each node, regardless of the cluster scale. The key ideas include virtualizing a pre-initialized kernel-space RDMA connection instead of creating one from scratch, and retrofitting advanced RDMA dynamic connected transport with static transport for both low connection overhead and high resource utilization. KRCORE can shorten the worker bootstrap time of existing disaggregated key-value store (namely RACE Hashing) by 83% under load spikes. In serverless computing (namely Fn), another popular elastic and ephemeral computing system, KRCORE can reduce the latency for transferring data through RDMA by 99%.