Efficient User-Level Multi-Path Utilization in RDMA Networks

Abstract

RDMA has become one of the most prominent networking technologies in DCNs by providing high bandwidth and ultra-low latency, especially for data-intensive applications. An important challenge with RDMA is to exploit multi-path for high throughput and reliability. Several studies have been proposed to utilize multi-path in RDMA networks, but they commonly require modification of RDMA NICs, which makes it hard to deploy them in practice. In this paper, we propose a user-level multi-path RDMA (UL-MPRDMA) scheme, in which a flow is partitioned into sub-flows, and transferred via multiple connections to make full use of multiple paths in DCNs. UL-MPRDMA quickly responds to sudden network trouble and congestion by performing dynamic sub-flow scheduling, and also effectively avoids the performance degradation problem due to the limited memory of RDMA NICs without the intervention of CPU. We implement UL-MPRDMA on a real test-bed with commercial RDMA NICs, and show that UL-MPRDMA can achieve 30% higher link utilization than an existing RDMA transport technique.

An Optimized RDMA QP Communication Mechanism for Hyperscale AI Infrastructure

“…While these modifications enhance QP scalability, the high cost and potential impact on NIC reliability make them impractical for deployment in hyper scale AI infrastructure. Additionally, congestion control algorithms like DCQCN and TIMELY, used in these studies, cannot completely prevent collisions of multiple large flows at specific nodes (usually refers to servers or switches) in extremely large-scale AI training networks [6,9,11,18], leading to congestion in network traffic at those points.…”