FlexDriver: a network driver for your accelerator

Abstract

We propose a new system design for connecting hardware and FPGA accelerators to the network, allowing the accelerator to directly control commodity Network Interface Cards (NICs) without using the CPU. This enables us to solve the key challenge of leveraging existing NIC hardware offloads such as virtualization, tunneling, and RDMA for accelerator networking. Our approach supports a diverse set of use cases, from direct network access for disaggregated accelerators to inline-acceleration of the network stack, all without the complex networking logic in the accelerator.To demonstrate the feasibility of this approach, we build Flex-Driver (FLD), an on-accelerator hardware module that implements a NIC data-plane driver. Our main technical contribution is a mechanism that compresses the NIC control structures by two orders of magnitude, allowing FLD to achieve high networking scalability with low die area cost and no bandwidth interference with the accelerator logic.The prototype for NVIDIA Innova-2 FPGA SmartNICs showcases our design's utility for three different accelerators: a disaggregated LTE cipher, an IP-defragmentation inline accelerator, and an IoT cryptographic-token authentication offload. These accelerators reach 25 Gbps line rate and leverage the NIC for RDMA processing, VXLAN tunneling, and traffic shaping without CPU involvement. CCS CONCEPTS• Hardware → Hardware accelerators; Networking hardware; • Computer systems organization → Heterogeneous (hybrid) systems; Distributed architectures.

DiAD – Distributed Acceleration for Datacenter FPGAs

“…FlexDriver [25] is a hardware module for FPGAs, developed by NVIDIA. It provides a data-plane driver designed to communicate with a dedicated NIC using peer-to-peer PCIe, giving RDMA capabilities to accelerators without a full FPGA NIC offload.…”

“…This is attributed to the cost of traversing the PCIe interface (around 450ns per-hop [38]). This shows the benefits of our architecture in allowing direct control of the networking components from the FPGA fabric itself, as opposed to an architecture such as FlexDriver [25] in which NIC and FPGA fabric are separated by a PCIe hop, particularly for pointer-chasing workloads.…”

ORCA: A Network and Architecture Co-design for Offloading us-scale Datacenter Applications

“…ORCA can also be included to these co-design frameworks as another compute resource. Lynx [158] proposes Smart NIC-based communication offloading for accelerator-rich systems, and FlexDriver [39] proposes PCIe-based NIC control by accelerator. ORCA takes one step further to let the client directly communicate with the accelerators, which also controls the NIC more efficiently in the coherence domain.…”

Cooperative Concurrency Control for Write-Intensive Key-Value Workloads

“…Emerging SmartNIC frameworks propose a split-KVS design, where a small amount of NIC memory stores the KVS' hottest items [26,81], embodying the "small cache, big effect" principle [28]. Like NetCache [44], such designs focus on reads and are inefficient for write-intensive workloads, because writes are handled on a slow path to keep the CPU's main memory up-to-date.…”

“…System Software for RPCs. Although we implement C-4 over the NeBuLa architecture, our design could be implemented over a variety of baselines (e.g., NanoPU [39], FlexNIC [52], or NICA [26]). Due to the unique challenges of µs-scale RPCs that characterize KVS, several proposals pursue synchronization-free RPC load balancing [17,38,45,54,62,77,83,93].…”