Remote Direct Memory Access (RDMA) Protocol Extensions

Sharp, Robert; Shah, Hemal; Martí, Fèlix; Eiriksson, A.; Noureddine, Waël

doi:10.17487/rfc7306

Cited by 9 publications

(6 citation statements)

References 3 publications

(3 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As the RDMA protocol [30,33] is a significant departure from traditional transport methods, its design takes a whole stack approach. RDMA is for the most part managed at the network interface card (NIC), which sits on the PCIe bus.…”

Section: Background 21 Rdmamentioning

confidence: 99%

Acuerdo: Fast Atomic Broadcast over RDMA

Izraelevitz

Wang

Hanscom

et al. 2022

Proceedings of the 51st International Conference on Parallel Processing

View full text Add to dashboard Cite

Atomic broadcast protocols ensure that messages are delivered to a group of machines in some total order, even when some of these machines can fail. These protocols are key to making distributed services fault-tolerant, as their total order guarantee allows keeping multiple service replicas in sync. But, unfortunately, atomic broadcast protocols are also notoriously expensive.We present a new protocol, called Acuerdo, that improves atomic broadcast performance by using remote direct memory addressing (RDMA). Acuerdo is built from the ground up to perform communication using one-side RDMA writes, which do not use the CPU of the remote machine, and is explicitly designed to minimize waiting on the critical path. Our experimental results demonstrate that Acuerdo provides raw throughput comparable to or exceeding other RDMA atomic broadcast protocols, while improving latency by almost 2𝑥. CCS CONCEPTS• Computer systems organization → Dependable and faulttolerant systems and networks; • Networks → Network algorithms.

show abstract

Section: Background 21 Rdmamentioning

confidence: 99%

Acuerdo: Fast Atomic Broadcast over RDMA

Izraelevitz

Wang

Hanscom

et al. 2022

Proceedings of the 51st International Conference on Parallel Processing

View full text Add to dashboard Cite

show abstract

“…Operations: The RDMA programming model [23,31,34] contains a QPAIR abstraction that is used to represent an endpoint of a communication channel (connection) between computers (nodes) over and RDMA network. A connection management system establishes a connection between QPAIRs of communicating nodes.…”

Section: Rdmamentioning

confidence: 99%

“…RDMA WRITEIMM does require the responder's CPU to process the message with immediate data, like RDMA SEND. RDMA also contains one-sided ATOMIC operations (RDMA Fetch-And-Add (FAA) and RDMA Compare-And-Swap (CAS)) -64-bit wide atomic read-modify-write operations that can be used for synchronization between remote requesters [34,41], but can incur significant performance overheads [19].…”

Section: Rdmamentioning

confidence: 99%

“…The RDMA standard specifies interesting ordering rules for RDMA operations [8,31,34]. In particular, it separates RDMA operations into two categories: (i) operations that produce a return value that is consumed by the requester (RDMA READ, CAS, FAA, and now, FLUSH), informally referred to as "non-posted" operations in the community [10,28]; and (ii) operations that do not require a return value (RDMA SEND, WRITE, and WRITEIMM), also referred to as "posted" operations.…”

Section: Rdma Operation Orderingmentioning

confidence: 99%

“…For MHP, visibility of RDMA updates at the responder is equivalent to persistence, as long as the updates are directed to the remote PM. Existing RDMA ordering semantics [8,31,34] guarantee in-order visibility of consecutive posted updates. As a result, two dependent updates can be pipelined back-to-back as RDMA WRITEs.…”

Section: Mhpmentioning

confidence: 99%

See 2 more Smart Citations

Correct, Fast Remote Persistence

Kashyap,

Qin,

Byan

et al. 2019

Preprint

View full text Add to dashboard Cite

Persistence of updates to remote byte-addressable persistent memory (PM), using RDMA operations (RDMA updates), is a poorly understood subject. Visibility of RDMA updates on the remote server is not the same as persistence of those updates. The remote server's configuration has significant implications on what it means for RDMA updates to be persistent on the remote server. This leads to significant implications on methods needed to correctly persist those updates. This paper presents a comprehensive taxonomy of system configurations and the corresponding methods to ensure correct remote persistence of RDMA updates. We show that the methods for correct, fast remote persistence vary dramatically, with corresponding performance trade offs, between different remote server configurations.

show abstract