Practical DCB for improved data center networks

Stephens, Brent; Cox, Alan L.; Singla, Ankit; Carter, John; Dixon, Colin; Felter, Wes

doi:10.1109/infocom.2014.6848121

Cited by 68 publications

(26 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some recent proposals aim to overcome these issues with PFC using ECN markings to maintain low queue occupancy. TCP-Bolt [14] uses modified DCTCP algorithm within the kernel TCP stack. DCQCN [48] uses a combination of ECN markings with a QCN-inspired rate-based congestion control algorithm implemented in the NIC.…”

Section: Related Workmentioning

confidence: 99%

Timely

Mittal

Lam

Dukkipati

et al. 2015

SIGCOMM Comput. Commun. Rev.

191

View full text Add to dashboard Cite

Datacenter transports aim to deliver low latency messaging together with high throughput. We show that simple packet delay, measured as round-trip times at hosts, is an effective congestion signal without the need for switch feedback. First, we show that advances in NIC hardware have made RTT measurement possible with microsecond accuracy, and that these RTTs are sufficient to estimate switch queueing. Then we describe how TIMELY can adjust transmission rates using RTT gradients to keep packet latency low while delivering high bandwidth. We implement our design in host software running over NICs with OS-bypass capabilities. We show using experiments with up to hundreds of machines on a Clos network topology that it provides excellent performance: turning on TIMELY for OS-bypass messaging over a fabric with PFC lowers 99 percentile tail latency by 9X while maintaining near line-rate throughput. Our system also outperforms DCTCP running in an optimized kernel, reducing tail latency by 13X. To the best of our knowledge, TIMELY is the first delay-based congestion control protocol for use in the datacenter, and it achieves its results despite having an order of magnitude fewer RTT signals (due to NIC offload) than earlier delay-based schemes such as Vegas.

show abstract

Section: Related Workmentioning

confidence: 99%

Timely

Mittal

Lam

Dukkipati

et al. 2015

SIGCOMM Comput. Commun. Rev.

191

View full text Add to dashboard Cite

show abstract

“…D 2 TCP [50] builds upon DCTCP to prioritize flows based on deadlines. TCP Bolt [49] uses flow-level congestion control via ECN to address PFC's limitations. ICTCP [51] iteratively adjusts the TCP receive window before incastinduced packet drops.…”

Section: Related Workmentioning

confidence: 99%

Dart: Divide and Specialize for Fast Response to Congestion in RDMA-Based Datacenter Networks

Xue

Chaudhry

Vamanan

et al. 2020

IEEE/ACM Trans. Networking

View full text Add to dashboard Cite

Though Remote Direct Memory Access (RDMA) promises to reduce datacenter network latencies significantly compared to TCP (e.g., 10x), end-to-end congestion control in the presence of incasts is a challenge. Targeting the full generality of the congestion problem, previous schemes rely on slow, iterative convergence to the appropriate sending rates (e.g., TIMELY takes 50 RTTs). Several papers have shown that even in oversubscribed datacenter networks most congestion occurs at the receiver. Accordingly, we propose a divide-and-specialize approach, called Dart, which isolates the common case of receiver congestion and further sub-divides the remaining in-network congestion into the simpler spatially-localized and the harder spatially-dispersed cases. For receiver congestion, we propose direct apportioning of sending rates (DASR) in which a receiver for n senders directs each sender to cut its rate by a factor of n, converging in only one RTT. For the spatially-localized case, Dart provides fast (under one RTT) response by adding novel switch hardware for in-order flow deflection (IOFD) because RDMA disallows packet reordering on which previous load balancing schemes rely. For the uncommon spatially-dispersed case, Dart falls back to DCQCN. Small-scale testbed measurements and at-scale simulations, respectively, show that Dart achieves 60% (2.5x) and 79% (4.8x) lower 99 th -percentile latency, and similar and 58% higher throughput than InfiniBand, and TIMELY and DCQCN.

show abstract

“…Recent work has shown that, replacing PFC with IRN helps DCQCN but does not remove the need for congestion control [22]. DCTCP [3] and TCP Bolt [32] are software sender-based congestion control schemes. These support selective retransmission but have higher CPU overhead and latency than RDMA hardware.…”

Section: Related Workmentioning

confidence: 99%

“…(iv) Our evaluation is limited to RDMA congestion control algorithms. We believe our hop-by-hop approach will compare favorably to TCP congestion control [3,32], but we do not present a quantitative comparison at this time.…”

Section: Introductionmentioning

confidence: 98%

Backpressure Flow Control

Goyal¹,

Shah

Sharma

et al. 2019

Proceedings of the 2019 Workshop on Buffer Sizing

View full text Add to dashboard Cite

Effective congestion control in a multi-tenant data center is becoming increasingly challenging with rapidly increasing workload demand, ever faster links, small average transfer sizes, extremely bursty traffic, limited switch buffer capacity, and one-way protocols such as RDMA. Existing deployed algorithms, such as DCQCN, are still far from optimal in many plausible scenarios, particularly for tail latency. Many operators compensate by running their networks at low average utilization, dramatically increasing costs.In this paper, we argue that we have reached the practical limits of end-to-end congestion control. Instead, we propose a new clean slate design based on hop-by-hop per-flow flow control. We show that our approach achieves near optimal tail latency behavior even under challenging conditions such as high average link utilization and in-cast cross traffic. By contrast with prior hop-by-hop schemes, our main innovation is to show that per-flow flow control can be achieved with limited metadata and packet buffering. Further, we show that our approach generalizes well to cross-data center communication.

show abstract

Practical DCB for improved data center networks

Cited by 68 publications

References 19 publications

Timely

Timely

Dart: Divide and Specialize for Fast Response to Congestion in RDMA-Based Datacenter Networks

Backpressure Flow Control

Contact Info

Product

Resources

About