First experiences with congestion control in InfiniBand hardware

Gran, Ernst Gunnar; Eimot, Magne; Reinemo, Sven-Arne; Skeie, Tor; Lysne, Olav; Huse, Lars Paul; Shainer, Gilad

doi:10.1109/ipdps.2010.5470419

Cited by 49 publications

(35 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gran et al studied the performance of IBA ECN in hardware on an InfiniScale IV router [13] and confirmed the effectiveness of ECN in hardware using synthetic traffic patterns and HPC benchmarks. Unfortunately, the hardware study was limited to seven nodes and two routers, in a configuration similar to the simple scenario we presented in Section 2.…”

Section: Related Workmentioning

confidence: 88%

“…To regulate the sender transmission rate, in the case of Infiniband, an inter-packet delay is added between successive packet transmissions to the same destination. With proper configuration, ECN has been shown to be effective in combating network congestion for long traffic flows [13]. However, due to the incremental nature of the algorithm, the reliance on buffer thresholds, and the round trip time of congestion information, ECN can have a slow response to the onset of congestion [10].…”

Section: Motivationmentioning

confidence: 99%

“…The congestion signal is sent via an explicit message or piggybacked on acknowledgment packets from the destination. ECN has been well studied in the context of system area networks, particularly the InfiniBand Architecture (IBA) [11,13,23], and has shown to be effective in combating congestion. However, studies have also pointed out limitations such as reduced system stability, the need for parameter adjustment, and slow congestion response time [10,20].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Network congestion avoidance through Speculative Reservation

Jiang

Becker

Michelogiannakis

et al. 2012

IEEE International Symposium on High-Performance Comp Architecture

View full text Add to dashboard Cite

Congestion caused by hot-spot traffic can significantly degrade the performance of a computer network. In this study, we present the Speculative Reservation Protocol (SRP), a new network congestion control mechanism that relieves the effect of hot-spot traffic in high bandwidth, low latency, lossless computer networks. Compared to existing congestion control approaches like Explicit Congestion Notification (ECN), which react to network congestion through packet marking and rate throttling, SRP takes a proactive approach of congestion avoidance. Using a light-weight endpoint reservation scheme and speculative packet transmission, SRP avoids hot-spot congestion while incurring minimal overhead. Our simulation results show that SRP responds more rapidly to the onset of severe hot-spots than ECN and has a higher network throughput on bursty network traffic. SRP also performs comparably to networks without congestion control on benign traffic patterns by reducing the latency and throughput overhead commonly associated with reservation protocols.

show abstract

Section: Related Workmentioning

confidence: 88%

Section: Motivationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Network congestion avoidance through Speculative Reservation

Jiang

Becker

Michelogiannakis

et al. 2012

IEEE International Symposium on High-Performance Comp Architecture

View full text Add to dashboard Cite

show abstract

“…The differences rely on how they detect network congestion. Several proposals rely on explicit congestion notification (ECN), such as QCN, Datacenter TCP [13] or recent Infiniband implementations [14]. In such cases, the network equipment detects congestion based on the occupancy of router queues.…”

Section: Related Workmentioning

confidence: 99%

OFAR-CM: Efficient Dragonfly Networks with Simple Congestion Management

García

Vallejo

Beivide

et al. 2013

2013 IEEE 21st Annual Symposium on High-Performance Interconnects

View full text Add to dashboard Cite

Abstract-Dragonfly networks are appealing topologies for large-scale Datacenter and HPC networks, that provide high throughput with low diameter and moderate cost. However, they are prone to congestion under certain frequent traffic patterns that saturate specific network links. Adaptive nonminimal routing can be used to avoid such congestion. That kind of routing employs longer paths to circumvent local or global congested links. However, if a distance-based deadlock avoidance mechanism is employed, more Virtual Channels (VCs) are required, what increases design complexity and cost.OFAR (On-the-Fly Adaptive Routing) is a previously proposed routing that decouples VCs from deadlock avoidance, making local and global misrouting affordable. However, the severity of congestion with OFAR is higher, as it relies on an escape subnetwork with low bisection bandwidth. Additionally, OFAR allows for unlimited misroutings on the escape subnetwork, leading to unbounded paths in the network and long latencies.In this paper we propose and evaluate OFAR-CM, a variant of OFAR combined with a simple congestion management (CM) mechanism which only relies on local information, specifically the credit count of the output ports in the local router. With simple escape subnetworks such as a Hamiltonian ring or a tree, OFAR outperforms former proposals with distance-based deadlock avoidance. Additionally, although long paths are allowed in theory, in practice packets arrive at their destination in a small number of hops. Altogether, OFAR-CM constitutes the first practicable mechanism to the date that supports both local and global misrouting in Dragonfly networks.

show abstract

“…ECN detects congestion at the root of the congestion tree and signals to the contributing sources to throttle down via a control packet or by piggybacking onto acknowledgment packets. While ECN can be effective [37,21], it is also considerably sensitive to its configuration parameters which depend on numerous and sometimes unpredictable factors, including the traffic pattern [37]. Past work has also highlighted ECN's slow adaptation to changes in the traffic pattern [15,28].…”

Section: Introductionmentioning

confidence: 99%

Channel reservation protocol for over-subscribed channels and destinations

Jiang

Becker

Dally

2013

Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

Channels in system-wide networks tend to be over-subscribed due to the cost of bandwidth and increasing traffic demands. To make matters worse, workloads can overstress specific destinations, creating hotspots. Lossless networks offer attractive advantages compared to lossy networks but suffer from tree saturation. This led to the development of explicit congestion notification (ECN). However, ECN is very sensitive to its configuration parameters and acts only after congestion forms. We propose channel reservation protocol (CRP) to enable sources to reserve bandwidth in multiple resources in advance of packet transmission and with a single request, but without idling resources like circuit switching. CRP prevents congestion from ever occurring and thus reacts instantly to traffic changes, whereas ECN requires 300,000 cycles to stabilize in our experiments. Furthermore, ECN may not prevent congestion formed by short-lived flows generated by a large combination of source-destination pairs.

show abstract

First experiences with congestion control in InfiniBand hardware

Cited by 49 publications

References 12 publications

Network congestion avoidance through Speculative Reservation

Network congestion avoidance through Speculative Reservation

OFAR-CM: Efficient Dragonfly Networks with Simple Congestion Management

Channel reservation protocol for over-subscribed channels and destinations

Contact Info

Product

Resources

About