Advanced Flow-control Mechanisms for the Sockets Direct Protocol over InfiniBand

Balaji, Pavan; Bhagvat, S.; Panda, Dhabaleswar K.; Thakur, Rajeev; Gropp, William

doi:10.1109/icpp.2007.14

Cited by 4 publications

(3 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use a version of Booksim [47] modified to cycle-accurately simulate a fat tree topology with bandwidth steering and Infiniband-style lossless credit-based flow control [15]. We compare our "steered" fat tree against a vanilla fat tree.…”

Section: System-scale Evaluation 71 Methodsmentioning

confidence: 99%

Bandwidth steering in HPC using silicon nanophotonics

Shen

Teh

Meng

et al. 2019

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

As bytes-per-FLOP ratios continue to decline, communication is becoming a bottleneck for performance scaling. This paper describes bandwidth steering in HPC using emerging reconfigurable silicon photonic switches. We demonstrate that placing photonics in the lower layers of a hierarchical topology efficiently changes the connectivity and consequently allows operators to recover from system fragmentation that is otherwise hard to mitigate using common task placement strategies. Bandwidth steering enables efficient utilization of the higher layers of the topology and reduces cost with no performance penalties. In our simulations with a few thousand network endpoints, bandwidth steering reduces static power consumption per unit throughput by 36% and dynamic power consumption by 14% compared to a reference fat tree topology. Such improvements magnify as we taper the bandwidth of the upper network layer. In our hardware testbed, bandwidth steering improves total application execution time by 69%, unaffected by bandwidth tapering.

show abstract

Section: System-scale Evaluation 71 Methodsmentioning

confidence: 99%

Bandwidth steering in HPC using silicon nanophotonics

Shen

Teh

Meng

et al. 2019

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

show abstract

“…This stack handles basic flow control and provides access to advanced hardware features, including RDMA, hardware flow control [2] and a hardware offloaded transport and network stack. Due to the RDMA features used by SDP, it can only operate over a reliable network, and so it utilizes the InfiniBand reliable connection mode for transmissions.…”

Section: A Sockets Direct Protocolmentioning

confidence: 99%

A study of hardware assisted IP over InfiniBand and its impact on enterprise data center performance

Grant

Balaji

Afsahi

2010

2010 IEEE International Symposium on Performance Analysis of Systems &Amp; Software (ISPASS)

View full text Add to dashboard Cite

Abstract-High-performance sockets implementations such as the Sockets Direct Protocol (SDP) have traditionally showed major performance advantages compared to the TCP/IP stack over InfiniBand (IPoIB). These stacks bypass the kernel-based TCP/IP and take advantage of network hardware features, providing enhanced performance. SDP has excellent performance but limited utility as only applications relying on the TCP/IP sockets API can use it and other IP stack uses (IPSec, UDP, SCTP) or TCP layer modifications (iSCSI) cannot benefit from it. Recently, newer generations of InfiniBand adapters, such as ConnectX from Mellanox, have provided hardware support for the IP stack itself, such as Large Send Offload and Large Receive Offload. As such high performance socket networks are likely to be deployed or converged with existing Ethernet networking solutions, the performance of such technologies is important to assess. In this paper we take a first look at the performance advantages provided by these offload techniques and compare them to SDP. Our micro-benchmarks and enterprise data-center experiments show that hardware assisted IPoIB can provide competitive performance with SDP and even outperform it in some cases.

show abstract

“…Wormhole routing [7,14,[19][20][21]25] is a popular routing strategy based on known fixed links. It is a subset of flow control methods called flit-buffer flow control [6,11,15,26]. Each message is transmitted as a contiguous sequence of flow control units.…”

Section: Introductionmentioning

confidence: 99%

Performance bounds in wormhole routing, a network calculus approach

Farhi¹,

Gaujal²

2010

Preprint

View full text Add to dashboard Cite

We present a model of performance bound calculus on feedforward networks where data packets are routed under wormhole routing discipline. We are interested in determining maximum end-to-end delays and backlogs of messages or packets going from a source node to a destination node, through a given virtual path in the network. Our objective here is to give a network calculus approach for calculating the performance bounds. First we propose a new concept of curves that we call packet curves. The curves permit to model constraints on packet lengths of a given data flow, when the lengths are allowed to be different. Second, we use this new concept to propose an approach for calculating residual services for data flows served under non preemptive service disciplines. Third, we model a binary switch (with two input ports and two output ports), where data is served under wormhole discipline. We present our approach for computing the residual services and deduce the worst case bounds for flows passing through a wormhole binary switch. Finally, we illustrate this approach in numerical examples, and show how to extend it to feedforward networks.

show abstract

Advanced Flow-control Mechanisms for the Sockets Direct Protocol over InfiniBand

Cited by 4 publications

References 8 publications

Bandwidth steering in HPC using silicon nanophotonics

Bandwidth steering in HPC using silicon nanophotonics

A study of hardware assisted IP over InfiniBand and its impact on enterprise data center performance

Performance bounds in wormhole routing, a network calculus approach

Contact Info

Product

Resources

About